The Tao of Testing | Dan Widing from ProdPerfect

DISCLAIMER: Below is an AI generated transcript. There could be a few typos but it should be at least 90% accurate. Watch video or listen to the podcast for the full experience!

Dan Widing 0:00
You will have to take a risk based prioritization approach to what should become an untimed test.

Alexander Ferguson 0:13
Welcome to UpTech Report. This is our applied tech series UpTech Report is sponsored by TeraLeap. Learn how to leverage the power of video at Teraleap.io. Today, I’m excited to be joined by my guest. Dan Widing is based in the Bay Area. He’s the Co founder and CEO at ProdPerfect. Welcome, Dan. Good to have you on. Thank you, Alexander, it’s good to be here. Now ProdPerfect automates the testing of web applications for quality assurance. Or on your website. If you dig a little bit further, a toss of bills maintains evolves and regression testing Suites by analyzing live user traffic on your application. However you look at it, how many understand though that what was the problem that you initially saw, and and set out to solve with perfect.

Dan Widing 0:54
So this was the start, we are looking on you There are many, many layers to what we do. And to go all the way to the top layer of the onion again, I suppose I’d really start with software engineering as a whole, it’s kind of chaos. There’s a big industry turnover constantly and for new folks. Little understanding of practices, it becomes kind of an eternal September continuously. And you know, there are folks that really know what they’re doing. But they’re kind of like the Zen masters. And some of them have the tidbits of wisdom that are like just perfect Cohen’s some kind of look like Zen masters and just be children with sex. And that kind of like frustration in the field has driven me to do a lot of things differently. But one of the most infuriating things is sort of because of all this chaos, a developer really has to put

Unknown Speaker 1:56
at least half

Dan Widing 1:56
their time into making sure that what they’re doing is building good code. It’s actually moving the ball forward.

Alexander Ferguson 2:04
Yeah, it takes things back. See, you’ve been in the software development world for a long time, you saw the issues. And you’re saying, basically, you see the turnover, you see the issues? And does everyone really know what’s going on. And some people seem to know everything and Zen master can understand. But in many cases, it’s just, it’s difficult to maintain a good team that can can do this. And so your focus is to come in and help the workflow, particularly when it comes to the testing.

Dan Widing 2:31
Right? Yeah, there’s kind of, there’s a spectrum of different tools, a spectrum of different practices, each of which help figure out if you’re building the right code in different ways. And sizes, groups represented as a testing pyramid, the one that people frequently put at the top, the one that’s most difficult to slowest to write most expensive, most painful, bla bla bla, bla, bla, and antastic testing something the way an end user would, through a full experience of the entire system under test. Because of how complicated that is, I, it’s where a lot of teams testing practices really start to break down in the pot.

Alexander Ferguson 3:22
There’s you mentioned, there’s there’s a lot of ways one could approach this, this testing. And and there’s even solutions out there, where you guys are focused on is performance testing, regression testing, load testing, is I get that right. I actually just I, let’s see end to end regression testing is our core, gotcha end to end regression testing, and where How were you kind of approaching it differently? or How is your approach? And why would someone want to say, Okay, perfect, I get it, and now I see the value.

Dan Widing 3:50
So one of the things that throughout the Zen master skit, but nobody, like every head struggles to apply is you have to take a risk based prioritization approach to what should become an intern test. If you try to test everything, or if you let just subjective opinions drive, what gets tested,

you end up just constantly building a morass, building a like, kind of a ball of scraps of testing, as opposed to something coherent. So the first big thing we do is figure out, hey, what should be an intent test in the first place?

Alexander Ferguson 4:37
It gets a lot of can take a lot of time itself. Okay, what should we test what should we focus on because if you’re going to do it, it takes time and energy. So knowing where it should be testing is a good use of the time Good, good energy to focus on.

Dan Widing 4:52
It’s to be honest though, it’s just the first of the layers. It’s just the starting point. Because you can’t there’s enough complication in what should be tested that in order to really figure out, have you tested it, you have to have a practice around legends, figuring out what to test, but making those tests of reality making sure that they’re continuously useful. And then as we stayed evolving them over time.

Alexander Ferguson 5:29
Are you guys 2018 began incorporating started? Can you can you walk me through just like the development and where the product is today like someone to sign up? But what would they be getting? How does it work?

Dan Widing 5:44
So let’s see, by Initially, I suppose we were noodling the idea. I know for me, November, October range of 2017. It was actually my co founder and I, I, we went to Poland to go and check out tech hubs in Eastern Europe, just sort of, on a lark, I was thinking I would move to the Bay Area and join some company. But in the process, we had a meeting all these guys sort of grinding out startups, and kind of got inspired and started pitching a startup idea round that we got traction

Unknown Speaker 6:27
on

Dan Widing 6:28
with your just words, like just talking to people, like people got excited about this thing. But we didn’t have anything, we’re just talking about effect. That’s what we came back to the states incorporate in January, initially, to your point, we were looking at performance testing as being sort of more niche easier to break into, but had a number of conversations with folks where we were pitching them an idea, be like, Oh, yeah, we do this. And we do this. And we do this, and then we analyze it based upon how much should be sent ads, then get a load test for you guys. And a couple of conversations, I’d like well Shut up, you’re going way too far. If you just stopped halfway through that, and just automatically build and tests and kept them maintained, we would pay a ton of money for that. And that was the early ish, 2018 moment where we really realized that we had something, there’s no, no one that was going and doing the full spectrum. And there are a few even tools that people were sufficiently satisfied with, or even understood sufficiently to do the job. Let’s see, after that, we actually got a couple of customers based on a very consultative product for that very early on, we were able to MVP the crap out of this. And it was a lot of man behind the curtain. Had you know the phrase,

Alexander Ferguson 7:57
yeah, you’re there. Yeah. Let’s just make that perfect.

Dan Widing 8:01
All right. So many tweaks. But that got our first customers got to first implementations got us in the door with an accelerator era out of New York. Era, we went and got a bunch of customers through just their network. got us our seed round, and then it’s just snowballed from there.

Alexander Ferguson 8:23
So starting from you go over to for all our to Eastern Europe. And then you see these folks are grinding out startups, you’re like, oh, wow, this, this could be maybe we maybe we could do this. And you’re like with spitballing ideas, you see, actually maybe around the testing space or something. And people you see there’s interest you already right there, this could be built into something. So you come back, you start it up, then you start to narrow it down once you built it. And finding that there’s a space unmet need when it comes to this end to end testing, regression analysis. And you get your first client a customer and building more consultative give, but man behind the scenes, you build it, it happens and seed round. And then it goes from there. So tell me understand like today, if someone signs up, walk me through the text, like like, how does it How does it work?

Dan Widing 9:11
So start the conversation. Hey, what if you didn’t have to hire a bunch of people in QA engineering to scale up testing of your development project? What would that mean to you? How much faster Could you go? How much cheaper? Could it be? Great. World’s probably significantly better. So many fewer nightmares so much faster? Usually, the next question is Oh, that’s impossible, though. Great. Perfect. Here we got a black box that makes it possible. Let’s open up the box start walking through how does each of those pieces work? How Can it really be done? We look at the customer’s application and say, here’s what end to end testing would look like for you guys in particular. This is what a spectrum of tests might look like. Here’s where you’d want to put extra attention. Here’s what you might What to ignore? Here’s what we

Alexander Ferguson 10:01
can do a use case of one of your customers that you can talk about of how they’re used.

Dan Widing 10:06
Sure. So the one we like going back to fairly heavily is negotiate us. They’re one of our customers they’ve picked up through era, on, they have a system for procuring office supplies, like office supplies, ecommerce organizations that have a bunch of office managers need to acquire the second. We figured out with them, okay, you’ve got a fairly straightforward application and testing works really well for you. You, we figure out sort of scope of initial engagement, a pilot structure, what we do with them, and then they install our tracking step. Our tracking snippet, very similar Google Analytics gives us information about the entire spectrum of what you start doing. We then have this analysis engine or system behind the scenes that grox that data and breaks it down into, okay, based upon what we’re seeing, there should be a test case, there should be a test case, there should be a test case, it just keeps spitting them all out. At a certain point, it’s able to project Okay, you will have covered X percent of user behavior, if you build tests up to this point. This is our projected test suite. This is what we think should be the thing. Some of the pilot that we’ve go in and start building this out and showing them what they look like in a test suite. If they go forward with us with the pilot, okay, now we’re we’ve got these build scripts, we’re continuously analyzing and user behavior. We’re continuously analyzing what the test suite is currently covering, we can figure out, Okay, what should be the next test suite? Or what should be the next test script? Or how should we change an existing script to handle more behavior?

Alexander Ferguson 11:57
That’s, that’s the one time that is building in Okay, now, we haven’t This is running, but it’s always announcing that first step, so you can adjust the second part where you’re doing the actual and then testing,

Dan Widing 12:06
right, which intuitively makes sense. Like if you hire a QA engineer to work on a QA engineering project, you don’t hire them for like two months, and then

Unknown Speaker 12:13
you’re gone?

Dan Widing 12:14
Like, no, you’re there through the life of the project, you are continuously working on? likely the same as like a same sort of framework. It’s almost like a test cafe Cypress Selenium test suite, that you’re just building up additional features, improving quality over time,

Alexander Ferguson 12:31
gotcha. When it comes to the space of QA testing, what’s what is what are you most excited about as far as the direction that you guys are headed and like the roadmap and what you could share?

Dan Widing 12:41
So let’s see, I, oh, we’re excited to get this down faster and faster and faster.

Alexander Ferguson 12:48
If they could be more efficient, more, introducing more efficiency?

Dan Widing 12:52
Yes, I one of our one of our issues, Meantime, to magic, you have to install the tracking snippet, that tracking snippet has to send us data, we have to process the data to test cases. And we have to actually build them all out before you really see what you get. The faster more we make that loop, a shorter timeframe. Obviously, faster iteration times better, per

Alexander Ferguson 13:15
se, it’s, it’s being able to eventually get to the point of knowledge. All right, boom, click a button. And then you we now know what your users are doing. And we can build tests around it. Exactly. Yeah. And there’s,

Dan Widing 13:32
it’s, there’s so many hairy problems all the way down. There’s like framework improvements you have to make constantly make, there’s infrastructure improvements you have to make because you don’t just want to test one browser, you want to test multiple browsers. And actually one of the usually like showstopper kinds of problems that everybody has to figure out is, every application is wildly different. How do you handle test data across these systems? And so any, all of those things contribute to our roadmap of like, Oh, this thing starting to break down at the scale, we got to work on this up, this thing’s breaking down of the scale, we got to work on this.

Alexander Ferguson 14:06
How do you manage the quantity of of, say, application interactions? And is it so varied? Or do you truly is that you see the commonalities among it? And it’s not actually that very well, it comes across? Every on every web application

Dan Widing 14:23
has unique behavior. There’s certain like, obvious features that everybody has, right? We’re almost everybody. There’s In fact, even exceptions to that, like a login flow is generally the same sort of thing. We said password flow is generally the same. A settings page is going to look like a settings page. There’s certain like, shopping cart. Even those variants, there’s certain things yeah, common features, but every the things that are truly truly common and like sufficiently comment that like you don’t really need any testing. It’s like a blog or Shopify page, everything Your handle, we’re everything we like, if you’re hiring an engineer, a software engineer to build a product, you’re doing something custom, like because otherwise, you would have gone to WordPress, you would have gone to Squarespace, you would have gone to Shopify. Right,

Alexander Ferguson 15:12
right. What, as far as the tools that have been used in the past for for this type of testing and QA testing, I mean, are you whether you focus on any integrations, or is it really just like you’ve built all your own custom solutions to be able to do that all the testing. So we do have

Dan Widing 15:29
a, like a vertically integrated approach, where we work with test cafe as our framework open source framework. We have obviously a bunch of open source frameworks that we use for our infrastructure. But we also have integrations here said, if folks want to rely on browser stack or Sauce Labs for setting up browsers, they could do that we have a partnership with applitools. So hey, if you want our tests to feed applitools, visual regression testing, visual AI comparison, they can do that. And then the way that we build out test suites, they just naturally integrated with every car on the planet. Anything that can run the terminal bash, can run ProdPerfect.

Alexander Ferguson 16:21
So let me switch. Switching back to this this conversation you said earlier, how you introduce the product is to someone who say, do you really need your QA engineer person? Can we just remove them from the equation? Do you truly see the future where they’re they’re just no more QA people needed? And just it’s all automated.

Dan Widing 16:39
So let’s see, for the kind of end and testing and simulation of users. Long term Yeah, I see that being taken up by more and more smart to like,

I think, actually, when we compare this but I started my career as a software performance consultant. Somebody who knows how to profile and application at load.

Mitch field, not very many of us, not many, very many needed. Ai application performance management tools came up things like New Relic, nowadays, like data dog. And they kind of obviated the need for us to show up to a lot of engagements. They were just doing all this stuff all the time for everybody. And it’s not like we disappeared as a field. But fewer were needed. It was more specialized. Instead of being something where you had a stock engagement, or you had something that’s truly like, Oh, this is interesting engagement. This is like, buy a novel experience. And to be honest, for the most part, QA engineers are kind of inherently like it’s inherently support. It’s apparently back office one way or another. Most QA engineers find the better career path is to move into software engineering, just as like, on average far better paying for a glory

Alexander Ferguson 18:17
if QA engineering like like a first step in the in the field that someone verified. Yeah, so what’s what what do you see is that just as pontificating here if truly it does get automated that that entire field and just like your current your your micro field change, what will be the new entry field, that you’re pretty pre secure engineer would have done that now that they could do.

Dan Widing 18:41
So the thing that’s actually almost a little bit of a mystery to me, like I’ve taught a number of boot camps at this point, you know, how to become a software engineer how to get a job, six figures off the bat, no experience anywhere else. And truly, like some of the tools that we relied on entirely online, every single thing that you’ve wanted to learn free courseware online, the training projects, literally everything on so far there like only a very tiny number of people that will take on just learning those things by default as a single thing without some push from a boot camp some push from a social experience do it my guess i i the training materials, the training programs will begin even cheaper, be more effective when people see how they progress you eventually will have enough people like go from high school degree to job at Google based upon nothing but the internet that’s like oh, this is sort of an obvious path we should do this for

Alexander Ferguson 19:54
it’s gonna go instead of just I’ll get a low level job is like I’ll get the specific training. I need all the tool sets Then I can just immediately go to a good job.

Dan Widing 20:03
Yes. And there’s, I mean, obviously any market eventually use it up, but there’s so much demand for software. It’s not like, we’re just gonna make more software projects. It’s not. It’s not like a loss,

Alexander Ferguson 20:18
which then will have great automated tools for the QA testing of it. Yes,

Dan Widing 20:22
yes. And the ideal is that with good tooling with good practices, it makes it easier to level up your art. So to be able to focus on Oh, you know, I can, I can have the safety net behind me. And I can focus on what we don’t do unit testing, unit testing is a fantastic way to level up your skills as an engineer. I can easily recommend practices to folks to say like, Hey, here’s what you need to look at, to build out your unit tests so that you’re thinking of how to write good code,

Alexander Ferguson 20:53
right? Right. Or your the people that are usually hiring you guys, or rather you using your platform, your service. They’re like VP of engineering, or CTOs or head of engineering that they’re like, Alright, let’s, let’s just change how we’re doing an in our in our organizations and how we’re approaching this. If you’re speaking to them, I mean, what what kind of word of advice would you give them? As far as you know, where we’re headed? What we’re looking at?

Dan Widing 21:17
Hmm. I mean, obviously, like use pride perfect. But I the best singular advice I get to other CTOs, VP engineers is to go and network with the ones around you. Most folks get so deep in their own projects, or their own things that they don’t have a chance to, like, what is it like to be an engineer or an engineering manager and some other company, you only get that with like decades of experience in a long network. But if you actually put the time into like, go and talk to the other CTO next to you like, oh, man, you can really level up your skills, level up your expertise really fast. In terms of the stuff that’s coming down the road,

there’s going to be more and more intelligence, testing tools and tooling, like perfect on your do always want to keep track of those tool sets. And you want to figure out, what’s the stuff that can make you a more effective cross functional team, than having to silo down skill sets.

So that’s what I generally look at, like things like serverless, or Docker, are great tools to help you be able to do that.

Alexander Ferguson 22:41
I mean, you you pass expires, you actually were Director of Engineering at another firm previously, so you were in the space, you were in that role. And you saw this this position, understanding, it’s good to be looking around and seeing all your options, both when it comes to technology, as well as people. Like

Dan Widing 22:59
my last CEO, Susan, she, by one of the best moves I ever did for my career was but Susan, tell me that like, okay, you should really have a mentor in the field, like, Oh, yeah, obviously. And she helped me find a great mentor. in the Boston area, the person who was the director of engineering at localytics, at the time,

Alexander Ferguson 23:30
it is this journey of constant learning and it never ends. I think, for any of us, when you think of the future, and if you could wave a magic wand just for fun, I’m curious what if you could have any sci fi futuristic tech?

Dan Widing 23:48
With a singular magical tack? Hill i’m i’m actually like, half tempted to say crazy for generalized AI. Like, let’s just bring on this full singularity. Bring it on right now. Like, we’re all looking at this, like, we don’t know where the future is going anyway, so why don’t just dive right in and see what that happens.

Alexander Ferguson 24:17
You have no nothing holding you back from saying let’s just let’s just see what I sometimes you look at the world the like, can’t be worse, right? So it’d be worse. Let’s, let’s see where it goes. I wonder actually how I imagined in general AI would actually have their built in their own automatic testing selves that they’d be doing. I can’t What’s wrong, it looks fake, likes fake steps. Make where goes. Thank you so much, Dan, for both the insights of this journey that you’ve been on as well as the product itself. For those that want to learn more. They can go to ProdPerfect.com, and what’s a good kind of first step for them to take when they get there.

Dan Widing 25:00
I easiest thing to do is just move. There’ll be a bot, a chat bot right in the site that’ll go and message us directly. And we’d love to talk to you guys.

Alexander Ferguson 25:08
Awesome. Well, thanks again, Dan. It was good to have you on the series. Absolutely. Thank you for having me. Again, everyone. Thanks for joining us on UpTech. Report. Go to Uptechreport.com to see this full interview and other interviews, and we’ll see you on the next episode of UpTech Report. That concludes the audio version of this episode. To see the original and more visit our UpTech Report YouTube channel. If you know a tech company, we should interview you can nominate them at UpTech report.com. Or if you just prefer to listen, make sure you subscribe to this series on Apple podcasts, Spotify or your favorite podcasting app.