Twenty years ago, the term “beta testing” might not have meant anything to your average person on the street, but today, with apps integrated so deeply with daily life, most everyone is familiar with the idea. But despite how ubiquitous the need to road test software before a full release, few companies have actually developed a formalized process. Instead, they just wing it.
Luke Freiler discovered this as a Web Development Manager at Ericsson. He was tasked with beta testing a massive product and discovered, to his horror, that the company lacked any procedures whatsoever on how to conduct such an endeavor.
When he was given permission to look outside the company for help developing beta testing protocols, he discovered no such services existed. That’s (eventually) how Centercode was born.
On this edition of UpTech Report, Luke discusses how he’s managed to leverage a once-in-a-while need into a full engagement lifecycle for some of the biggest brands in the world.
More information: https://www.centercode.com/
Luke Freiler is the Founder and CEO of Centercode, a beta and delta testing company that’s helped hundreds of enterprise and high-growth tech companies continuously perfect their products by leveraging the enthusiasm of their customers. With a network of over 250,000 testers, a team of expert consultants, and a software platform that automates the whole process, Centercode makes it simple to manage pre-release product tests.
Product managers, engineers, and testing professionals use Centercode to collect more product feedback with less time invested. They use pre-prioritized insights from Centercode to make data-driven product decisions and impact success at release.
With a background in user experience and software development, Luke continues to lead the design of the Centercode Platform, a comprehensive SaaS solution that enables companies to continuously engage with their audiences throughout product development and iteration. Before founding Centercode, Luke worked for Native Instruments as a Product Specialist and for Ericsson as a Web Development Manager to help develop, test, and launch a variety of consumer and business products.
A self-identified Tech Idealist, Luke’s vision is for a world where technology truly serves to solve real problems. He is passionate about working with product makers to turn that vision into a reality — one product at a time.
TRANSCRIPT
DISCLAIMER: Below is an AI generated transcript. There could be a few typos but it should be at least 90% accurate. Watch video or listen to the podcast for the full experience!
Luke Freiler 0:00
One of the best things that I’ve learned as an entrepreneur is to think of the company as a product, and to design and engineer it like I would anything else.
Alexander Ferguson 0:15
Welcome everyone to UpTech Report. This is our apply tech series UpTech Report is sponsored by TeraLeap. Learn how to leverage the power of video at teraleap.io. Today, I am very excited to be joined by Luke Freiler, who’s based in Laguna Hills, California. He’s the CEO of Centercode. Welcome, Luke. Good to have you on.
Luke Freiler 0:33
Thank you, Alex. Great to be here.
Alexander Ferguson 0:35
Absolutely. So your platform and service is really those who want to connect their companies, with their customers, whether you’re trying to run beta tests, or as you like to really coin and move the emphasis towards is, is delta test, always testing. Basically, if you’re out there, you’re a product manager, or other business leader who needs to recruit great testers improve your engagement and discover, okay, what needs to be improved fix and change to get your product better? This might be an intriguing platform and service you guys want to check out? Now on your on your site, Luke, you say on demand access to your targeted testers Tell me, what problem did you set out to solve? And how has that changed over time?
Luke Freiler 1:18
Absolutely. So this, like many businesses was sort of born out of necessity. I started my career in tech, very young, I worked for Samsung. And then I moved over to Ericsson with a group of peers from Samsung. And we were tasked with building a really big product. And my part of that was basically everything web facing. And this is nearly or just over 20 years ago. For me, that meant the firmware on the product, which had a web interface, the marketing, web presence, internet’s tools, just anything that was web sat under me and I had a team and a product manager came to me and he said, Hey, I need you to run a beta test for the product we were working on it was this home phone system kind of aimed at small businesses. And I said, Okay, what does that mean, exactly? And he said, Well, you need to get some testers. No, no, I understand the beta is, what does that mean, in terms of Erickson, we, how do we do that? What was the process? Where do I go? And he said, we don’t have one best I can tell. And he was a product manager, pretty senior guy. And I said, Well, that’s impossible. We have a three inch guide for how to use the logo, there’s no chance this thing you’re telling me we have to do isn’t figure it out. It’s not a solved problem. It’s 100,000 person, 100 year old tech company very established. And he said, Look, man, if you can figure it out, make it happen. And I said, Okay, well, can I outsource it? And he said, sure. He said, Okay, I’ll go find your solution. And I started to sort of scour the web. And I realized that this wasn’t a solved problem. There was one company who’s now long gone, that was doing it. But as far as how we recruit from our audience, bring them in, solicit feedback, do something useful with that feedback, it just wasn’t a solved problem. And personally, the reason I gravitated toward the web is it seemed like the next user experience, it seemed like the thing that was obviously going to be very important. And it was just ripe to grow. And for me, as someone who really loved the idea of technology being useful in solving problems, I was already enamored with this, this new idea at the time, this novel concept of usability, that technology should be friendly. And I sort of started to put one in one together and realize that, hey, this idea of of finding enthusiastic color customers and engaging with them, is harder than it looks. And as I started to talk to lots of different product managers, I found out that virtually everyone had the same problem. This was something that was thrown on their lap a couple times a year. And they didn’t really have a solution. And it was sort of an 11th hour responsibility of most product managers. Everybody felt it was important, but it got dubbed unnecessary evil. And that was when I just struck at the opportunity was, okay, unnecessary evil, something you have to do, but it sucks. That’s an opportunity to fix a problem. So I started to do my due diligence and reach out to my network and beyond. And what I realized in time was that the reason this was the problem was is because it was this infrequent need. It was something that once a year each product manager had dropped on their lap. They went out there and did it, they forced it to happen. But by the time they did it again, they forgot everything they learned. And because it was infrequent and episodic, there wasn’t really as firm in need for a consistent tool set a process written down. It was something that nobody ever stepped back to fix for the rest of the organization. They just all kind of forced it as they went. And that became an interesting opportunity to me to solve that problem. And then as the world evolved and technology changed, products got more sophisticated, they got more connected, and most importantly, they became continuous and product didn’t just kind of go out in this fire. Forget mentality. But rather, you would put out a product and you would evolve it over time. And customers would expect it to work with the latest other products they bought. And it was creating this web of complexity that was only getting worse. And on top of that it was getting more competitive. And the audience’s for technology for the last decade or so have gotten much more mainstream, which means they have less patience for things going wrong, and not working well and easily so. So that shift created an even bigger opportunity, in that there’s now this continuous need. And in the engineering world, we all moved from what was called Waterfall development to Agile development. And this idea of a big effort to put something out and then sort of do it again, shifting into this new mentality of small slices and constantly getting new value and customer hands. What hadn’t really evolved in those companies was how to take this many decades old idea of a beta test and evolve it to that which when you reference Delta test, definitely something we’re passionate about is shifting the mentality from this episodic infrequent approach to customer testing, to something more continuous. So we saw that as the opportunity opportunities grown, it’s evolved, we’ve grown, we’ve evolved. And it’s been a pretty fun, cool space to be in.
Alexander Ferguson 6:15
And such a code was born. I love the intersection where you you see a problem. And people come across it regularly. But it’s something they don’t want to touch. And you’re like, I can solve that. Let me let me make life easier now that this was 19 years ago. Was that correct? Yeah. Coming up in 2002 2001,
Luke Freiler 6:34
the worst possible time to start a startup outside of maybe now? Yeah, we got it right. As the internet was crashing and burning, we decided we wanted to have a software company,
Alexander Ferguson 6:48
I really am excited to hear more of that founding journey. And for those that want to hear to definitely stick around for part two of our interview with Luke, but to give a taste, what’s what’s one thing that you wish you had known 1920 years ago that you know, now,
Luke Freiler 7:05
um, the thing that, you know, I started as an engineer, so most of my kind of operations are based around that the way I think of the company and whatnot. But I definitely separated my approach to engineering from my approach to the company. And one of the best things that I’ve learned as an entrepreneur is to think of the company as a product, and to design and engineer it like I would anything else. So as an example, as we’ve grown, you know, just to be blunt, one of the things I’ve never enjoyed is sort of HR and operations and, and just everything surrounding the what feels like bureaucracy of a bunch of good smart people working to do a big good thing. And I’ve had endless conversations and hours about those things that frankly, I didn’t enjoy, they just felt like I was being dragged in. And at a certain point, I came to this revelation that, well, if I think of this company, as a product, and a, you know, happy group of motivated smart people, or just a piece of that product, then I can design all of the systems that run the company just like I would the platform. And it’s really, really helped me wrap my head around how to solve new problems that arise from a growing company and more people. And even, you know, I would say this year, even more so because we all had this enormous curveball thrown at us. And we needed to think absolutely differently and reshape our company reshape how we approach every policy there is. And again, thinking about the company, like a product that’s evolving, just like every other product, I think has allowed us to be pretty agile in how we’ve approached COVID. And pretty successful as a result. So I think thinking about the company itself as a product was probably something I was a little late to the game on probably took me about a dozen years to get to that point,
Alexander Ferguson 8:59
a powerful perspective shift that can make a difference. I’m excited to hear more about that in our second part of our discussion, coming back to dissenter code and the platform, and the service that you provide, can you give me a use case a case study of one of your clients and the typical walkthrough of how your platform works?
Luke Freiler 9:17
Sure. So to step back one bit, there’s two sides to the business. And companies engage with us in one of two ways. So there’s the platform, half of the business where companies licensed our platform and sort of a subscription model. But there’s also a service side where companies that have no interest in doing that and just need to get the work done, leverage our managed services. So our revenue is typically pretty split a little more on the software side, but we do both. The reason that’s so important is because that service side means that we eat our own dog food a lot more than most companies would half of my revenue is based on not only our customers succeeding on our platform, but us succeeding on our platform. So that’s my test kitchen. I have a bunch of people that run these things for a living. And that allows us to continually evolve and whatnot in some really interesting ways. And it’s a wonderful thing. So to answer your question directly, what the platform basically does is allows any company to stand up a community of their most enthusiastic customers to recruit them in and to profile them on the way in and some very unique ways. Not only do they want to learn about who these people are the kind of general demographic stuff that that’s pretty normal. But what we do uniquely is we also solicit a lot of information about the technology that surrounds these people. So if you come in and and for example, I’ll take someone like, peloton you know, peloton puts up a community recruits, they’re very enthusiastic audience, not only would they collect information about who they are, and age and all those things that are kind of typical marketing. But in our world, we have something called Technographics, in addition to demographics, and the success of a product is not only dictated by how well it meets the sort of market needs, in terms of who those people are and how they think. But it has to work. It has to work with other technology that’s outside their control. So not only do we profile the people, but we profile their ecosystem. So What phone do they own, what computers do they use, what streaming devices do they use on their TV, and and technology that may even go far beyond what peloton is thinking about today into where they might be thinking eventually, and allows them to sort of build this very profile database about their users that that is going to help influence and shape the product. So it starts with sort of building a community, then it comes down to okay, we’ve got a product we’re working on, we want to recruit from that community, we want to make sure we have a great representation of the target market. Again, both the demographics and the Technographics, make sure it works. And then it allows them to distribute product in the case of software, or help coordinate and orchestrate the distribution of hardware, and then start engaging with those users started sort of taking them through a pathway of testing that product on a feature by feature basis to collect feedback that they can take action on and in our world in the framework that we’ve developed. It’s about collecting very explicit feedback issues. What do we need to fix ideas? What should we be improving? And praise? What can we promote about this product. So fix, improve, promote everything that we want out of this is actionable, it’s what they can do to actually improve their product and sort of the secret sauce of the product is looking at all that feedback in real time and scoring it based on a huge variety of signals, and ensuring that as that feedback is coming in, it can be prioritized. So they know what they need to take action on what is the important security issue that’s impacting a lot of people versus what is the cosmetic issue that might just be an outlier and a pet peeve of one person, right. And we believe that in terms of taking in feedback and processing it, there’s two sort of empirical facts, one, not all feedback is created equal, some things are highly impactful, and are really going to hurt your chances at success. So not all feedback is created equal. And too, you can’t address everything. It’s a very complex product, in most cases nowadays. And again, it’s complex, it’s connected, it’s continuous, there’s so many moving parts, that it’s an unreasonable ask to fix everything to implement every idea and so on. So in that case, it’s all about how do we prioritize to know what we can do to have the biggest impact to perfect this product to get to market. And that platform basically facilitates that that entire process of leveraging that enthusiasm, getting the feedback, prioritizing it and taking action.
Alexander Ferguson 13:40
It’s amazing the the entire lifecycle, then that you’re able to facilitate. And also the fact that you said a man is services, I feel like she that’s a large movement overall for SAS, because some people just need the full solution. And the technology should enable whether it’s an end consumer yourself, if you’re providing said service, to just do it better do it faster. And it sounds like your platform definitely accomplishes that. The business model itself, is it based on a guess based the every time you do one of these lifecycle tests of a product, how does it How does that work?
Luke Freiler 14:13
So on the software side, you know, like most SAS, we have editions that are aimed at sort of different sized companies. So everything from you know, kind of the general SMB, to the biggest of the enterprises, and frankly, most of our plays on the enterprises and kind of the big brand names. Most of our customers are what we would call high growth tech companies, the pelicans and Roku and Fitbit, so the world. And then on the other side, it’s what we call the modern enterprise, those companies going through their digital transformation. And for us, that’s, you know, Procter and Gamble and Ford and Citibank, things like that. So, on that side, it’s it’s basically the sort of size of company dictates the feature set, they’re more likely to need, you know, is it a big program, they have a big team or is it something very small, and then from there, it’s how many programs they’re going to be running at any given time. So, you know, a small company might be running a few or a couple, whereas a large company might be running into the hundreds at one time. So that’s on the software side, it’s kind of licensed on a on a project model. On the other side, it’s it’s scoped around, really two things, how many testers for how long. And those are two really simple metrics, we just call it a test your week that allow us to kind of scope out and really that’s done based on what is the feature set that we’re looking to test, and getting a sense and really understanding what the product is. And then we can place that to a schedule, and also look at their market and decide how many they need is this something we can do with 50, testers, 100, testers and so on. And it’s typically sort of quoted from there. And then from that, that’s sort of a pilot phase, it actually transitions into kind of an annual service model, most companies would be looking to either leverage our platform to do this themselves, or leveraging our services to do this in a continuous model. And we’ve just got resources on staff, you know, at center code that are dedicated to being their team.
Alexander Ferguson 16:02
What would you say is like the most recent feature update that you’re very excited about that just came out, or you’re working on right now that you want to promote and tell people about?
Luke Freiler 16:10
Yeah, that that’s a rough one to put into a couple of minutes. But we’re actually working on an enormous release, that’s coming out in a couple months here. And the biggest thing that we’re doing with this release is we’re introducing a bot. It’s basically an agent that looks at the individual behavior of every individual user, as well as the ongoing ever changing state of the project. And it makes connections to increase the engagement of those users.
Alexander Ferguson 16:35
So you’re basically reaching out and talking to these testers.
Luke Freiler 16:39
Yeah, well, it’s it’s emailing it’s texting. It’s, it’s leveraging their behavior. So for example, it starts with some assumptions, what is their timezone? And what type of product is this base that we can decide when is an ideal time to reach out to them, but then it’s building a profile on them. So if they’re more interactive, and they’re more enthusiastic, then it’s actually going to start to reach out more, if they slow down, it’s going to slow down. But it’s adjusting its behavior based on what they’ve done, what types of things they do, if they’re more responsive to being texted, then it’s gonna use that if they don’t like being texted, it’s gonna shift back to email. So it’s basically taking what used to be a role in our organization, and completely digitizing it. And for us, that may sound bad, it may sound like a layoff, but it was actually a promotion, it just meant we took our test leads into test managers, and they’re now more empowered. For the average company, though, what we’re typically dealing with is really short staffed. And product managers don’t have someone to lean on for all of that heavy lifting. And all of that I just need to engage with users in our space. And I sort of skipped this but our space engagement is the big problem is I get testers in, they use it for a couple of weeks. And then they kind of get distracted with their day to day lives. And they move on to the actually feedback. They don’t get, you don’t give you actionable feedback. And more probably, they don’t give you consistent feedback. They don’t they don’t stick. And what this is designed to do is help them stick. So it’s a bot named Ted and we build a whole persona and had a lot of fun with it a lot of personality. And and the idea is that TED is the assistant forever is running the project. So it’s a digital assistant that is engaging in very unique and interesting ways, with with the testers, and that was just an incredibly fun thing to develop. It was new to us. But it is a very, very high value thing. It increases our addressable market to people that simply don’t have those resources. And therefore this was just an unreasonable ask. And now this gives them a digital resource that’s a right hand. And they just tell the system, what they want to test, the system gets it tested. It automates the engagement. And then we build a beautiful series of dashboards to summarize all of this. And the big takeaway, which, you know, again, I have 100 exciting features in this release. But the big takeaway is we’ve actually devised or we’ve announced or made available, something that we’ve been using internally, which is a scoring system for product success. So we believe that we can predictably look at the ratio of the impact of the feedback we’re getting, I’m getting deep here. But we can look at the ratio of the feedback and how it works, and then produce a score that predicts how well this product is going to do. More importantly, it then allows you to say, Okay, well, how many more issues and things would I need to address to reach this score, and it allows benchmarking from project to project. And this is something we’ve been testing and doing manually in internally, which again, the benefit of having a really nice Test Kitchen. But it’s something that we’re going to be exposing to to everybody so that they have a beautiful benchmarking system that can not only predict the success of a product, but can then allow them to figure out what knobs they need to turn to produce a more successful product. And not just what knobs to turn but what are the ideal, most optimized knobs to turn so very exciting thing for product development for us and I think our customers are really, really going to Joanne,
Alexander Ferguson 20:00
this is probably a great example of when you are eating your own dog food. If you’re actually using the platform yourself, you’re like, wow, our own people are spending so much time on this, how can we make this simpler? How can we use technology, and I like your point that they didn’t get fired, they got promoted, you actually the the technology is enabling them to do so much more. And that is the purpose of technology to allow us as humans to do more.
Luke Freiler 20:24
Yeah, and that’s the thing. You know, for me, the best thing about having this company is the just in the good we get to do you know, for me, I was, and it may sound cliche, but my first enthusiasm in tech was usability, it was experience. And even that, you know, that the maybe this goes in the background section later. But that itself is not that I’m, you know, just a very virtuous person, it’s actually that I’m a very impatient person. And I’ve often been, you know, had technology fail me and feel it should be better. So for me, I got a career to address that lack of patience, and hopefully, you know, make it better for everybody. But yeah, we think technology should solve problems. And we don’t look at this, as you know, our bot is actually putting anyone out of work, it’s quite the opposite, it enables people that do this to produce an enormous more value with the same time, or it enables people that didn’t have the time to do this in the first place. So it’s a very, very big turning point for us.
Alexander Ferguson 21:25
For those product managers out there, or those who are managing beta testers, or delta testers, a word of wisdom that you would share with them, even aside from from what you do, just turning knowing their role, what they have to do just a word of wisdom.
Luke Freiler 21:39
Sure, um, the the thing that I like to focus on, you know, two things, the first half is sort of transparency, make these people part of your team make them know that. But really, it’s about finding the right testers, that’s one of the more complex things to explain to people is, you can’t just implement a feature that’s going to solve all your problems, you have to have the right people. And for us, finding the right people means finding testers who have whatever the problem you is that you’re solving and literally thinking about it that way. So the first question we ask a customer when we’re talking to him or her prospect is what problem does your product solve? And from that, we can go so far, and, you know, if we know what problem you’re solving, then we can go out and find the people who have that problem, and figure out if you solve it, and if you solve it, everybody wins. They win as the first thing, you have the problem, you win, because they want your stuff. And we win because we made you happy and keep you as a customer. So yeah, just think about the problem you solve and find the people who have that problem.
Alexander Ferguson 22:38
What can you say is are you most excited about as far as the roadmap five years from now? Where do you want to be? And what can you share about that vision,
Luke Freiler 22:49
I’m really interested in automating as much of this process as possible so that humans can focus on solving the real problems. So what we’re doing with our bots is just the beginning, we’ve got some other great ideas for where to go with that. So that’s very important. Beyond that, we’re looking to expand a lot more internationally and provide this to a lot more markets. We do work internationally today. And ironically, this was the year we were planning to open an office in Europe, which you can imagine how much more difficult that just got. But we’re definitely looking to expand our footprint and all the sort of localization and globalization capabilities of the platform. Again, we already do that the platform is already localizable. But but we’re going further with that. Those are the big things. And then the last one that that’s very important to us is getting much, much deeper into that technographic side, I think the biggest plate people have with products nowadays is is really compatibility. It’s I bring it into my ecosystem, and it doesn’t work with two or five of the things that I have. And that problem is just getting worse. You know, it used to be a thermostat, didn’t talk to other products and just did thermostat things for decades, you know, until you replaced its batteries. And now my thermostat, you know, plays Spotify and controls my pool and you know, six other things that that are all other companies that are constantly shaping their products. It’s just moving targets everywhere. So we’d really like to deep think how to do that better and help companies really deal with the technology side of you know, there are the interoperability side of their technology.
Alexander Ferguson 24:24
Look, I really appreciate you sharing kind of the journey that center code has been on and of the problem that you’re seeking to solve. I mean, Delta testing and being able to continuously improve a product is going to be essential, as you stated going forward. For those that want to learn more, check out their platform and their managed service and go to centercode.com And you can give them a shout. Thank you so much for joining us. Thank you definitely stick around for part two, though, of our discussion. We’re gonna dig in a little bit deeper into Luke’s story and the lessons learned because I know over 1920 years there’s a lot there. Again, our sponsor for today’s episode is TeraLeap, if your company wants to learn how to better leverage the power of video to increase your sales and marketing results, head over to TeraLeap.io and learn more about the new product customer stories. Thank you guys. We’ll see you next time. That concludes the audio version of this episode. To see the original and more visit our UpTech Report YouTube channel. If you know a tech company, we should interview you can nominate them at UpTech report.com. Or if you just prefer to listen, make sure you subscribe to this series on Apple podcasts, Spotify or your favorite podcasting app.
