Finding Talent with Advanced Tech | Josh Millet from Criteria

Below is an AI generated transcript. There could be a few typos but it should be at least 90% accurate. Watch video or listen to the podcast for the full experience!

Josh 0:00
The goal is just to arm HR people, our customers with better tools for making more informed decisions, right? So we always position it as kind of a decision support tool.

Alexander 0:16
Welcome to UpTech report. This is our Applied Tech series UpTech report is sponsored by TeraLeap, learn how to leverage the power of video at TeraLeap.io. Today, I’m joined by my guest, Josh Millet, who’s based in LA California. He’s the founder and CEO of Criteria Corp. Good to have you on Josh.

Josh 0:34
Thanks, Alexander. Nice to be here.

Alexander 0:36
Now, your platform it’s a people analytics software company that in its base form is that and pulling from your site here. It’s a SaaS based pre employment Testing Service, it features after two personality skill tests, helping you to assess all these multi dimensional data points bringing it in. So for those whether you’re a CEO of a small, firm, or even enterprise companies up to 50,000, if you’re involved in hiring, this might be a platform you’re going to want to check out. Josh, help me understand here. What was the beginning genesis of the idea for Criteria Corp? And how did it start? And where did it lead to today? What is that problem that we’re solving?

Josh 1:15
Sure, yeah. And thanks, again, for having me on. You know, we basically help companies make better hiring decisions and better talent decisions throughout the HR lifecycle. So that’s, that’s what we’re about is, is arming companies with data to help them make better, more objective, more effective hiring decisions. And the genesis of the company was actually, almost 15 years ago, now. My first startup was acquired, and I was involved in hiring for the company that acquired us, which was a little bit odd, because I had no experience in that realm. And but I began to play a role in the hiring process for the for the acquiring company, and was actually in in a really bad interview. You know, anyone who was was been involved in hiring, has has one of those stories where, you know, you’re, you’re sort of in an interview, and it’s only five or 10 minutes into the hour, and you look up at the clock, and you’re like, when is this going to be over? Because sometimes there’s just a realization that there’s not a fit, you know, from either from either side. And so it’s in one of those interviews, actually, that I was struck by the idea that like, Hey, there needs to be a way to use better tools to save some of these interview hours, you know, save some of these wasted hours. And so that was kind of the spark that led to Criteria to

Alexander 2:37
today. Now, was that the company number two, was the company that Yeah, we

Josh 2:41
were it was a test prep business that we sold to a company called zap, that that’s involved in college admissions. And they’re based in Culver City, California. So that’s I was on the east coast and came out to live in LA, as a result of that acquisition,

Alexander 2:57
then experiencing that problem of going through longer interviews, like, when will this be over? That’s the genesis of the idea. There’s got to be technology that can come in here and solve this. Now, today, there’s there’s hot buzzwords, of course, using AI and machine learning to be able to solve these, but what’s the Where have you actually apply the technology? And where does? Where does technology end? And people still need to be involved? Because that there is some concern out there? Okay, how much? Do we just let technology automates and help us? versus do? Is it enabling people?

Josh 3:29
that’s a that’s a great question. And, and we’re very careful to differentiate ourselves from some approaches that really have have what I call pure AI approaches that really looked to automate every part of the process, we’re really not doing that we’re using data, we’re using some machine learning even. But really, the goal is just to arm HR people, our customers with better tools for making more informed decisions, right. So we always position it as kind of a decision support tool. And I think what’s unique about us is, whereas some of the approaches you’ve seen in the last couple years are really solely data science focused. We’re combining research in the social sciences, specifically in in, in organizational psychology, with data science, to make sure that the the science underlying the system is, you know, we like to use science, it’s peer reviewed, that is transparent. You know, I think it’s really important for both scientific and ethical reasons that when a candidate is being evaluated or considered for a job of any kind, they should have some sort of idea of what they’re being evaluated on. And why and that’s where I think you you get into some issues with the the approaches that are pure AI driven is that often they’re just completely opaque and, and the candidate has no idea how The black box,

Alexander 5:00
I put my feet in there, but what actually is in there? How will I get hired? No One No one’s a big.

Josh 5:07
Yeah, the black box algorithms are problematic, I think, you know, and you’ll hear some providers say, Well, you know, our algorithms are proprietary, and that’s fine. But if an HR person or anyone who’s involved in hiring, which is kind of like the highest stakes decision a company makes in the area of their talent, right, who do I hire in the first place, sort of the highest impact decision they make? You know, you want to be real clear about what the technology you’re using is doing and how it’s helping select people. And if you can’t explain it, you probably shouldn’t be using it. Yeah.

Alexander 5:42
So that’s, that’s a great word of wisdom for anybody now utilizing software to help them in their job today, if they can’t understand is that like a universal thing? If you can’t truly understand if it is a black box, should we be staying away from those types of solutions?

Josh 5:58
Yeah, I mean, it’s a great question. I mean, more broadly, to think about it outside the HR space, it does have a lot of implications. But I think, you know, it’s fine. Every every software company has a secret sauce. But in the area of human capital, where there’s there’s ethical concerns, there’s legal concerns around hiring, it’s highly ranked, it’s pretty highly regulated area in the US, you know, you need to make sure that you’re aware of how the technology is working. I think that’s a really important point. And some of the cases in the last couple of years where some of this AI software, which is really important area, and you know, we’re incorporating machine learning into our own into our own software. But I think where it goes too far, sometimes is where algorithms are created that are, that are looking for correlations with performance, right, everyone’s trying to predict future performance. But if you’re only looking for correlations and not trying to measure the things that actually cause the high performance, you get into a problematic area.

Alexander 7:02
It’s almost like the root factoring of what you use as your base of who is a worthy candidate versus not, you’ve taken a different approach.

Josh 7:12
Yeah, I mean, we’re, we’re kind of looking at science where, you know, if I were to describe it to you, sort of the overall, the overall conclusions of a lot of the organizational science, in the last few few decades, are not terribly surprising. Like, I’ll give you a couple examples. You know, there’s a whole body of research that shows that cognitive ability is a really good predictor of of success, right? There’s other research that shows that certain behavioral traits, specifically conscientiousness, which is sort of how reliable, how organized, how hard working you are, that that is a good predictor of success across a lot of different roles. And, and so that’s, that’s all peer reviewed science. You know, hundreds of studies show that. But if you were to tell the average person who’s hiring, you know, in the US that, you know, smart, hardworking people tend to do well, they’d be like, yeah, duh, you know, it’s not exactly the frontiers of knowledge being pushed there in terms of, you know, the science that we’re using there. And you can combine that with data science and get some really exciting technology. But if you lose sight of what you’re looking for in candidates and just look for things that correlate, you can get into some really murky and ethically problematic territory.

Alexander 8:23
One of the things we had talked about before our discussion today was, is that the Amazon story, can you share how that how that ran into the play and try to avoid situations like that?

Josh 8:33
Yeah, I mean, that’s an example. Amazon’s obviously, you know, incredibly sophisticated company that has amazing engineers, and they had a kind of high profile flop in the area of AI, where they were modeling some some traits in their engineering population with the goal of increasing diversity in that population. Because, you know, engineering is a field that typically, women are under it, you know, representative, and, but they abandoned the effort. And it was kind of this public egg on the face moment, because the data they’ve been using to train the model was based off their largely male engineering, a population, their existing population of engineers. So it had the opposite effect and ended up disproportionately excluding women even more so. And that’s a company you know, as smart as Amazon. So there’s another another case of a company, one of our competitors that uses video interviewing technology to and uses some facial, not facial recognition, but facial uses, uses the candidates reaction facial reactions to draw correlations with performance, and it was heavily criticized because there’s no science to back up the idea that these correlations are meaningful. It’s just literally chasing correlations in the data set. And, you know, after after pretty significant criticism, they just just this week rolled back that part of their algorithm. And so they’re not doing it anymore. But that should never have seen the light of day, in my opinion. Because, you know, it’s, as you said, up front, it’s kind of creepy, right? That, that you would be doing this, and the candidate certainly has no idea of how to, you know, how to perform an evaluation where their facial movements are being analyzed, right? I put my face in my face, right? Who knows what the best, best, according to the AI?

Alexander 10:43
So a question for you, Josh. How can we use technology to enable us to find the right candidates that are proficient and will do the job that we need, but also support inclusion? And and actually encourage inclusion as a broader How can we use technology to cover both of those bases?

Josh 11:02
Yeah, it’s great point. And so one of the one of the central problems that people who are hiring, hiring hiring employees used to be sort of 10 or 15 years ago that like, okay, where do I find my candidates, you know, it was always competitive to go find candidates. And technology has really helped solve that problem pretty completely, because it’s really easy to post a job now. And it’s also from the applicant side. Luckily, it’s also very easy to apply, you know, LinkedIn, and sites like this have this one click Apply, creates this kind of phenomenon we call resume spamming where, you know, if you’ve got a, if you’ve got an hour and you know, can a red ball, you can apply to, you know, 20 to 30 jobs an hour, right, it’s very easy. And so that’s, that’s good that that process has become frictionless, but on the other hand, it creates another dilemma for employers, which is now Oh, my gosh, I have all these employee all these applicants to my job posts, and how do I sort through them? And how do I know where to start that, and that’s where technology driven tools and assessments like the ones we provide, can really help because they give you an objective data point. That’s very, you know, typically, if you’re using an assessment really early in the process, it can kind of help someone who’s an HR tell them where to start directing their energies first, you know, people who did did well on the assessment, for example. So it really creates a lot of efficiencies there. And to the second part of your question about diversity, I mean, that that’s something that we think about all the time, and it’s become so much more top of mind this year with all the all the events of 2020. And the Black Lives Matter movement, I think, really shone a light on on it and some other things as well, during COVID. And, you know, if you think about it, what do companies really use at scale to hire people, they use resumes and interviews, right? Those are the two most ubiquitous tools that everybody uses, right? Sometimes applications replace resumes for certain types of jobs, but it’s the same basic thing. And if you think about just the way that bias works, and unconscious bias, resumes and interviews, both are really bad. From a bias perspective, there’s a lot of research that shows that resumes are really problematic in terms of injecting unconscious bias into the process. I’ll give you an example of a study that was done. It’s not even a recent study. Now, it happened more than 10 years ago, but it’s kind of a famous one in HR, where these researchers sent out all these fake resumes to job posts in Boston, Chicago, and they tracked the callback rate to the resumes and and they found that when they changed the name at the top of the resume, nothing else about the resume, but just the name. From a stereotypically white sounding name to an African American sounding name, the callback rates went down dramatically. And these are the same, the same resumes, right? The same fake credentials on the resumes. And, and you know, HR departments, this is not intentional bias. So they tend to be very progressive as a general rule. But, you know, I think the point about unconscious bias is that everyone who’s human and has a brain has these images of what, for example, a successful salesperson looks like, or what a successful manager looks like. And often, I kind of call it the central casting agency. Rule, you know, you’d like okay, who would play a salesperson in my movie, you know, probably it’d be someone like Tom Cruise, right, who’s super charismatic and slick. And so, I’m going and you have these images that correspond to these roles. And so that can really be detrimental in the hiring process. And so the idea behind assessments is Hey, Let’s use objective data that can help you surface talent that might not look like your image of what that role is.

Alexander 15:09
Are you guys? What are you? What’s the most exciting upcoming feature or product that you can share on the product roadmap that you’re rolling out? That you’re talking about?

Josh 15:17
Yeah, so So there’s a couple things come to mind. One is just a new area of assessments for us, which is around emotional intelligence. It’s kind of a little bit newer field of research. And it’s a really exciting area. And we’ve we recently acquired an Australian company called rebellion that has this really great game based assessment of emotional intelligence called emotive fi. And so you know, emotional intelligence, again, is kind of a newer area of research. It’s not universally useful across all jobs, but especially in roles like managerial and a lot of sales roles that involve a lot of interaction with other humans. There’s, there’s growing evidence that, that AI is really helpful as a predictor of success. So that’s, that’s one area we’re excited about. Another is a product we call talent insights, which we just rolled out this past week, actually. So it’s kind of an early release. And it’s around using assessments post hire, so not so much for answering the question of who should I hire, but for growth and development and, and team building and fostering collaboration, that kind of thing?

Alexander 16:31
If you had to give a word of advice to someone who’s involved in hiring, whether it is the CEO or HR manager, or diversity inclusion person, in today’s environment, what word of advice would you provide? Yeah, I

Josh 16:48
mean, to sum it up, I think, both for getting great hiring results. And for encouraging diversity, I think taking the approach of hiring for potential rather than for experience will be will go a long way towards accomplishing both both goals

Alexander 17:06
is, especially with this last year, the pandemic. I’m curious, how do you see if we were to three years from now looking back? How will this this period of COVID affect us?

Josh 17:19
Yeah, it’s a really interesting question. I think, you know, in the world of work, the world of HR, you know, the first answer that everyone would give is around remote work and how that’s changed. And I think that’s a, that’s a meaningful long term trend. I mean, at our company, we’re, we’re definitely experiencing that. I think so many companies are going through that now that we’re, you know, hopefully getting past the worst of COVID, the question becomes, okay, how do you how do you put the genie back in the bottle, you know, how do you go back to in person, but also accommodate remote, which we now know, works so well, for a lot of companies. So, so remote work is certainly a big trend. But I think another trend that will be just as enduring, if not potentially more is, is the renewed focus on diversity and inclusion. Like, that’s always been a part of what HR does, and you know, for a while now, so it’s certainly not new with COVID. But there’s just more buy into it, I think across a lot of different companies that, you know, have seen that, hey, there are real structural inequities and in society that, you know, companies are stakeholders in society, and they need to be a part of the solution there. So I think that heightened focus on DNI initiatives is going to be a very long lasting one along with the transition to remote work.

Alexander 18:44
Well, just I really appreciate you being able to share the insights as well some of the features that you guys are rolling out a Criteria core. For those that want to learn more, you can go to Criteriacorp.com. And looks like get a get a free trial there. But stick around for part two of our discussion where Josh is gonna be sharing more of his journey, building Criteria Corp and the experiences of what does it take to build a team of over 100 folks that and continuing growing, so stick around for part two, but thanks again, and we’ll see you guys on the next episode. That concludes the audio version of this episode. To see the original and more visit our UpTech report YouTube channel. If you know a tech company, we should interview you can nominate them at Uptechreport.com. Or if you just prefer to listen, make sure you’re subscribed to this series on Apple podcasts, Spotify or your favorite podcasting app.