The AI Revolution – Current impact & future progress

DISCLAIMER: Below is an AI generated transcript. There could be a few typos but it should be at least 90% accurate. Watch video or listen to the podcast for the full experience!

Alexander Ferguson 0:00
Welcome back to UpTech Report series on AI. In this episode, we continue our conversation with Alicia Clodfelter, research scientist for Nvidia. Alicia is an expert in her field, she has a PhD in electrical engineering from the University of Virginia. Since joining in video, her focus has turned towards high performance hardware, including machine learning circuits and systems. In this episode, we start off by asking her what is her view of the AI revolution?

Alicia Klinefelter 0:29
Um, you know, I think of it a little bit differently than maybe someone who is more on the algorithms or kind of theoretic side. For me, I see it more as a revolution of finally having enough compute power, essentially, to do a lot of these complex algorithms that, you know, have been limiting this revolution for years. Because I mean, a lot of the algorithms and the, you know, the underlying mathematics of machine learning have been around for decades since the 1950s. And really, what kind of revolutionize these things is, you know, finally, in the mid 2000s, to late 2000s, we’ve been having this, you know, this enormous progression of compute power that has enabled a lot of us to finally, kind of implement these algorithms that a larger scale,

Alexander Ferguson 1:13
what key factors do you think played a large part in enabling this revolution?

Alicia Klinefelter 1:17
I actually, I think that usually, when you talk to a lot of people in the space, and they kind of, you know, they’ll highlight a handful of things that have kind of enabled this AI revolution in the last five to 10 years in particular, you know, the first one people will say, is compute power, as I already mentioned, which I think is really important. But another one that people will cite is kind of open source, open sourcing of everything, whether it’s the software, or the models, or just any infrastructure around machine learning, open sourcing that is so important, because in order to generate a complex model, essentially, it just takes so much of like a end to end stack that you would have to develop yourself that if I mean, it would be so time intensive to do that, on your own, that if these open source tools weren’t there, then no one could do anything very quickly.

Alexander Ferguson 2:05
You mentioned the impact of compute power and open source, can you give a bit more insight on compute power and progression?

Alicia Klinefelter 2:12
You know, usually, a lot of hardware engineers will talk about Moore’s law, which is this law that has existed from Gordon Moore from Intel, you know, when he founded it, you know, 5060 years ago, which basically is the compute power, I mean, we’re you can basically fit as many transistors in a die, you know, or like, double the amount of transistors on a die in a period of 18 months, every 18 months. And that’s held totally steady, and allowed the miniaturization of all of our electronics for the last, you know, few decades, and it’s steadily kind of moved along. But of course, in the last, you know, five years, probably, you know, we’ve started to see the stagnation, finally, of Moore’s law, you know, just to get into specifics like Intel, you know, who is often release, you know, since they have a foundry, they usually are releasing new technologies every two years. And usually, we describe those technologies at something we call their feature size, which is really indicative of the size of the transistor getting smaller and smaller, smaller. And you know, and they kind of have stuck at this one node, as we call it at 10 nanometre, which sounds incredibly small, but they kind of haven’t been able to move beyond that for the last few years. And we’ve seen other foundries as well, the other main one TSMC, in Taiwan, you know, we kind of see them have struggles as they move down to the single digit nanometer scale. So we’re starting to see this massive stagnation in technology, in terms of how much we can really scale it. And the one thing to note, you know, when we scale these technologies to smaller, smaller and smaller, we also see benefits of scaling these other hardware parameters such as voltage and was voltage goes down, and frequency can go up, meaning your performance can increase. So that means that you get more performance for less power, because power is a function of voltage. So basically, when we start to see that slowdown, then we basically, you know, we have to find more creative ways to get to what we want to get to, we can no longer rely on the scaling this technology scaling we’ve kind of relied on for so long with Moore’s law. So so that gets a little bit tricky, because right now, you know, we’re kind of, you have to kind of think about it. As you know, we have what we have in terms of technology in terms of the underlying silicon that we use to create these chips. And so we have to start getting really creative with different types of like architectures on ship to implement these algorithms and find creative ways to implement these neural network topologies and hardware to basically minimize the power or get more specialized with whatever you’re trying to do to keep power down. Because as I mentioned before, there’s this constant trade off that exists between hardware flexibility and energy efficiency, and you’re always trying to find that sweet spot. So I think you know, what we’re going to see in the next couple years is you’re going to see hardware that’s much more dedicated for the phone And it needs to be four so that you can basically minimize power for a very specific function. So I think we’ll see a lot more of that. And edge computing is very dedicated ships that are very energy efficient. And I think we’ll go from there,

Alexander Ferguson 5:15
what does the future of hardware in relation to AI look like.

Alicia Klinefelter 5:20
But for hardware, you’re actually you’re starting to see a massive boom of right now, especially in the last few years, especially when it comes to startups is you’re seeing a lot of people that are trying to push the the power envelope, or I guess, reduce it, because GPUs traditionally have been known to be very power hungry, you know, multiple Watts essentially. And that’s not necessarily good for what people are now referring to as learning on the edge. Or it would be something such as edge computing, which we usually refer to, I mean, it’s a really broad term. But usually people mean a power constrained application, like putting a chip on your cell phone, to do learning, people even extend it to do like automotive learning. But it could be any device that might be a little bit power constrained compared to say something on a server in a rack somewhere, that can compete, you know, use 10s of watts of power. So so now we’re starting to see this huge group of startups that are trying to do this low power inference and training, we’re doing this kind of inference on the edge processing. And we’re starting to see people do very dedicated what we call hardware accelerators for this, and when we say accelerator, it’s exactly what it sounds like, instead of having something very general purpose, like a GPU, that could compute any number of algorithms based on how you program or configure it. A hardware accelerator is very, very dedicated, and its function. And the benefit of that is that it’s very energy efficient, if you want to do something, but then of course, it’s not as flexible. So we’re starting to see this really big boom of kind of how to find the perfect trade off between those two things. And that’s what a lot of companies are looking for right now. And what a lot of startups are doing is how can I make something flexible enough, in order to future proof myself hardware wise, for new algorithms, because the field is moving so quickly, there’s a new, you know, machine learning algorithm you need every single month. But still, how do I keep it low power enough that, you know, people are going to actually want to buy it, and it’s going to be a competitive chip. So that’s kind of the big boom, we’re seeing right

Alexander Ferguson 7:22
now, what’s the plus of having machine learning on the edge,

Alicia Klinefelter 7:27
big eye, at least, I think one of the big I guess positives of doing machine learning on the edge that we’re going to start to see soon is actually related to security. So I think, you know, having, so usually, I guess what the typical bottle now you know, if you want to do, or if your phone, if we’re using the voice example, again, if it wants to learn something about your voice, as it stands right now, you know, maybe if you have an Android phone, it collects that data, and it kind of sends it out to the cloud, but then to do the actual compute, and then it gets sent back to your phone, and then kind of that information might get stored locally. But there’s kind of this loop that exists from your phone out into the cloud and back. And obviously, you could imagine how there could be some security issues that exist in that loop. So if you have something that’s very security critical, then you if you could actually do all of that processing, but just on your cell phone without having to go back out and come in, then you’d basically be keeping your device a lot more secure your data more secure, in particular. And I think a lot of people see that as being a big benefit. Now that people are more concerned about data security, I guess societally. Um, so I think that’ll be a huge, huge benefit. And I think that’s the main benefit people are thinking of so Alexa is a great example. And is it’s definitely an edge device. But you know, it’s it’s a, an interesting edge device, because it’s actually connected to mains power. So, you know, it’s not necessarily battery operated. You know, it is it is power limited, basically, by, you know, the connection to the wall, of course, but I think, you know, it does give you a lot of flexibility there if you really wanted to beef up security, and basically do all of your inference locally, essentially, without having to go out and come back in. But yeah, it does help. This is where I might take into the hardware details a little bit. You know, it’s, it’s, it’s really interesting, because I mentioned before that, you know, compute power scaled to enable this revolution. I mean, I definitely think it’s the best way to scale up very quickly, again, thinking just more from a hardware perspective, you know, because as of right now, as the networks get more complex that getting any less complex, and as we handle more data, essentially, to train them, then we really need that compute power that might exist more on the server side to handle that. So, so for right now, I think it’s the most practical way forward in order to do complex workloads. But you know, moving forward, we can maybe, maybe scale down.

Alexander Ferguson 9:51
That concludes the audio version of this episode. To see the original and more visit our UpTech Report YouTube channel. If you know a tech company we should interview You can nominate them at UpTech report.com. Or if you just prefer to listen, make sure you’re subscribed to this series on Apple podcasts, Spotify or your favorite podcasting app.