Transcript: From Academia to Industry: Scott Hanson on the Future of AI and Edge Computing
0:00 Brad First of all, a lot of the hardware development process these days is just software development.
0:17 Scott CTOs will share the stories untold.
0:20 Brad Hello, everybody.
0:22 Brad Welcome to another week of the How Any CTOs Does It Take podcast.
0:27 Scott My name is Scott Porad. I'm with my trusty co-host.
0:30 Brad Brad Hefta-Gaub-Gaub.
0:32 Scott And Brad, who is our guest today?
0:35 Brad Today we have Scott Hansen,
0:37 Brad the founder and CTO of Ambic, right?
0:45 Brad Am I getting that right?
0:46 Scott Got it.
0:47 Scott Scott,
0:47 Scott you want to maybe give us a little introduction,
0:49 Scott tell us a little bit about what brought you here,
0:52 Scott the story of how you basically built Ampic.
0:57 Brad Yeah,
0:57 Brad that's a big question.
0:58 Brad So first of all, Ambiq is a company that builds world's most energy efficient chips from the start. The goal has been to put intelligence everywhere.
1:07 Brad And when you want to put intelligence everywhere,
1:09 Brad you know, in your clothing, in your doorknob, in your car and in the bridges that we drive over.
1:14 Brad drive over energy matters right so you want to build the lowest power silicon that can run as as much intelligent compute as you can but on a tiny little battery and and so that's what ampic has done we've shipped 300 million plus units into all kinds of devices that that need intelligence on a power constraining package so think this Garmin watch that I'm wearing has our chip inside of it this
1:38 Brad swoop band that i'm wearing uh has our chip inside of it you could look at a bunch of google uh wearables that use our chip um there's a teardown showing that the the recent uh or a ring five is using our chip so we build really small microprocessors uh ultra low power that that make intelligence on the edge happen and you know look in the last 10 years uh ai has kind of become the the catalyst for a lot of that
2:03 Brad A lot of that,
2:04 Brad right?
2:04 Brad So I founded the company as Ambic. That means ambient IQ,
2:08 Brad ambient intelligence. So we kind of had this inkling that intelligence everywhere was going to be the thing.
2:14 Brad It's with the emergence of deep neural networks and AI that things have really taken off.
2:19 Brad So really,
2:20 Brad it's been a fantastic last 10 years or even five years as that's happened.
2:25 Brad But you asked about the founding.
2:26 Brad So founded the company in 2010,
2:29 Brad going back to University of Michigan,
2:31 Brad I did my PhD there.
2:33 Brad We were, you know, back then we were doing what's called smart dust development. So the professors at Berkeley had this idea that.
2:41 Brad that we should build smart dust and so we at Michigan were wanted wanted to do that so think you know little cubic millimeter devices that have a battery and a radio and a processor and sensors and everything all in one and you know use it for everything we were using it for intraocular pressure sensors so in eye pressure sensors for glaucoma patients super cool super cool stuff and at some point I want to say that they implanted it in a pig's eye or something like that
3:06 Brad that was after I had already left, but really cool volume constrained computing,
3:13 Brad and then that ultimately led to Ambic.
3:14 Brad So we've got this core technology that was developed at the university and then refined here at Ambic, but think of it as a circuit design platform.
3:24 Brad So it relies on a concept called subthreshold computing.
3:29 Scott Yep.
3:29 Brad The basic idea is that
3:32 Brad We're running the transistors, the underlying building blocks of the chip,
3:35 Brad at much lower voltages than would be normal.
3:39 Brad And it turns out to do that, it sounds simple,
3:42 Brad the benefits are amazing because energy is voltage squared.
3:46 Brad So as I drop voltage down,
3:47 Brad I'm getting this quadratic reduction in energy.
3:50 Brad So there are very few, trust me that there are very few knobs in computing.
3:54 Brad that are that sensitive and so it sounds simple drop voltage get energy savings the truth is it's actually quite hard you get become exponentially sensitive to just like temperature variations to manufacturing fluctuations to to voltage changes and so we've had to develop a whole new circuit design platform to make that possible but now that we've done it the cool thing is we can design anything we want right I can build microprocessors that go
4:21 Brad is it going here if I wanted to build chips for automotive I could do that if I wanted to build even chips for the data center we could do that we've got the platform we've done hard part solve the hard physics problems and you know now here at Ambic we're building the chips as a result of that
4:34 Scott And that my understanding is that that was what if I'm correct,
4:38 Scott that was your that was what your Ph.D. was all about was like how to get these, you know, sub threshold compute platform,
4:46 Scott but make them.
4:47 Scott make them actually like reliable and reproducible, I guess is the way I think about it.
4:52 Brad Yeah, so my PhD work was, yeah, it was focused around that.
4:57 Brad We were building microprocessors in academia that were running down to 130 millivolts, right?
5:04 Brad So 0.13 volts,
5:05 Brad which for anybody who builds hardware,
5:07 Brad that sounds crazy, right?
5:08 Brad It's like we're basically using noise to power our microprocessors.
5:12 Brad we did some really cool stuff at the university and it was you know at first it was what's the limit of voltage scaling how low can I take voltage and still compute then it was well how low should we take voltage and then it was okay how do we make it manufacturable and actually most of what we've done at Ambic has been that latter part it's how do we make it manufacturable because there's you know if I had just focused on doing the things we did at University of Michigan we would never have gotten anything to
5:37 Brad to market right because there you know you build one chip you you you make it work at room temperature and they you know they give you a paper and a PhD and say be on your way
5:45 Scott Right.
5:45 Brad we have to make millions of chips here right and and that's a whole different beast and and it's been a lot of fun to figure all that out
5:53 Scott And I want my Garmin to work in the middle of Death Valley as well as at the summit of Mount Everest.
5:59 Brad exactly
5:59 Scott So like temperature variations are important.
6:03 Brad Yep.
6:04 Scott Yeah,
6:04 Scott yeah.
6:05 Scott Okay,
6:05 Scott can I ask a few questions?
6:06 Brad Please.
6:07 Scott The first question I have has to do with animals.
6:10 Brad Perfect.
6:11 Brad Okay.
6:12 Scott Cells, you know, I don't know how to call it, but cells,
6:16 Scott you know, animals,
6:17 Scott creatures,
6:17 Scott I would imagine those cells are operating a very low voltage.
6:22 Scott Did you guys, when you were doing this,
6:24 Scott look at all at how biology to try to take an inspiration for how that whole world works?
6:30 Brad So,
6:30 Brad there is a whole, there's a couple of ways to take this.
6:34 Brad There is a whole branch of subthreshold design in particular that is inspired by the brain in particular. And so, you know, trying to mimic the way the brain works by building, you know, little networks of neurons that kind of look like that.
6:48 Brad And so that is one branch. It's a fairly esoteric branch of research that has never quite gained.
6:54 Brad uh gain a lot of traction um there's a whole other thing which is you know the body is full of of uh electrical signals um can we power ourselves using those electrical signals
7:06 Scott Yeah.
7:06 Brad so that's one of the things that's gotten a lot of attention uh and in fact um whether we look at thermal gradients or you look at at um you know like uh voltages uh flowing through the body there are opportunities to
7:20 Brad to power electronics and that is something that we we for sure looked at we actually had customers early in AMBIC's history that were using thermoelectric generators to generate use body heat basically to power electronics so that is at least one angle that we looked at a lot of interesting things in the body actually the other thing that was always interesting was could we make one of the professors at Michigan was looking could we make
7:44 Brad uh are sensors small enough to actually place them in a cell and and monitor what's going on there and
7:50 Scott It's insane.
7:51 Brad yeah it sounds crazy
7:52 Scott I mean, it depends on the size of the cell, I guess.
7:54 Brad yeah yeah yeah right and i cannot remember i wish i should have done some research on that before coming on with you guys but i can't remember where that landed but yeah a lot of cool opportunities there
8:03 Scott You're saying that, you know, there's this notion of a self-winding watch,
8:08 Scott but,
8:09 Scott you know, with an Apple watch.
8:10 Scott watch or a Garmin watch like you've got it can't that doesn't have any winding to it but you're kind of saying it could use the heat from your wrist to self-wind self-charge
8:20 Brad Yeah.
8:21 Scott that battery theoretically that's possible wow that's amazing okay here's my second question how did you get it this is
8:29 Scott You know, this is pretty cool.
8:30 Scott How did you get into this? How did you find like were you in were you in the eighth grade thinking you wanted to work with little teeny processors?
8:37 Scott Like how did this all come about?
8:38 Brad I was an undergrad.
8:42 Brad in electrical engineering.
8:43 Brad I looked at computer science or electrical engineering and EE spoke to me more than computer science, more than, you know, writing code.
8:51 Brad And that was actually weird, by the way, because at that time, everybody was going software.
8:56 Brad I was in, I was an undergrad from 2001 to 2004.
8:59 Brad And so this was pre-iPhone,
9:01 Brad but shortly after that iPhone immersion, everybody wanted to be, you know, writing code, writing apps and all this stuff.
9:07 Brad So I was it was weird for me to go EE.
9:09 Brad And then I went to do an internship.
9:12 Brad I won't name the company.
9:14 Brad I don't want to I don't want to offend this company.
9:16 Brad I went to a big well-known technology company. I went to do an internship and.
9:21 Scott Acme core,
9:22 Scott Acme
9:22 Brad Yeah,
9:22 Scott core,
9:22 Brad exactly.
9:23 Scott right?
9:24 Scott The Roadrunner and Coyote.
9:26 Brad Exactly.
9:26 Brad I was bored.
9:27 Brad I was bored with my time there.
9:29 Brad By the way, I had this moment.
9:31 Brad Does my LinkedIn profile show this and it does not show this company.
9:33 Brad this company says um you
9:36 Scott know what Scott it's totally okay only 40 people listen to this podcast
9:40 Brad perfect perfect
9:42 Scott oh we're up to like 200 now after the bob
9:45 Brad oh
9:46 Scott martin uh john
9:47 Brad there
9:47 Scott paul
9:47 Brad you
9:48 Scott bob
9:48 Brad go i like that um so i i did the internship um i looked around at people with a master's degree and um they were doing work that i just wasn't at all excited about i thought i'd go
9:59 Brad go to get this amazing master's degree and all I get to do is this work right here and so I came back and I had a the guy that became my thesis advisor and actually eventually my co-founder at Ambic Dennis Sylvester he said hey Scott I'm going to pay for your your master's degree and I think you'll like it so much that you're going to stick around for a PhD and I was kind of like okay I had the bad experience with the with the internship so maybe I'll give that a try and he gave me enough
10:24 Brad Give me an opportunity to work on some really cool stuff. The whole subject of subthreshold computing wasn't something that I invented.
10:30 Brad It had actually been around for decades.
10:31 Scott Like the 70s, right?
10:32 Brad Yeah, exactly.
10:33 Brad So it was think Swiss watch industry was early on it and as well as the Japanese watch industry.
10:41 Brad They were early on that.
10:42 Brad But there was this resurgence of interest in that in the early and mid 2000s. And I was fortunate that I got I got into it.
10:50 Brad And there was a point where, you know, when you're operating at the University of Michigan, there's a ton of research dollars coming in from all kinds of cool companies and they're always coming and visiting and talking to you. And I remember getting asked a lot about it at these,
11:04 Brad you present your poster of your work and a lot of companies are going,
11:07 Brad hey,
11:08 Brad is this, you know, what's the commercial potential here?
11:09 Brad And at some point I heard that enough where I thought,
11:12 Brad okay,
11:12 Brad I actually think this is going to be commercialized. And I do remember.
11:16 Brad being in an elevator at the University of Michigan and thinking you know if if someone's going to if it's going to be commercialized why not it be me right and until then I would say that it would have been for me I was thinking let's I'll go work at a university as a professor or I'll go be a researcher at one of these companies or maybe I'll go to business school I don't even know I had I had a wide range of options but when that when that happened
11:42 Brad I thought,
11:42 Brad OK,
11:42 Brad I want to go start this company.
11:44 Brad I want to make this technology happen.
11:45 Brad And I was in a really fortunate position to do that because I wasn't married.
11:52 Brad uh,
11:53 Brad I, you know, I had no mortgage. My family was such that I knew that if, if everything,
11:59 Brad you know, went poorly and I could go live with them and, and,
12:03 Brad you know, um, uh, uh, cry on their shoulder for a little bit before going to get a job. And so it felt like, Hey, if there's ever a time for me to try this now,
12:11 Brad now is the time.
12:12 Brad Um,
12:12 Brad and then on top of it,
12:13 Brad I had the university's resources to draw upon.
12:16 Brad Right.
12:16 Brad Um,
12:17 Brad One of the bits of advice I always give to young entrepreneurs who are coming out of the university is don't be in a rush to leave.
12:22 Brad You have this fantastic place where they've got every CAD license.
12:26 Brad We use these CAD tools and they're so expensive,
12:29 Brad right?
12:30 Brad And even at Ambic now,
12:32 Brad the number of licenses,
12:35 Brad the number of seats,
12:36 Brad the amount,
12:36 Brad the tools that we can draw upon are so limited compared to what's available at the university.
12:41 Brad They have these beautiful labs.
12:42 Brad I'm like, just stay there,
12:43 Brad do as much.
12:44 Brad Do as much as you can there and then and then spin out when you need to spin out.
12:47 Brad And so I was able to do that. It was great.
12:49 Brad I basically got paid as a.
12:52 Brad Um,
12:52 Brad as a postdoc for a while, I graduated with my PhD,
12:55 Brad I found a good commercialization grant.
12:59 Brad I was getting paid, I won't say how much but it was by today's standards an extremely modest amount, but I felt like I was I had made it right it felt like an amazing
13:07 Scott Yeah.
13:07 Brad thing was,
13:08 Brad I was like getting paid to just go figure out how to build this, you know, so it was it was a really fun time and I miss sometimes I miss those early days.
13:17 Scott Mm-hmm.
13:18 Scott Yeah.
13:19 Scott Okay.
13:20 Scott Brad,
13:20 Scott I got so many more questions. Can I keep going?
13:23 Scott Sure.
13:24 Scott Yeah.
13:24 Scott Yeah.
13:24 Scott Let
13:25 Scott Yeah.
13:25 Scott me just, let me just, let me just ask one more.
13:28 Scott Briefly, for people who are listening to this and might be inspired by you and are thinking maybe they're in a PhD, I'm curious about that spin out process.
13:38 Scott How does that work to take out,
13:40 Scott does the university own the IP?
13:42 Scott Like how does that, how does that all work when you want to?
13:44 Scott go figure something out like that and then go commercialize it.
13:46 Brad Well,
13:47 Brad I'll tell you how it worked for me. And then there's a bunch of other iterations of how it's possible.
13:51 Brad But for me, I had two co-founders that were professors at the university that were my thesis advisors,
13:56 Brad Dennis Sylvester and David Blau. And we had a set of inventions that we wanted to take that I wanted to take with us to go found the company,
14:06 Brad they were going to stay as professors,
14:07 Brad I was going to go found the company as the as the
14:09 Brad the founder is the CEO,
14:11 Brad whatever.
14:11 Brad So we went to the Office of Technology Transfer and we got access to these inventions. They weren't yet even patents,
14:20 Brad right?
14:20 Brad So we hadn't even filed patent applications.
14:22 Brad They were inventions that we wanted to file as patents.
14:24 Brad Office of Tech Transfer at the University of Michigan nominally owns that process.
14:29 Brad They took
14:31 Brad an ownership stake in the company small right in the end they did well we went public last year i'm sure that they did well i should go look and figure out how much money they made but it was successful for them and then and then what we did is we collectively filed patent applications for these inventions and and what we committed to do in addition to giving them an ownership stake in the company is we get we would give them a royalty for every chip that we shipped and
14:55 Brad That continued for a while and they actually had a nice little cash flow going from that and then we ultimately bought them out of that deal.
15:05 Brad And it was, I would say it was a pretty good process.
15:08 Brad They were,
15:09 Brad University of Michigan at the time was not necessarily known as a, you know, entrepreneurial hub.
15:16 Brad If you were going to rattle off places, you would say Stanford,
15:18 Brad right? Or
15:18 Scott Yeah,
15:18 Brad Berkeley
15:19 Scott sure.
15:19 Brad or some of these other companies or universities.
15:22 Brad And so they were going through a process of making making it all easier and so forth. And there were some really good I would say one of the things I did really well personally is I found.
15:34 Brad good advisors who had been there and done it before there was this guy named Dave Hartman who worked at the university he was an entrepreneur who had been successful he worked at the office of tech transfer and he was basically there to be my ally in negotiating with them also it was weird because I was a PhD I was a kid negotiating with the university right I was simultaneously negotiating with my co-founders who were professors who until then had been my bosses right they were friends like today we're
15:59 Brad They were friends, but they were my bosses.
16:01 Brad So negotiating with them, I'm negotiating with the university, but I had this guy, Dave Hartman, who kind of became my advocate and that was critical for me.
16:10 Brad So maybe the lesson,
16:12 Brad the best lesson in all of this is for university students out there, find a good set of advisors,
16:18 Brad listen to them.
16:19 Brad And that kind of allows you to foresee many of the challenges ahead.
16:24 Scott All right. I'm going to take the con here for a second,
16:26 Scott Bo Rad.
16:28 Brad Do it. This is great.
16:29 Scott I love it. I have two questions that are kind of related to this early part of the company.
16:35 Scott But maybe they relate back – at least the first one I want to ask you is about – you mentioned patents, and you clearly were doing research.
16:43 Scott One of the things that –
16:44 Scott Since it is very,
16:46 Scott you know, I'm going to say it's more in vogue now because of sort of the sort of like fervor over AI,
16:53 Scott all of these labs,
16:56 Scott right,
16:57 Scott and internal entities inside of, you know, mostly in corporate environments like Google, et cetera,
17:03 Scott are publishing research papers.
17:06 Scott And I'm kind of curious, how did, I'm assuming that you would.
17:10 Scott had published papers about some of your inventions,
17:12 Scott how did that affect the patenting process,
17:14 Scott right?
17:15 Scott Doesn't that, doesn't that, isn't that like pre-disclosure and doesn't that impact your patenting process?
17:19 Brad Yeah, it does. So you have in the United States, at least when you publicly disclose an invention,
17:25 Brad you have 12 months to file
17:27 Scott Yeah,
17:27 Brad a patent.
17:27 Scott but it closes off international patents,
17:29 Brad It
17:29 Scott right?
17:29 Brad does close off international patents. Yes.
17:31 Scott Yeah.
17:32 Brad So in our case, one of the one of the base patents,
17:38 Brad man,
17:39 Brad I wish I could remember this. I want to say that we either filed before it was published.
17:45 Brad or we only filed in the U.S., and then what we did is we,
17:47 Brad okay,
17:48 Brad once we really thought that the core invention,
17:51 Brad once we really thought about how we would productize it, we realized that there was more circuitry that you need to wrap around it.
17:57 Brad So there was an opportunity to add extra elements
17:59 Scott Sure,
17:59 Brad and make
17:59 Scott totally.
17:59 Brad it internationally patentable.
18:01 Brad So I want to say that we did a sequence of things and ultimately ended up with both U.S. patents and international patents because we added elements.
18:09 Brad And then,
18:09 Brad hey,
18:10 Brad the truth is today.
18:11 Brad Ambic has 90 plus patents that are granted or pending and only two of them were the original two that we licensed
18:19 Scott Yeah,
18:19 Brad out of the university.
18:20 Scott sure.
18:20 Brad I mean, I felt a lot of pressure early on that, oh my gosh, these were the things we invented, I must use them.
18:26 Brad And what I figured out at some point is actually what I need to do is I need to just like continue to expand,
18:33 Brad I need to investigate new directions.
18:35 Brad If that turns out to be the wrong direction.
18:36 Brad shouldn't have no emotion about tossing it aside the
18:38 Scott Yeah.
18:39 Brad cool thing is we actually still use those those two circuits today and we're you know 16 years into this thing oh
18:46 Scott Yeah.
18:47 Scott Obviously, and if you've got a great patent team that knows how to do strategy around that,
18:51 Scott then like I'm getting patents filed in my name for a company that I haven't worked for in 20 years because like
18:58 Brad my god
18:59 Scott they've they've been just doing the extensions and figuring out.
19:02 Scott out extra things and it's like oh my god so
19:04 Brad Sure.
19:04 Scott it's kind of funny in that regard but my next question kind of relates to um you started out as the founder ceo now you're the cto um we've kind of talked a little bit in the past and on other episodes in the podcast we've talked to folks that have kind of transitioned in that way and i'm just kind of curious if there's anything you feel like you can share to other entrepreneurial folks that about
19:27 Scott You know, the what that shift feels like, how you made that decision.
19:32 Scott I mean, there's I'm sure everybody's story is a little different there,
19:35 Scott but I'm just kind of curious if you could speak a little bit about that journey for you.
19:38 Brad Yeah, it was an interesting one.
19:41 Brad I started being CEO for the first couple of years and very early on I established with the investors,
19:48 Brad the venture capitalists that came on board that I would be happy to step aside and be CTO when the time is right because I acknowledge I was a kid. I didn't know anything.
19:55 Brad I didn't know what a CEO is supposed to do. Right.
19:59 Brad And at some point, you know, they start giving me good feedback. Hey,
20:02 Brad Scott,
20:03 Brad you're the needs of what we need out of it or the list of what we need out of a CEO is kind of growing,
20:08 Brad but you're keeping up with that growth curve.
20:10 Brad Right.
20:11 Brad And we like what you're doing. And I started thinking,
20:13 Brad OK,
20:13 Brad maybe this is long term for me. And, you know, your ego gets attached to being the CEO of the company. And maybe more importantly, as a CEO,
20:20 Brad yes,
20:21 Brad you report to the board, but ultimately you're the top dog and you're the one that gets to set.
20:24 Brad to set the culture and everything else and then at some point they came to me the the board said hey we got to grow here right and and we got to get a professional ceo and i was disappointed my first it was it was immense disappointment right and which was funny because i wouldn't have predicted that a couple years before i just gotten attached to the job um and you know they see that and what they said to me was or they saw that and what they said to me was hey look
20:51 Brad You've got two choices here. You can be the CEO.
20:53 Brad We can go find a CTO for the company because we need one.
20:57 Brad And or you can go be the CTO and we can find a CEO.
21:01 Brad And the truth was there were thousands of people that could have been a great CEO for the company. I was probably the only, if not one of five people that could have been the CTO for this company.
21:11 Brad And so that made sense to me.
21:13 Brad My logical mind says,
21:14 Brad okay,
21:14 Brad that makes sense.
21:15 Brad We brought on a guy who was, in hindsight, he was a very successful CEO,
21:21 Brad had done it before.
21:22 Brad And we clashed immediately.
21:24 Brad Ultimately today,
21:25 Brad I would say we have a good relationship.
21:26 Brad This is not our current CEO.
21:27 Brad This is a guy who was in between our current CEO and me.
21:31 Brad You know, there was some real tension at the early part of it.
21:35 Brad There was a good potential for him leaving given how I was kind of continue to hold on to Yeah. the reins. And what I realized at some point after a stern talking to by one of my.
21:47 Brad close advisors.
21:50 Brad What I realized is that I had to release a little bit of control.
21:54 Brad I can still be a strong influencer of the culture.
21:57 Brad I can still be driving technology roadmap and all these things, but ultimately I needed to let him be CEO.
22:02 Brad And that means I report to him, right?
22:04 Brad I was used to being, again,
22:05 Brad the top dog and now I've got a boss,
22:07 Brad right?
22:08 Brad And so what I'll say is that our current CEO is a guy that has been at the company now,
22:14 Brad Humiyosaka, who's been at the company now.
22:16 Brad now for more than 10 years and I think that's worked in part because I had that first go-round with the other CEO I it went poorly at first we figured it out
22:26 Scott You had a test model.
22:27 Brad yeah yeah and so I you know we've made it work so now we've got a good mutual respect relationship but the truth is he's the CEO he's my boss right and so you know he he's in charge now what I will say to
22:41 Brad CTOs out there looking for a CEO is find somebody who respects you and treat you treats you like a partner I've seen different versions of that I've seen founder CTOs who've you know had CEOs that that you know don't really care what what their opinion is and all that and I have a CEO who treats me like a partner right we in a closed door session we can have really good discussions about what's going on how we're going to attack things but
23:07 Brad outside that room, I have to remember again,
23:08 Brad he's my boss.
23:09 Scott Yeah.
23:10 Scott Well,
23:10 Scott thanks for sharing that, by the way.
23:11 Scott I thought I was, you know, I didn't know what your story was going to be exactly.
23:16 Scott And it's awesome to hear you kind of like be genuine about like what that felt like, right?
23:22 Scott You know, you're
23:22 Brad It's
23:22 Scott the founder,
23:24 Scott you know, this was your baby,
23:26 Scott you know, it is your baby.
23:27 Scott And so, yeah,
23:28 Scott that's awesome.
23:29 Scott I love the advice that you think you're giving,
23:31 Scott you know, like find someone that's really going to treat you like a partner.
23:35 Brad Yeah.
23:36 Scott There are many founder CEOs who can't make that transition and their companies suffer or fail as a result of it. And it's admirable that you're able to do it.
23:48 Brad Yeah,
23:48 Brad I appreciate that.
23:49 Brad Listen,
23:49 Brad I distinctly remember getting that stern talking to by this close advisor who was, I want to say at the time he was also a board member and I went home and laid on the couch.
24:03 Brad and complain to my wife and as I was just talking she's by the way she's like my in-house psychologist she actually has a degree and a master's in psychology and so she's the most empathetic person I met she we can sit there we can have these conversations and as I was talking it out with her I realized how much in the wrong I was so yeah that was a good learning experience
24:23 Scott Well, it's good that you have a, you know, you had the support network that
24:26 Scott Yeah,
24:26 Scott you
24:27 Scott yeah,
24:27 Scott had, you know, in your advisor and in your wife to be able to kind of help you through that transition, which obviously was is hard.
24:33 Scott It's a hard transition.
24:34 Scott It's got to be a hard transition,
24:35 Scott right? You got it's your ego is going to be so so much tied into everything about it.
24:41 Scott So I
24:41 Scott yeah.
24:41 Scott love it.
24:42 Scott I want to pivot to let's talk about your tech.
24:45 Scott Let's talk a little bit more about what you guys have been working on. I know.
24:48 Scott I know recently,
24:49 Scott I guess maybe in January, you announced sort of this new product that you're focused on as a neural processing unit that you're targeting.
25:00 Brad Yep.
25:00 Scott You want to tell us a little bit about that?
25:03 Brad Yeah.
25:04 Brad So let me start by saying we have this flagship family that we call Apollo. It's a system on chip that has a microprocessor core.
25:10 Brad It has a radio GPU.
25:12 Brad for graphics,
25:14 Brad not for AI,
25:14 Brad but for graphics,
25:15 Brad as well as some lightweight AI acceleration.
25:18 Brad But what we saw our customers doing with Apollo...
25:22 Brad was increasingly AI workloads and and and at first that we had more than enough hardware to support that but as we started looking at their roadmaps we saw the need to have neural processing unit inside so to have specific hardware acceleration to take on really big models actually years ago I would say it's probably 2019 2020 we first had the idea of building this chip with which ultimately became atomic that has all this
25:47 Brad all these processors the memory and all that but also the neural processing unit but the customers weren't ready at the time right they were kind of dipping their toes in in edge AI and bringing neural networks out of the cloud and out of these devices and they just had the lightest weight of needs but as their roadmap started taking shape we saw that there was this need for atomic and atomic is the compute horsepower that this thing has is crazy if you had told me
26:13 Brad When I founded the company that I was going to build a chip like this, I would say no way.
26:16 Brad But Moore's law is a pretty wild thing, right?
26:19 Brad I mean, the number of transistors we can pack onto these chips,
26:22 Brad even chips that are using 10-year-old manufacturing technology can have an enormous amount of compute on board.
26:30 Brad And so Atomic came from that.
26:33 Brad I've got a ton of memory on board,
26:35 Brad some really cool memory innovations.
26:37 Brad You know, there's been a lot of talk in the cloud about,
26:39 Brad hey,
26:40 Brad if I want to run whatever my latest large language model is in the cloud,
26:44 Brad memory becomes the key bottleneck from a performance standpoint,
26:47 Brad from a supply chain standpoint,
26:48 Brad from an energy standpoint.
26:49 Brad All those same things are true at the edge. When I want to put, you know, an AI model on a device like this or like this,
26:56 Brad I care a lot about memory. So Atomic includes that neural process.
27:00 Brad neural processing unit as well as some some really cool memory innovations but maybe to quantify for you if with Apollo the latest Apollo that you can find on store shelves today and a bunch of our customers products you can support a an AI model that's maybe 2 million parameters okay which by by cloud standards is nothing right Yeah, it's tiny but
27:19 Scott sure.
27:19 Scott But it's
27:19 Brad our customers
27:20 Scott in your wrist.
27:20 Brad that's yeah I mean for our customers though it's a it's a you know it's amazing
27:25 Brad With Atomic, you'll be able to build models that have in excess of 200 million parameters,
27:30 Brad so 100x, 100-fold increase in the complexity of model.
27:34 Brad And so what that means is our customers can take on vastly more complex tasks,
27:40 Brad right?
27:40 Brad So if today,
27:41 Brad if I break open my typical customer's device,
27:44 Brad whether it's a wearable like this or it's a smart home sensor or whatever.
27:49 Brad They're doing things like fall detection,
27:51 Brad you know, is the person falling?
27:53 Brad They're doing heart health analysis and activity recognition.
27:59 Brad They're doing leak detection,
28:02 Brad right?
28:02 Brad There's a water sensor and it's doing leak detection.
28:04 Brad They're doing anomaly detection,
28:05 Brad figuring out, hey, this piece of industrial machinery,
28:08 Brad the vibration signature just changed. What's going on, right? But with Atomic, when I jump to that next generation where I can do 200 million.
28:15 Brad million parameters i can add a real conversational voice user interface to these things so i can talk naturally my much like you might talk to chat gpt you're not going to be able to do the wide breadth of things you can do with chat gpt but hey there's only like i don't know 500 things i might do with my watch or my band or my my smart ring or whatever and so
28:37 Brad That is feasible in that.
28:38 Brad And that's a big deal, right?
28:41 Brad And there's all kinds of other use cases that our customers have envisioned.
28:44 Brad But I am really excited about what that enables.
28:47 Brad The other thing that Atomic makes possible.
28:51 Brad is until now we have largely not supported cameras.
28:54 Brad Cameras generate so much data,
28:56 Brad they require so much compute and so much AI that it's been impractical for us to do it. And more importantly,
29:01 Brad I just don't have the plumbing on my chips to bring a video feed onto my chips. And so Atomic will have variants that are powerful enough to process all that video data,
29:12 Brad but also will have the plumbing needed to bring that data on board. Think about
29:16 Brad AR glasses or
29:17 Scott Yeah.
29:17 Brad smart glasses,
29:19 Brad I see a ton of really cool and aggressive roadmaps from companies in that space and from our customers in that space.
29:24 Brad So I'm really excited about what's coming there.
29:26 Brad And then also think about cameras of all sorts, whether security cameras,
29:30 Brad doorbell cameras,
29:31 Brad hunting cameras and outdoor cameras,
29:33 Brad all kinds of really cool use cases where you want to do processing there on the edge. You want to detect motion and track things and track gestures and track.
29:42 Brad track your eyes and all that.
29:44 Brad So a lot of really cool opportunities there that we think Atomic will make possible.
29:49 Scott So one of the things that immediately jumps to my mind, I love that list of your laundry list there.
29:54 Scott What about like, you know, you know.
29:58 Scott hearing aids with full translation support,
30:01 Scott right?
30:01 Brad Absolutely.
30:02 Scott Like I'm assuming you could, could you do that sort of thing before Atomic or really Atomic is what's making that sort of thing possible?
30:09 Brad So some of our customers products do that today.
30:15 Brad So actually a product I really like is called Even Realities. It's a smart glasses, super slim, and they're using our product.
30:25 Brad um they have a heads up display built in we have a gpu so we can we can kind of manage display and do the animation and all that uh and i want to say that they they public they i believe that they support translation i'm not 100 sure
30:39 Scott But
30:39 Brad on that are but no
30:39 Scott they doing it on the device or are they paired with a separate device?
30:42 Brad yeah yeah that would definitely not be done on device so there's a certain you know experience there so how does that experience change if you can do that on device i don't know that's it
30:50 Brad That's a that's a great question. And I do think that that will be possible at a minimum you can what we can do on device is we can you know we can go from.
31:01 Brad you know a sound wave to like phonemes we can we can convert you know basically do a speech maybe even a speech to text translation there where where then you're just sending a text stream instead of a raw audio stream so that opens up a lot of possibilities there but yeah I would say that's one of the definitely one of the coolest applications I also really like there's a whole class of devices today that's you doing audio only and it's
31:26 Brad It's taking transcripts and notes and all that.
31:29 Brad And so, you know, that could be in a glasses form factor where we're all talking, it's recording what we're saying,
31:34 Brad and then it sends me a summary of the key things we've learned afterwards.
31:37 Brad Of course, you know, teams and other programs already offer that capability, but not necessarily in the real world.
31:44 Scott Real-time closed captioning for every conversation I ever have in my glasses.
31:49 Brad That would be cool, right?
31:51 Brad Yeah.
31:51 Scott Yeah.
31:52 Scott I mean, it sounds like you feel like the atomic platform has a sort of shape to it that those sort of applications start to become practical.
32:04 Scott Absolutely,
32:05 Scott yeah.
32:06 Scott Yeah.
32:07 Scott Very cool.
32:08 Scott Have either of you ever read a book called Super Sad True Love Story?
32:13 Scott Oh, you've talked about this on the podcast before,
32:16 Scott Scott,
32:16 Scott and I haven't read it yet.
32:19 Scott it's a novel and it's set in the future and some of the things that you're talking about or that you're doing with Atomic like you know this book's probably 15 years old and at the time well the iPhone had come out when the book was written but nevertheless the point is is like all of a sudden some of these you know space age things that they were talking about in the book are becoming real
32:40 Scott Yeah.
32:41 Brad Yeah.
32:41 Scott That's what it reminded me of.
32:42 Scott All right. Here's a question I have for Atomic. So one of the things that we see happening in sort of like the desktop space,
32:47 Scott people that are like trying to go to local inference,
32:50 Scott right?
32:50 Scott There's a handful of different.
32:54 Scott research papers and implementations where we've taken like super large models of mixtures of experts and doing this thing where we're sort of like streaming it like we're only lighting up the experts that matter for the tokens that you're processing and sort of bringing in the rest of the experts only as you know the tokens space moves past that and i think that one of the and of course what we see in the desktop space is where that
33:19 Scott the bottleneck ends up becoming this sort of like
33:22 Scott the lesser of how much available memory do you have versus the speed of your memory,
33:28 Scott your cached memory bus into your active memory.
33:32 Scott And I'm just kind of curious, like, how does that apply to below threshold compute?
33:37 Scott Like, it feels like we've got this tension of like the speed of below threshold compute being in the megahertz. And so
33:45 Brad Yeah.
33:45 Scott I'm just kind of curious, have you guys started to
33:48 Scott Sorry to think about that or.
33:49 Brad Yeah,
33:50 Brad there's a lot. So there's a lot of things here.
33:51 Brad So first of all, one of the beautiful things about AI compute is that it's highly parallelizable.
33:55 Brad So as we yes,
33:57 Brad we run at lower we tend to run at lower frequencies than you would in conventional voltages that that's improved immensely as we've gotten a more advanced manufacturing nodes.
34:07 Brad But AI is super parallelizable. So these neural processing units can do a ton of
34:13 Brad of processing in parallel so I can you know you can you can do a couple hundred max multiply accumulates every single clock cycle right and so actually then the problem becomes how do I keep those multiply accumulate units full how do I how do I provide enough data via the bus to do that and so that does tend to be the bottleneck even for our for our style of processor memory bandwidth does get to be a bottleneck if you're not careful now
34:39 Brad Now,
34:39 Brad the cool thing is because, yes,
34:42 Brad we're running wide,
34:43 Brad but we're running at low, relatively low frequency, so you kind of go low and slow.
34:48 Brad What you can do is you can run memories at relatively higher speeds and higher voltages and just kind of fill up the bandwidth that way.
34:57 Brad And so it ends up perhaps not being a problem. Now the problem is actually it's one of energy. It's not a bandwidth problem because I can meet.
35:04 Brad can meet that.
35:05 Brad It's one where I've brought the energy of the processor down so low that suddenly the memory is sticking out as an energy problem,
35:12 Brad right? Yep. It's whack-a-mole, right?
35:13 Brad I've knocked down one mole and I've got one that's popped up.
35:16 Brad And so the bigger problem is how do I bring the energy of that memory down?
35:20 Brad And we are, let's just say that we're investing on a couple of different fronts to solve that problem.
35:25 Brad It is a problem that is being solved and it will be a couple steps.
35:30 Brad steps we've got in atomic some really cool innovations coming and then post atomic we have some other ones coming so there are a whole variety of other technologies that are out there too that that are being talked about so one of the approaches that it gets talked about is so-called compute in memory and the idea is that
35:47 Brad You know, rather than having memory over here and your processor over here, let's merge them into one unit.
35:52 Brad And there's there's so-called near memory compute where you just you've got a memory cell and you've got your compute cell and they just are very closely located and and that works.
36:01 Brad There are approaches that are analog in nature where you basically you kind of reconfigure the memories to be like analog circuits and they do they do the multiple.
36:13 Brad multiply an accumulation in an analog domain and in that way they can be ultra efficient you can like one bit of memory can be configured to represent an eight bit weight or an eight bit activation or whatever that's super cool it's probably one of the hardest um compute problems i've ever seen and there's a number of companies that are trying to do this go check out mythic is a actually a startup here in austin that was founded by a couple friends of mine and they do some really
36:38 Brad really cool stuff on that front so they're taking a super creative approach to attacking the memory uh bandwidth problem in ai uh and and it's just one that is immensely complex our approach has been a little less aggressive but but perhaps a little bit more pragmatic and realistic and achievable in the short term so definitely a big a big issue
37:00 Scott Very cool.
37:01 Scott It's so funny because I told you beforehand that I'm not an
37:05 Scott Not a double E guy,
37:05 Scott but I'm able to read enough to be like, ooh,
37:08 Scott that sounds like an interesting problem. How does one solve that?
37:11 Scott So there you go. I love you entertaining the teaching the CS guy a little bit about the EE problems.
37:20 Scott What I am curious to know about as you're talking about this.
37:24 Scott I've always been an engineering leader for software companies they're online software companies and you know there are a set of say management or leadership principles I have about how to
37:37 Scott do that to evolve over the years right and I'm just curious to know like what's it like to leave I'm wondering if they're different for a chip company is ultimately what I'm wondering or if you know like the first note I have this thing that I've been writing notes of questions to ask as we've been talking the first thing I asked was like is there tech debt does that exist in your world Yeah, yeah
37:57 Scott what's tech debt look like in hardware?
38:01 Scott yeah yeah yeah absolutely absolutely
38:03 Brad Um,
38:04 Brad first of all, a lot of the hardware development process these days is just software development. So I'll be a little more specific there.
38:11 Brad So, you know, if you break open our chips,
38:13 Brad there's maybe three classes of circuit inside there. There's you got digital circuits,
38:19 Brad you got analog circuits and you got memory and digital circuits are largely coded today,
38:25 Brad right? You write Verilog.
38:27 Brad register it's rtl it's a it's basically code that you guys would probably understand if i showed it to you'd probably get to some extent what's going on there um you can write you know c equals a times b and it you know will synthesize a multiplier so a lot of a lot of um chips these days are developed in that way so a lot of the the concepts that you might have about software development are going to apply here um that said there's this one major difference with hardware development
38:52 Brad development.
38:53 Brad And that is that a compile is
38:56 Scott Mm-hmm.
38:57 Brad me turning
38:59 Brad That code into a chip, getting super expensive masks made,
39:05 Brad getting it manufactured,
39:06 Brad and then checking if it worked, right?
39:07 Brad And that is a,
39:08 Brad you know, the design is a 12-month process. You've got a couple months of manufacturing and you've got a couple months of bring up.
39:15 Brad That's a long cycle. Whereas,
39:17 Brad you know, I laugh because the software guys,
39:19 Brad they hit compile and then if it didn't work,
39:21 Brad they just recompile. They fix it and recompile,
39:23 Brad right?
39:23 Brad It's a very different.
39:24 Brad So the concept of agile,
39:27 Brad for example, I think makes a ton of sense in the software world.
39:30 Brad Mm It has
39:31 Scott -hmm.
39:31 Brad to be modified somewhat in the hardware world.
39:33 Brad So you would probably look at hardware development methodologies and think,
39:38 Brad oh, wow,
39:38 Brad that is very different. It's maybe a little bit unique,
39:40 Brad but it's the right thing to do. I will say that.
39:44 Brad AI is completely changing the way we're, you know, approaching a lot of this, or at least it will dramatically change things. So a lot of the same uses that you're seeing with coding agents on the software side apply to the hardware side. So we can generate Verilog, we can generate digital circuits.
39:59 Brad using AI agents but there's a whole bunch of other uses there that that that are exciting so I do think that hardware design is going through the design flow is going through a bit of a revolution that's gonna who knows where it will go but it's gonna be fun few years
40:15 Scott And are there like, I assume there would be, but maybe I'm maybe I'm wrong about this, like clearly when you compile to the point that you're actually, you know, doing the manufacturing like it's like you better.
40:26 Scott You got a long lead cycle there,
40:28 Scott but do you have like software test beds to test your Verilogic and stuff like that? Like you
40:35 Brad Yeah,
40:35 Scott get you
40:35 Brad we
40:35 Scott get shorter cycles on that kind of stuff.
40:38 Brad have, of course, simulators of various types that can kind of simulate various stages of the design and you can look at waveforms and debug and we'll, for example, a lot of our processes,
40:49 Brad we build.
40:52 Brad We build the hardware, we write C code that effectively executes on the processor and then we simulate the processor running through that code and see if it does what we expected.
41:02 Brad that's running on you know computers it's very time intensive and there's only so many cycles you can run right so so it's it's limited in its in its value the other big thing we do is we just take fpgas you guys know what fpgas are okay
41:13 Scott I don't.
41:13 Brad so fpga is a is a chip that i can basically configure to do whatever i want it's a super malleable thing it's got a bunch of let's think about it's got a bunch of lego pieces that i can assemble in whatever way i want and then and then go so what i can do is i can
41:27 Brad I can upload an image of my chip of my Verilog onto that FPGA and it's this giant thing right I mean the board is it's this big it's got a fan attached to it like this is a heavy duty piece of silicon but I can upload my design onto it and then run software my software team is out there right now running code on an FPGA representation of our processor and maybe it runs at one-tenth the speed of the ultimate processor or one one-hundredth
41:53 Brad hundred but that's a whole lot faster than simulation world so we can actually get through a lot more many more testing cycles as a result of that we even have customers that will do that right so we we always have what we call an alpha customer who signs up with us early as a for a chip
42:10 Brad And what we'll do in some cases is we will give them that FPGA and say,
42:14 Brad all right, you guys want to start early pre-silicon development.
42:16 Brad Let's go, you know, get your operating system going,
42:18 Brad get your drivers going,
42:19 Brad whatever you want to do.
42:20 Brad And so that is a great way to build, you know, close relationship with the customer.
42:24 Scott That makes a ton of sense.
42:26 Scott She found ways to shorten the cycle,
42:29 Scott but ultimately the cycle is long.
42:30 Scott It's got very long feedback loops in it.
42:33 Brad It is. It is. Yeah.
42:34 Brad And again, back to AI,
42:35 Brad the hope would be that we can shorten that feedback loop. You can do more parallel experiments.
42:40 Brad You can you can make better use of 24 hours in the day to be doing assessment.
42:45 Brad My engineers have to go sleep and an AI agent that is run by an engineer doesn't necessarily have to do that.
42:50 Brad So, yeah,
42:51 Brad loops are long.
42:52 Brad I do think that they will shorten.
42:54 Brad The other problem is that these mask sets are really expensive. So the tooling that we need to manufacture our chip.
42:58 Brad chips is is really expensive and I can't disclose specific prices here but you know think for the most advanced nodes you're you're probably looking at tooling that's in the tens of millions of dollars so every time you do a a chip manufacturing run where you're changing you know all the layers it's it's tens of millions of dollars and um that's prohibitive and that's it's it's changed the nature of chip startups you know when I founded this company
43:23 Brad back in 2010 nobody wanted to invest in chips because of exactly this problem because there was this perception that hey any new chip company you need to raise a couple hundred million dollars because of all the tooling because of the expensive CAD tools and I at the time I convinced my investors that hey we're this edge computing thing which we weren't really calling edge back then but but you know the intelligence everywhere vision we were going to be using older manufacturing nodes that were cheaper and
43:49 Brad and uh and you know the tool the cad tools were going to be cheaper and and they all bought into that and then i turned out i was totally wrong right it turns out that we actually also had to do the more's law thing get down to the advanced nodes and it got expensive you know fortunately it worked out and we were we were successful but uh it's expensive to build chip companies i mean i talked to you know chip companies that that made it back in the early 2000s they raised you know one-tenth of what we raised
44:15 Brad ways to get where they needed to go because the tooling was cheaper the chips were less complex and it's just it's a different ball game today right
44:22 Scott Well, they weren't trying to put their chips inside somebody's eyeball.
44:25 Brad that too that too
44:27 Scott That might be the big difference there.
44:31 Scott I love it. I love it.
44:32 Scott Well, Scott,
44:33 Scott this has been a delightful conversation.
44:36 Brad I appreciate it.
44:37 Scott Thank you very much for taking the time to come on today.
44:40 Scott Brad,
44:40 Scott I don't know if you had any final questions to
44:42 Scott No,
44:42 Scott ask I before
44:43 Scott was
44:43 Scott we wrap
44:43 Scott just going to
44:43 Scott up.
44:43 Scott say, I know I was going to say I appreciate it.
44:45 Scott This is I I think your tech is really cool.
44:49 Scott I love your story. You know, this it's wild that this is your first job.
44:54 Brad Yeah, it should be my last one. That's what I tell everybody is, hey,
44:57 Brad this is the only one. I want this to be the only one.
44:59 Scott There you go.
45:00 Scott That'd be sweet.
45:00 Scott There you go.
45:01 Scott Well,
45:02 Scott thank you very much for coming on today.
45:04 Scott We really appreciate it.
45:05 Scott And good luck.
45:06 Scott Yeah.
45:07 Brad I appreciate it, guys. Thanks for having me. It was fun.
45:09 Scott Awesome. And what do we say, everybody?
45:10 Scott We always say smash that subscribe button,
45:13 Scott Smash
45:14 Scott rate
45:14 Scott that subscribe
45:14 Scott and
45:14 Scott button.
45:14 Scott tell
45:14 Scott Rate and review.
45:15 Scott your mom.
45:16 Scott Tell your mom.
45:18 Scott mom's big part of our audience.
45:21 Scott Anyway,
45:22 Scott nice meeting you,
45:22 Scott Scott.
45:23 Scott Have a good one.
45:24 Scott This episode of the How Many CTOs Does It Take podcast was produced and edited by Saffron Heptagow.
45:30 Scott Our theme song was AI-generated by Brad.
45:33 Scott Please follow us on all the social media platforms at HowManyCTOsPod.
45:38 Scott You can find us on YouTube,
45:40 Scott LinkedIn,
45:41 Scott Instagram,
45:42 Scott Facebook,
45:42 Scott ThreadsX.
45:43 Scott TikTok and Patreon or at our website at howmanycto'spod.com and remember to rate, review, and subscribe on the podcast platform of your choice.
45:53 Scott Thanks for listening.