From Academia to Industry: Scott Hanson on the Future of AI and Edge Computing

0:00 Brad: First of all, a lot of the hardware development process these days is just software development.

0:17 Scott: CTOs will share the stories untold.

0:20 Brad: Hello, everybody.

0:22 Welcome to another week of the How Any CTOs Does It Take podcast.

0:27 Scott: My name is Scott Porad. I'm with my trusty co-host.

0:30 Brad: Brad Hefta-Gaub-Gaub.

0:32 Scott: And Brad, who is our guest today?

0:35 Brad: Today we have Scott Hansen,

0:37 the founder and CTO of Ambic, right?

0:45 Am I getting that right?

0:46 Scott: Got it.

0:47 Scott,

0:47 you want to maybe give us a little introduction,

0:49 tell us a little bit about what brought you here,

0:52 the story of how you basically built Ampic.

0:57 Brad: Yeah,

0:57 that's a big question.

0:58 So first of all, Ambiq is a company that builds world's most energy efficient chips from the start. The goal has been to put intelligence everywhere.

1:07 And when you want to put intelligence everywhere,

1:09 you know, in your clothing, in your doorknob, in your car and in the bridges that we drive over.

1:14 drive over energy matters right so you want to build the lowest power silicon that can run as as much intelligent compute as you can but on a tiny little battery and and so that's what ampic has done we've shipped 300 million plus units into all kinds of devices that that need intelligence on a power constraining package so think this Garmin watch that I'm wearing has our chip inside of it this

1:38 swoop band that i'm wearing uh has our chip inside of it you could look at a bunch of google uh wearables that use our chip um there's a teardown showing that the the recent uh or a ring five is using our chip so we build really small microprocessors uh ultra low power that that make intelligence on the edge happen and you know look in the last 10 years uh ai has kind of become the the catalyst for a lot of that

2:03 A lot of that,

2:04 right?

2:04 So I founded the company as Ambic. That means ambient IQ,

2:08 ambient intelligence. So we kind of had this inkling that intelligence everywhere was going to be the thing.

2:14 It's with the emergence of deep neural networks and AI that things have really taken off.

2:19 So really,

2:20 it's been a fantastic last 10 years or even five years as that's happened.

2:25 But you asked about the founding.

2:26 So founded the company in 2010,

2:29 going back to University of Michigan,

2:31 I did my PhD there.

2:33 We were, you know, back then we were doing what's called smart dust development. So the professors at Berkeley had this idea that.

2:41 that we should build smart dust and so we at Michigan were wanted wanted to do that so think you know little cubic millimeter devices that have a battery and a radio and a processor and sensors and everything all in one and you know use it for everything we were using it for intraocular pressure sensors so in eye pressure sensors for glaucoma patients super cool super cool stuff and at some point I want to say that they implanted it in a pig's eye or something like that

3:06 that was after I had already left, but really cool volume constrained computing,

3:13 and then that ultimately led to Ambic.

3:14 So we've got this core technology that was developed at the university and then refined here at Ambic, but think of it as a circuit design platform.

3:24 So it relies on a concept called subthreshold computing.

3:29 Scott: Yep.

3:29 Brad: The basic idea is that

3:32 We're running the transistors, the underlying building blocks of the chip,

3:35 at much lower voltages than would be normal.

3:39 And it turns out to do that, it sounds simple,

3:42 the benefits are amazing because energy is voltage squared.

3:46 So as I drop voltage down,

3:47 I'm getting this quadratic reduction in energy.

3:50 So there are very few, trust me that there are very few knobs in computing.

3:54 that are that sensitive and so it sounds simple drop voltage get energy savings the truth is it's actually quite hard you get become exponentially sensitive to just like temperature variations to manufacturing fluctuations to to voltage changes and so we've had to develop a whole new circuit design platform to make that possible but now that we've done it the cool thing is we can design anything we want right I can build microprocessors that go

4:21 is it going here if I wanted to build chips for automotive I could do that if I wanted to build even chips for the data center we could do that we've got the platform we've done hard part solve the hard physics problems and you know now here at Ambic we're building the chips as a result of that

4:34 Scott: And that my understanding is that that was what if I'm correct,

4:38 that was your that was what your Ph.D. was all about was like how to get these, you know, sub threshold compute platform,

4:46 but make them.

4:47 make them actually like reliable and reproducible, I guess is the way I think about it.

4:52 Brad: Yeah, so my PhD work was, yeah, it was focused around that.

4:57 We were building microprocessors in academia that were running down to 130 millivolts, right?

5:04 So 0.13 volts,

5:05 which for anybody who builds hardware,

5:07 that sounds crazy, right?

5:08 It's like we're basically using noise to power our microprocessors.

5:12 we did some really cool stuff at the university and it was you know at first it was what's the limit of voltage scaling how low can I take voltage and still compute then it was well how low should we take voltage and then it was okay how do we make it manufacturable and actually most of what we've done at Ambic has been that latter part it's how do we make it manufacturable because there's you know if I had just focused on doing the things we did at University of Michigan we would never have gotten anything to

5:37 to market right because there you know you build one chip you you you make it work at room temperature and they you know they give you a paper and a PhD and say be on your way

5:45 Scott: Right.

5:45 Brad: we have to make millions of chips here right and and that's a whole different beast and and it's been a lot of fun to figure all that out

5:53 Scott: And I want my Garmin to work in the middle of Death Valley as well as at the summit of Mount Everest.

5:59 Brad: exactly

5:59 Scott: So like temperature variations are important.

6:03 Brad: Yep.

6:04 Scott: Yeah,

6:04 yeah.

6:05 Okay,

6:05 can I ask a few questions?

6:06 Brad: Please.

6:07 Scott: The first question I have has to do with animals.

6:10 Brad: Perfect.

6:11 Okay.

6:12 Scott: Cells, you know, I don't know how to call it, but cells,

6:16 you know, animals,

6:17 creatures,

6:17 I would imagine those cells are operating a very low voltage.

6:22 Did you guys, when you were doing this,

6:24 look at all at how biology to try to take an inspiration for how that whole world works?

6:30 Brad: So,

6:30 there is a whole, there's a couple of ways to take this.

6:34 There is a whole branch of subthreshold design in particular that is inspired by the brain in particular. And so, you know, trying to mimic the way the brain works by building, you know, little networks of neurons that kind of look like that.

6:48 And so that is one branch. It's a fairly esoteric branch of research that has never quite gained.

6:54 uh gain a lot of traction um there's a whole other thing which is you know the body is full of of uh electrical signals um can we power ourselves using those electrical signals

7:06 Scott: Yeah.

7:06 Brad: so that's one of the things that's gotten a lot of attention uh and in fact um whether we look at thermal gradients or you look at at um you know like uh voltages uh flowing through the body there are opportunities to

7:20 to power electronics and that is something that we we for sure looked at we actually had customers early in AMBIC's history that were using thermoelectric generators to generate use body heat basically to power electronics so that is at least one angle that we looked at a lot of interesting things in the body actually the other thing that was always interesting was could we make one of the professors at Michigan was looking could we make

7:44 uh are sensors small enough to actually place them in a cell and and monitor what's going on there and

7:50 Scott: It's insane.

7:51 Brad: yeah it sounds crazy

7:52 Scott: I mean, it depends on the size of the cell, I guess.

7:54 Brad: yeah yeah yeah right and i cannot remember i wish i should have done some research on that before coming on with you guys but i can't remember where that landed but yeah a lot of cool opportunities there

8:03 Scott: You're saying that, you know, there's this notion of a self-winding watch,

8:08 but,

8:09 you know, with an Apple watch.

8:10 watch or a Garmin watch like you've got it can't that doesn't have any winding to it but you're kind of saying it could use the heat from your wrist to self-wind self-charge

8:20 Brad: Yeah.

8:21 Scott: that battery theoretically that's possible wow that's amazing okay here's my second question how did you get it this is

8:29 You know, this is pretty cool.

8:30 How did you get into this? How did you find like were you in were you in the eighth grade thinking you wanted to work with little teeny processors?

8:37 Like how did this all come about?

8:38 Brad: I was an undergrad.

8:42 in electrical engineering.

8:43 I looked at computer science or electrical engineering and EE spoke to me more than computer science, more than, you know, writing code.

8:51 And that was actually weird, by the way, because at that time, everybody was going software.

8:56 I was in, I was an undergrad from 2001 to 2004.

8:59 And so this was pre-iPhone,

9:01 but shortly after that iPhone immersion, everybody wanted to be, you know, writing code, writing apps and all this stuff.

9:07 So I was it was weird for me to go EE.

9:09 And then I went to do an internship.

9:12 I won't name the company.

9:14 I don't want to I don't want to offend this company.

9:16 I went to a big well-known technology company. I went to do an internship and.

9:21 Scott: Acme core,

9:22 Acme

9:22 Brad: Yeah,

9:22 Scott: core,

9:22 Brad: exactly.

9:23 Scott: right?

9:24 The Roadrunner and Coyote.

9:26 Brad: Exactly.

9:26 I was bored.

9:27 I was bored with my time there.

9:29 By the way, I had this moment.

9:31 Does my LinkedIn profile show this and it does not show this company.

9:33 this company says um you

9:36 Scott: know what Scott it's totally okay only 40 people listen to this podcast

9:40 Brad: perfect perfect

9:42 Scott: oh we're up to like 200 now after the bob

9:45 Brad: oh

9:46 Scott: martin uh john

9:47 Brad: there

9:47 Scott: paul

9:47 Brad: you

9:48 Scott: bob

9:48 Brad: go i like that um so i i did the internship um i looked around at people with a master's degree and um they were doing work that i just wasn't at all excited about i thought i'd go

9:59 go to get this amazing master's degree and all I get to do is this work right here and so I came back and I had a the guy that became my thesis advisor and actually eventually my co-founder at Ambic Dennis Sylvester he said hey Scott I'm going to pay for your your master's degree and I think you'll like it so much that you're going to stick around for a PhD and I was kind of like okay I had the bad experience with the with the internship so maybe I'll give that a try and he gave me enough

10:24 Give me an opportunity to work on some really cool stuff. The whole subject of subthreshold computing wasn't something that I invented.

10:30 It had actually been around for decades.

10:31 Scott: Like the 70s, right?

10:32 Brad: Yeah, exactly.

10:33 So it was think Swiss watch industry was early on it and as well as the Japanese watch industry.

10:41 They were early on that.

10:42 But there was this resurgence of interest in that in the early and mid 2000s. And I was fortunate that I got I got into it.

10:50 And there was a point where, you know, when you're operating at the University of Michigan, there's a ton of research dollars coming in from all kinds of cool companies and they're always coming and visiting and talking to you. And I remember getting asked a lot about it at these,

11:04 you present your poster of your work and a lot of companies are going,

11:07 hey,

11:08 is this, you know, what's the commercial potential here?

11:09 And at some point I heard that enough where I thought,

11:12 okay,

11:12 I actually think this is going to be commercialized. And I do remember.

11:16 being in an elevator at the University of Michigan and thinking you know if if someone's going to if it's going to be commercialized why not it be me right and until then I would say that it would have been for me I was thinking let's I'll go work at a university as a professor or I'll go be a researcher at one of these companies or maybe I'll go to business school I don't even know I had I had a wide range of options but when that when that happened

11:42 I thought,

11:42 OK,

11:42 I want to go start this company.

11:44 I want to make this technology happen.

11:45 And I was in a really fortunate position to do that because I wasn't married.

11:52 uh,

11:53 I, you know, I had no mortgage. My family was such that I knew that if, if everything,

11:59 you know, went poorly and I could go live with them and, and,

12:03 you know, um, uh, uh, cry on their shoulder for a little bit before going to get a job. And so it felt like, Hey, if there's ever a time for me to try this now,

12:11 now is the time.

12:12 Um,

12:12 and then on top of it,

12:13 I had the university's resources to draw upon.

12:16 Right.

12:16 Um,

12:17 One of the bits of advice I always give to young entrepreneurs who are coming out of the university is don't be in a rush to leave.

12:22 You have this fantastic place where they've got every CAD license.

12:26 We use these CAD tools and they're so expensive,

12:29 right?

12:30 And even at Ambic now,

12:32 the number of licenses,

12:35 the number of seats,

12:36 the amount,

12:36 the tools that we can draw upon are so limited compared to what's available at the university.

12:41 They have these beautiful labs.

12:42 I'm like, just stay there,

12:43 do as much.

12:44 Do as much as you can there and then and then spin out when you need to spin out.

12:47 And so I was able to do that. It was great.

12:49 I basically got paid as a.

12:52 Um,

12:52 as a postdoc for a while, I graduated with my PhD,

12:55 I found a good commercialization grant.

12:59 I was getting paid, I won't say how much but it was by today's standards an extremely modest amount, but I felt like I was I had made it right it felt like an amazing

13:07 Scott: Yeah.

13:07 Brad: thing was,

13:08 I was like getting paid to just go figure out how to build this, you know, so it was it was a really fun time and I miss sometimes I miss those early days.

13:17 Scott: Mm-hmm.

13:18 Yeah.

13:19 Okay.

13:20 Brad,

13:20 I got so many more questions. Can I keep going?

13:23 Sure.

13:24 Yeah.

13:24 Yeah.

13:24 Let

13:25 Yeah.

13:25 me just, let me just, let me just ask one more.

13:28 Briefly, for people who are listening to this and might be inspired by you and are thinking maybe they're in a PhD, I'm curious about that spin out process.

13:38 How does that work to take out,

13:40 does the university own the IP?

13:42 Like how does that, how does that all work when you want to?

13:44 go figure something out like that and then go commercialize it.

13:46 Brad: Well,

13:47 I'll tell you how it worked for me. And then there's a bunch of other iterations of how it's possible.

13:51 But for me, I had two co-founders that were professors at the university that were my thesis advisors,

13:56 Dennis Sylvester and David Blau. And we had a set of inventions that we wanted to take that I wanted to take with us to go found the company,

14:06 they were going to stay as professors,

14:07 I was going to go found the company as the as the

14:09 the founder is the CEO,

14:11 whatever.

14:11 So we went to the Office of Technology Transfer and we got access to these inventions. They weren't yet even patents,

14:20 right?

14:20 So we hadn't even filed patent applications.

14:22 They were inventions that we wanted to file as patents.

14:24 Office of Tech Transfer at the University of Michigan nominally owns that process.

14:29 They took

14:31 an ownership stake in the company small right in the end they did well we went public last year i'm sure that they did well i should go look and figure out how much money they made but it was successful for them and then and then what we did is we collectively filed patent applications for these inventions and and what we committed to do in addition to giving them an ownership stake in the company is we get we would give them a royalty for every chip that we shipped and

14:55 That continued for a while and they actually had a nice little cash flow going from that and then we ultimately bought them out of that deal.

15:05 And it was, I would say it was a pretty good process.

15:08 They were,

15:09 University of Michigan at the time was not necessarily known as a, you know, entrepreneurial hub.

15:16 If you were going to rattle off places, you would say Stanford,

15:18 right? Or

15:18 Scott: Yeah,

15:18 Brad: Berkeley

15:19 Scott: sure.

15:19 Brad: or some of these other companies or universities.

15:22 And so they were going through a process of making making it all easier and so forth. And there were some really good I would say one of the things I did really well personally is I found.

15:34 good advisors who had been there and done it before there was this guy named Dave Hartman who worked at the university he was an entrepreneur who had been successful he worked at the office of tech transfer and he was basically there to be my ally in negotiating with them also it was weird because I was a PhD I was a kid negotiating with the university right I was simultaneously negotiating with my co-founders who were professors who until then had been my bosses right they were friends like today we're

15:59 They were friends, but they were my bosses.

16:01 So negotiating with them, I'm negotiating with the university, but I had this guy, Dave Hartman, who kind of became my advocate and that was critical for me.

16:10 So maybe the lesson,

16:12 the best lesson in all of this is for university students out there, find a good set of advisors,

16:18 listen to them.

16:19 And that kind of allows you to foresee many of the challenges ahead.

16:24 Scott: All right. I'm going to take the con here for a second,

16:26 Bo Rad.

16:28 Brad: Do it. This is great.

16:29 Scott: I love it. I have two questions that are kind of related to this early part of the company.

16:35 But maybe they relate back – at least the first one I want to ask you is about – you mentioned patents, and you clearly were doing research.

16:43 One of the things that –

16:44 Since it is very,

16:46 you know, I'm going to say it's more in vogue now because of sort of the sort of like fervor over AI,

16:53 all of these labs,

16:56 right,

16:57 and internal entities inside of, you know, mostly in corporate environments like Google, et cetera,

17:03 are publishing research papers.

17:06 And I'm kind of curious, how did, I'm assuming that you would.

17:10 had published papers about some of your inventions,

17:12 how did that affect the patenting process,

17:14 right?

17:15 Doesn't that, doesn't that, isn't that like pre-disclosure and doesn't that impact your patenting process?

17:19 Brad: Yeah, it does. So you have in the United States, at least when you publicly disclose an invention,

17:25 you have 12 months to file

17:27 Scott: Yeah,

17:27 Brad: a patent.

17:27 Scott: but it closes off international patents,

17:29 Brad: It

17:29 Scott: right?

17:29 Brad: does close off international patents. Yes.

17:31 Scott: Yeah.

17:32 Brad: So in our case, one of the one of the base patents,

17:38 man,

17:39 I wish I could remember this. I want to say that we either filed before it was published.

17:45 or we only filed in the U.S., and then what we did is we,

17:47 okay,

17:48 once we really thought that the core invention,

17:51 once we really thought about how we would productize it, we realized that there was more circuitry that you need to wrap around it.

17:57 So there was an opportunity to add extra elements

17:59 Scott: Sure,

17:59 Brad: and make

17:59 Scott: totally.

17:59 Brad: it internationally patentable.

18:01 So I want to say that we did a sequence of things and ultimately ended up with both U.S. patents and international patents because we added elements.

18:09 And then,

18:09 hey,

18:10 the truth is today.

18:11 Ambic has 90 plus patents that are granted or pending and only two of them were the original two that we licensed

18:19 Scott: Yeah,

18:19 Brad: out of the university.

18:20 Scott: sure.

18:20 Brad: I mean, I felt a lot of pressure early on that, oh my gosh, these were the things we invented, I must use them.

18:26 And what I figured out at some point is actually what I need to do is I need to just like continue to expand,

18:33 I need to investigate new directions.

18:35 If that turns out to be the wrong direction.

18:36 shouldn't have no emotion about tossing it aside the

18:38 Scott: Yeah.

18:39 Brad: cool thing is we actually still use those those two circuits today and we're you know 16 years into this thing oh

18:46 Scott: Yeah.

18:47 Obviously, and if you've got a great patent team that knows how to do strategy around that,

18:51 then like I'm getting patents filed in my name for a company that I haven't worked for in 20 years because like

18:58 Brad: my god

18:59 Scott: they've they've been just doing the extensions and figuring out.

19:02 out extra things and it's like oh my god so

19:04 Brad: Sure.

19:04 Scott: it's kind of funny in that regard but my next question kind of relates to um you started out as the founder ceo now you're the cto um we've kind of talked a little bit in the past and on other episodes in the podcast we've talked to folks that have kind of transitioned in that way and i'm just kind of curious if there's anything you feel like you can share to other entrepreneurial folks that about

19:27 You know, the what that shift feels like, how you made that decision.

19:32 I mean, there's I'm sure everybody's story is a little different there,

19:35 but I'm just kind of curious if you could speak a little bit about that journey for you.

19:38 Brad: Yeah, it was an interesting one.

19:41 I started being CEO for the first couple of years and very early on I established with the investors,

19:48 the venture capitalists that came on board that I would be happy to step aside and be CTO when the time is right because I acknowledge I was a kid. I didn't know anything.

19:55 I didn't know what a CEO is supposed to do. Right.

19:59 And at some point, you know, they start giving me good feedback. Hey,

20:02 Scott,

20:03 you're the needs of what we need out of it or the list of what we need out of a CEO is kind of growing,

20:08 but you're keeping up with that growth curve.

20:10 Right.

20:11 And we like what you're doing. And I started thinking,

20:13 OK,

20:13 maybe this is long term for me. And, you know, your ego gets attached to being the CEO of the company. And maybe more importantly, as a CEO,

20:20 yes,

20:21 you report to the board, but ultimately you're the top dog and you're the one that gets to set.

20:24 to set the culture and everything else and then at some point they came to me the the board said hey we got to grow here right and and we got to get a professional ceo and i was disappointed my first it was it was immense disappointment right and which was funny because i wouldn't have predicted that a couple years before i just gotten attached to the job um and you know they see that and what they said to me was or they saw that and what they said to me was hey look

20:51 You've got two choices here. You can be the CEO.

20:53 We can go find a CTO for the company because we need one.

20:57 And or you can go be the CTO and we can find a CEO.

21:01 And the truth was there were thousands of people that could have been a great CEO for the company. I was probably the only, if not one of five people that could have been the CTO for this company.

21:11 And so that made sense to me.

21:13 My logical mind says,

21:14 okay,

21:14 that makes sense.

21:15 We brought on a guy who was, in hindsight, he was a very successful CEO,

21:21 had done it before.

21:22 And we clashed immediately.

21:24 Ultimately today,

21:25 I would say we have a good relationship.

21:26 This is not our current CEO.

21:27 This is a guy who was in between our current CEO and me.

21:31 You know, there was some real tension at the early part of it.

21:35 There was a good potential for him leaving given how I was kind of continue to hold on to Yeah. the reins. And what I realized at some point after a stern talking to by one of my.

21:47 close advisors.

21:50 What I realized is that I had to release a little bit of control.

21:54 I can still be a strong influencer of the culture.

21:57 I can still be driving technology roadmap and all these things, but ultimately I needed to let him be CEO.

22:02 And that means I report to him, right?

22:04 I was used to being, again,

22:05 the top dog and now I've got a boss,

22:07 right?

22:08 And so what I'll say is that our current CEO is a guy that has been at the company now,

22:14 Humiyosaka, who's been at the company now.

22:16 now for more than 10 years and I think that's worked in part because I had that first go-round with the other CEO I it went poorly at first we figured it out

22:26 Scott: You had a test model.

22:27 Brad: yeah yeah and so I you know we've made it work so now we've got a good mutual respect relationship but the truth is he's the CEO he's my boss right and so you know he he's in charge now what I will say to

22:41 CTOs out there looking for a CEO is find somebody who respects you and treat you treats you like a partner I've seen different versions of that I've seen founder CTOs who've you know had CEOs that that you know don't really care what what their opinion is and all that and I have a CEO who treats me like a partner right we in a closed door session we can have really good discussions about what's going on how we're going to attack things but

23:07 outside that room, I have to remember again,

23:08 he's my boss.

23:09 Scott: Yeah.

23:10 Well,

23:10 thanks for sharing that, by the way.

23:11 I thought I was, you know, I didn't know what your story was going to be exactly.

23:16 And it's awesome to hear you kind of like be genuine about like what that felt like, right?

23:22 You know, you're

23:22 Brad: It's

23:22 Scott: the founder,

23:24 you know, this was your baby,

23:26 you know, it is your baby.

23:27 And so, yeah,

23:28 that's awesome.

23:29 I love the advice that you think you're giving,

23:31 you know, like find someone that's really going to treat you like a partner.

23:35 Brad: Yeah.

23:36 Scott: There are many founder CEOs who can't make that transition and their companies suffer or fail as a result of it. And it's admirable that you're able to do it.

23:48 Brad: Yeah,

23:48 I appreciate that.

23:49 Listen,

23:49 I distinctly remember getting that stern talking to by this close advisor who was, I want to say at the time he was also a board member and I went home and laid on the couch.

24:03 and complain to my wife and as I was just talking she's by the way she's like my in-house psychologist she actually has a degree and a master's in psychology and so she's the most empathetic person I met she we can sit there we can have these conversations and as I was talking it out with her I realized how much in the wrong I was so yeah that was a good learning experience

24:23 Scott: Well, it's good that you have a, you know, you had the support network that

24:26 Yeah,

24:26 you

24:27 yeah,

24:27 had, you know, in your advisor and in your wife to be able to kind of help you through that transition, which obviously was is hard.

24:33 It's a hard transition.

24:34 It's got to be a hard transition,

24:35 right? You got it's your ego is going to be so so much tied into everything about it.

24:41 So I

24:41 yeah.

24:41 love it.

24:42 I want to pivot to let's talk about your tech.

24:45 Let's talk a little bit more about what you guys have been working on. I know.

24:48 I know recently,

24:49 I guess maybe in January, you announced sort of this new product that you're focused on as a neural processing unit that you're targeting.

25:00 Brad: Yep.

25:00 Scott: You want to tell us a little bit about that?

25:03 Brad: Yeah.

25:04 So let me start by saying we have this flagship family that we call Apollo. It's a system on chip that has a microprocessor core.

25:10 It has a radio GPU.

25:12 for graphics,

25:14 not for AI,

25:14 but for graphics,

25:15 as well as some lightweight AI acceleration.

25:18 But what we saw our customers doing with Apollo...

25:22 was increasingly AI workloads and and and at first that we had more than enough hardware to support that but as we started looking at their roadmaps we saw the need to have neural processing unit inside so to have specific hardware acceleration to take on really big models actually years ago I would say it's probably 2019 2020 we first had the idea of building this chip with which ultimately became atomic that has all this

25:47 all these processors the memory and all that but also the neural processing unit but the customers weren't ready at the time right they were kind of dipping their toes in in edge AI and bringing neural networks out of the cloud and out of these devices and they just had the lightest weight of needs but as their roadmap started taking shape we saw that there was this need for atomic and atomic is the compute horsepower that this thing has is crazy if you had told me

26:13 When I founded the company that I was going to build a chip like this, I would say no way.

26:16 But Moore's law is a pretty wild thing, right?

26:19 I mean, the number of transistors we can pack onto these chips,

26:22 even chips that are using 10-year-old manufacturing technology can have an enormous amount of compute on board.

26:30 And so Atomic came from that.

26:33 I've got a ton of memory on board,

26:35 some really cool memory innovations.

26:37 You know, there's been a lot of talk in the cloud about,

26:39 hey,

26:40 if I want to run whatever my latest large language model is in the cloud,

26:44 memory becomes the key bottleneck from a performance standpoint,

26:47 from a supply chain standpoint,

26:48 from an energy standpoint.

26:49 All those same things are true at the edge. When I want to put, you know, an AI model on a device like this or like this,

26:56 I care a lot about memory. So Atomic includes that neural process.

27:00 neural processing unit as well as some some really cool memory innovations but maybe to quantify for you if with Apollo the latest Apollo that you can find on store shelves today and a bunch of our customers products you can support a an AI model that's maybe 2 million parameters okay which by by cloud standards is nothing right Yeah, it's tiny but

27:19 Scott: sure.

27:19 But it's

27:19 Brad: our customers

27:20 Scott: in your wrist.

27:20 Brad: that's yeah I mean for our customers though it's a it's a you know it's amazing

27:25 With Atomic, you'll be able to build models that have in excess of 200 million parameters,

27:30 so 100x, 100-fold increase in the complexity of model.

27:34 And so what that means is our customers can take on vastly more complex tasks,

27:40 right?

27:40 So if today,

27:41 if I break open my typical customer's device,

27:44 whether it's a wearable like this or it's a smart home sensor or whatever.

27:49 They're doing things like fall detection,

27:51 you know, is the person falling?

27:53 They're doing heart health analysis and activity recognition.

27:59 They're doing leak detection,

28:02 right?

28:02 There's a water sensor and it's doing leak detection.

28:04 They're doing anomaly detection,

28:05 figuring out, hey, this piece of industrial machinery,

28:08 the vibration signature just changed. What's going on, right? But with Atomic, when I jump to that next generation where I can do 200 million.

28:15 million parameters i can add a real conversational voice user interface to these things so i can talk naturally my much like you might talk to chat gpt you're not going to be able to do the wide breadth of things you can do with chat gpt but hey there's only like i don't know 500 things i might do with my watch or my band or my my smart ring or whatever and so

28:37 That is feasible in that.

28:38 And that's a big deal, right?

28:41 And there's all kinds of other use cases that our customers have envisioned.

28:44 But I am really excited about what that enables.

28:47 The other thing that Atomic makes possible.

28:51 is until now we have largely not supported cameras.

28:54 Cameras generate so much data,

28:56 they require so much compute and so much AI that it's been impractical for us to do it. And more importantly,

29:01 I just don't have the plumbing on my chips to bring a video feed onto my chips. And so Atomic will have variants that are powerful enough to process all that video data,

29:12 but also will have the plumbing needed to bring that data on board. Think about

29:16 AR glasses or

29:17 Scott: Yeah.

29:17 Brad: smart glasses,

29:19 I see a ton of really cool and aggressive roadmaps from companies in that space and from our customers in that space.

29:24 So I'm really excited about what's coming there.

29:26 And then also think about cameras of all sorts, whether security cameras,

29:30 doorbell cameras,

29:31 hunting cameras and outdoor cameras,

29:33 all kinds of really cool use cases where you want to do processing there on the edge. You want to detect motion and track things and track gestures and track.

29:42 track your eyes and all that.

29:44 So a lot of really cool opportunities there that we think Atomic will make possible.

29:49 Scott: So one of the things that immediately jumps to my mind, I love that list of your laundry list there.

29:54 What about like, you know, you know.

29:58 hearing aids with full translation support,

30:01 right?

30:01 Brad: Absolutely.

30:02 Scott: Like I'm assuming you could, could you do that sort of thing before Atomic or really Atomic is what's making that sort of thing possible?

30:09 Brad: So some of our customers products do that today.

30:15 So actually a product I really like is called Even Realities. It's a smart glasses, super slim, and they're using our product.

30:25 um they have a heads up display built in we have a gpu so we can we can kind of manage display and do the animation and all that uh and i want to say that they they public they i believe that they support translation i'm not 100 sure

30:39 Scott: But

30:39 Brad: on that are but no

30:39 Scott: they doing it on the device or are they paired with a separate device?

30:42 Brad: yeah yeah that would definitely not be done on device so there's a certain you know experience there so how does that experience change if you can do that on device i don't know that's it

30:50 That's a that's a great question. And I do think that that will be possible at a minimum you can what we can do on device is we can you know we can go from.

31:01 you know a sound wave to like phonemes we can we can convert you know basically do a speech maybe even a speech to text translation there where where then you're just sending a text stream instead of a raw audio stream so that opens up a lot of possibilities there but yeah I would say that's one of the definitely one of the coolest applications I also really like there's a whole class of devices today that's you doing audio only and it's

31:26 It's taking transcripts and notes and all that.

31:29 And so, you know, that could be in a glasses form factor where we're all talking, it's recording what we're saying,

31:34 and then it sends me a summary of the key things we've learned afterwards.

31:37 Of course, you know, teams and other programs already offer that capability, but not necessarily in the real world.

31:44 Scott: Real-time closed captioning for every conversation I ever have in my glasses.

31:49 Brad: That would be cool, right?

31:51 Yeah.

31:51 Scott: Yeah.

31:52 I mean, it sounds like you feel like the atomic platform has a sort of shape to it that those sort of applications start to become practical.

32:04 Absolutely,

32:05 yeah.

32:06 Yeah.

32:07 Very cool.

32:08 Have either of you ever read a book called Super Sad True Love Story?

32:13 Oh, you've talked about this on the podcast before,

32:16 Scott,

32:16 and I haven't read it yet.

32:19 it's a novel and it's set in the future and some of the things that you're talking about or that you're doing with Atomic like you know this book's probably 15 years old and at the time well the iPhone had come out when the book was written but nevertheless the point is is like all of a sudden some of these you know space age things that they were talking about in the book are becoming real

32:40 Yeah.

32:41 Brad: Yeah.

32:41 Scott: That's what it reminded me of.

32:42 All right. Here's a question I have for Atomic. So one of the things that we see happening in sort of like the desktop space,

32:47 people that are like trying to go to local inference,

32:50 right?

32:50 There's a handful of different.

32:54 research papers and implementations where we've taken like super large models of mixtures of experts and doing this thing where we're sort of like streaming it like we're only lighting up the experts that matter for the tokens that you're processing and sort of bringing in the rest of the experts only as you know the tokens space moves past that and i think that one of the and of course what we see in the desktop space is where that

33:19 the bottleneck ends up becoming this sort of like

33:22 the lesser of how much available memory do you have versus the speed of your memory,

33:28 your cached memory bus into your active memory.

33:32 And I'm just kind of curious, like, how does that apply to below threshold compute?

33:37 Like, it feels like we've got this tension of like the speed of below threshold compute being in the megahertz. And so

33:45 Brad: Yeah.

33:45 Scott: I'm just kind of curious, have you guys started to

33:48 Sorry to think about that or.

33:49 Brad: Yeah,

33:50 there's a lot. So there's a lot of things here.

33:51 So first of all, one of the beautiful things about AI compute is that it's highly parallelizable.

33:55 So as we yes,

33:57 we run at lower we tend to run at lower frequencies than you would in conventional voltages that that's improved immensely as we've gotten a more advanced manufacturing nodes.

34:07 But AI is super parallelizable. So these neural processing units can do a ton of

34:13 of processing in parallel so I can you know you can you can do a couple hundred max multiply accumulates every single clock cycle right and so actually then the problem becomes how do I keep those multiply accumulate units full how do I how do I provide enough data via the bus to do that and so that does tend to be the bottleneck even for our for our style of processor memory bandwidth does get to be a bottleneck if you're not careful now

34:39 Now,

34:39 the cool thing is because, yes,

34:42 we're running wide,

34:43 but we're running at low, relatively low frequency, so you kind of go low and slow.

34:48 What you can do is you can run memories at relatively higher speeds and higher voltages and just kind of fill up the bandwidth that way.

34:57 And so it ends up perhaps not being a problem. Now the problem is actually it's one of energy. It's not a bandwidth problem because I can meet.

35:04 can meet that.

35:05 It's one where I've brought the energy of the processor down so low that suddenly the memory is sticking out as an energy problem,

35:12 right? Yep. It's whack-a-mole, right?

35:13 I've knocked down one mole and I've got one that's popped up.

35:16 And so the bigger problem is how do I bring the energy of that memory down?

35:20 And we are, let's just say that we're investing on a couple of different fronts to solve that problem.

35:25 It is a problem that is being solved and it will be a couple steps.

35:30 steps we've got in atomic some really cool innovations coming and then post atomic we have some other ones coming so there are a whole variety of other technologies that are out there too that that are being talked about so one of the approaches that it gets talked about is so-called compute in memory and the idea is that

35:47 You know, rather than having memory over here and your processor over here, let's merge them into one unit.

35:52 And there's there's so-called near memory compute where you just you've got a memory cell and you've got your compute cell and they just are very closely located and and that works.

36:01 There are approaches that are analog in nature where you basically you kind of reconfigure the memories to be like analog circuits and they do they do the multiple.

36:13 multiply an accumulation in an analog domain and in that way they can be ultra efficient you can like one bit of memory can be configured to represent an eight bit weight or an eight bit activation or whatever that's super cool it's probably one of the hardest um compute problems i've ever seen and there's a number of companies that are trying to do this go check out mythic is a actually a startup here in austin that was founded by a couple friends of mine and they do some really

36:38 really cool stuff on that front so they're taking a super creative approach to attacking the memory uh bandwidth problem in ai uh and and it's just one that is immensely complex our approach has been a little less aggressive but but perhaps a little bit more pragmatic and realistic and achievable in the short term so definitely a big a big issue

37:00 Scott: Very cool.

37:01 It's so funny because I told you beforehand that I'm not an

37:05 Not a double E guy,

37:05 but I'm able to read enough to be like, ooh,

37:08 that sounds like an interesting problem. How does one solve that?

37:11 So there you go. I love you entertaining the teaching the CS guy a little bit about the EE problems.

37:20 What I am curious to know about as you're talking about this.

37:24 I've always been an engineering leader for software companies they're online software companies and you know there are a set of say management or leadership principles I have about how to

37:37 do that to evolve over the years right and I'm just curious to know like what's it like to leave I'm wondering if they're different for a chip company is ultimately what I'm wondering or if you know like the first note I have this thing that I've been writing notes of questions to ask as we've been talking the first thing I asked was like is there tech debt does that exist in your world Yeah, yeah

37:57 what's tech debt look like in hardware?

38:01 yeah yeah yeah absolutely absolutely

38:03 Brad: Um,

38:04 first of all, a lot of the hardware development process these days is just software development. So I'll be a little more specific there.

38:11 So, you know, if you break open our chips,

38:13 there's maybe three classes of circuit inside there. There's you got digital circuits,

38:19 you got analog circuits and you got memory and digital circuits are largely coded today,

38:25 right? You write Verilog.

38:27 register it's rtl it's a it's basically code that you guys would probably understand if i showed it to you'd probably get to some extent what's going on there um you can write you know c equals a times b and it you know will synthesize a multiplier so a lot of a lot of um chips these days are developed in that way so a lot of the the concepts that you might have about software development are going to apply here um that said there's this one major difference with hardware development

38:52 development.

38:53 And that is that a compile is

38:56 Scott: Mm-hmm.

38:57 Brad: me turning

38:59 That code into a chip, getting super expensive masks made,

39:05 getting it manufactured,

39:06 and then checking if it worked, right?

39:07 And that is a,

39:08 you know, the design is a 12-month process. You've got a couple months of manufacturing and you've got a couple months of bring up.

39:15 That's a long cycle. Whereas,

39:17 you know, I laugh because the software guys,

39:19 they hit compile and then if it didn't work,

39:21 they just recompile. They fix it and recompile,

39:23 right?

39:23 It's a very different.

39:24 So the concept of agile,

39:27 for example, I think makes a ton of sense in the software world.

39:30 Mm It has

39:31 Scott: -hmm.

39:31 Brad: to be modified somewhat in the hardware world.

39:33 So you would probably look at hardware development methodologies and think,

39:38 oh, wow,

39:38 that is very different. It's maybe a little bit unique,

39:40 but it's the right thing to do. I will say that.

39:44 AI is completely changing the way we're, you know, approaching a lot of this, or at least it will dramatically change things. So a lot of the same uses that you're seeing with coding agents on the software side apply to the hardware side. So we can generate Verilog, we can generate digital circuits.

39:59 using AI agents but there's a whole bunch of other uses there that that that are exciting so I do think that hardware design is going through the design flow is going through a bit of a revolution that's gonna who knows where it will go but it's gonna be fun few years

40:15 Scott: And are there like, I assume there would be, but maybe I'm maybe I'm wrong about this, like clearly when you compile to the point that you're actually, you know, doing the manufacturing like it's like you better.

40:26 You got a long lead cycle there,

40:28 but do you have like software test beds to test your Verilogic and stuff like that? Like you

40:35 Brad: Yeah,

40:35 Scott: get you

40:35 Brad: we

40:35 Scott: get shorter cycles on that kind of stuff.

40:38 Brad: have, of course, simulators of various types that can kind of simulate various stages of the design and you can look at waveforms and debug and we'll, for example, a lot of our processes,

40:49 we build.

40:52 We build the hardware, we write C code that effectively executes on the processor and then we simulate the processor running through that code and see if it does what we expected.

41:02 that's running on you know computers it's very time intensive and there's only so many cycles you can run right so so it's it's limited in its in its value the other big thing we do is we just take fpgas you guys know what fpgas are okay

41:13 Scott: I don't.

41:13 Brad: so fpga is a is a chip that i can basically configure to do whatever i want it's a super malleable thing it's got a bunch of let's think about it's got a bunch of lego pieces that i can assemble in whatever way i want and then and then go so what i can do is i can

41:27 I can upload an image of my chip of my Verilog onto that FPGA and it's this giant thing right I mean the board is it's this big it's got a fan attached to it like this is a heavy duty piece of silicon but I can upload my design onto it and then run software my software team is out there right now running code on an FPGA representation of our processor and maybe it runs at one-tenth the speed of the ultimate processor or one one-hundredth

41:53 hundred but that's a whole lot faster than simulation world so we can actually get through a lot more many more testing cycles as a result of that we even have customers that will do that right so we we always have what we call an alpha customer who signs up with us early as a for a chip

42:10 And what we'll do in some cases is we will give them that FPGA and say,

42:14 all right, you guys want to start early pre-silicon development.

42:16 Let's go, you know, get your operating system going,

42:18 get your drivers going,

42:19 whatever you want to do.

42:20 And so that is a great way to build, you know, close relationship with the customer.

42:24 Scott: That makes a ton of sense.

42:26 She found ways to shorten the cycle,

42:29 but ultimately the cycle is long.

42:30 It's got very long feedback loops in it.

42:33 Brad: It is. It is. Yeah.

42:34 And again, back to AI,

42:35 the hope would be that we can shorten that feedback loop. You can do more parallel experiments.

42:40 You can you can make better use of 24 hours in the day to be doing assessment.

42:45 My engineers have to go sleep and an AI agent that is run by an engineer doesn't necessarily have to do that.

42:50 So, yeah,

42:51 loops are long.

42:52 I do think that they will shorten.

42:54 The other problem is that these mask sets are really expensive. So the tooling that we need to manufacture our chip.

42:58 chips is is really expensive and I can't disclose specific prices here but you know think for the most advanced nodes you're you're probably looking at tooling that's in the tens of millions of dollars so every time you do a a chip manufacturing run where you're changing you know all the layers it's it's tens of millions of dollars and um that's prohibitive and that's it's it's changed the nature of chip startups you know when I founded this company

43:23 back in 2010 nobody wanted to invest in chips because of exactly this problem because there was this perception that hey any new chip company you need to raise a couple hundred million dollars because of all the tooling because of the expensive CAD tools and I at the time I convinced my investors that hey we're this edge computing thing which we weren't really calling edge back then but but you know the intelligence everywhere vision we were going to be using older manufacturing nodes that were cheaper and

43:49 and uh and you know the tool the cad tools were going to be cheaper and and they all bought into that and then i turned out i was totally wrong right it turns out that we actually also had to do the more's law thing get down to the advanced nodes and it got expensive you know fortunately it worked out and we were we were successful but uh it's expensive to build chip companies i mean i talked to you know chip companies that that made it back in the early 2000s they raised you know one-tenth of what we raised

44:15 ways to get where they needed to go because the tooling was cheaper the chips were less complex and it's just it's a different ball game today right

44:22 Scott: Well, they weren't trying to put their chips inside somebody's eyeball.

44:25 Brad: that too that too

44:27 Scott: That might be the big difference there.

44:31 I love it. I love it.

44:32 Well, Scott,

44:33 this has been a delightful conversation.

44:36 Brad: I appreciate it.

44:37 Scott: Thank you very much for taking the time to come on today.

44:40 Brad,

44:40 I don't know if you had any final questions to

44:42 No,

44:42 ask I before

44:43 was

44:43 we wrap

44:43 just going to

44:43 up.

44:43 say, I know I was going to say I appreciate it.

44:45 This is I I think your tech is really cool.

44:49 I love your story. You know, this it's wild that this is your first job.

44:54 Brad: Yeah, it should be my last one. That's what I tell everybody is, hey,

44:57 this is the only one. I want this to be the only one.

44:59 Scott: There you go.

45:00 That'd be sweet.

45:00 There you go.

45:01 Well,

45:02 thank you very much for coming on today.

45:04 We really appreciate it.

45:05 And good luck.

45:06 Yeah.

45:07 Brad: I appreciate it, guys. Thanks for having me. It was fun.

45:09 Scott: Awesome. And what do we say, everybody?

45:10 We always say smash that subscribe button,

45:13 Smash

45:14 rate

45:14 that subscribe

45:14 and

45:14 button.

45:14 tell

45:14 Rate and review.

45:15 your mom.

45:16 Tell your mom.

45:18 mom's big part of our audience.

45:21 Anyway,

45:22 nice meeting you,

45:22 Scott.

45:23 Have a good one.

45:24 This episode of the How Many CTOs Does It Take podcast was produced and edited by Saffron Heptagow.

45:30 Our theme song was AI-generated by Brad.

45:33 Please follow us on all the social media platforms at HowManyCTOsPod.

45:38 You can find us on YouTube,

45:40 LinkedIn,

45:41 Instagram,

45:42 Facebook,

45:42 ThreadsX.

45:43 TikTok and Patreon or at our website at howmanycto'spod.com and remember to rate, review, and subscribe on the podcast platform of your choice.

45:53 Thanks for listening.