Transcript: Rediscovering the Joy of Coding through AI with Paul Brown
0:00 Paul You know, the resistance is due to a sense of, you know, disempowerment or some kind of hypothesis that the AI is going to somehow supplant, you know, the engineer.
0:12 Paul I think that's misplaced.
0:29 Brad Welcome, everybody, to another episode of the How Many CTOs Does It Take podcast.
0:34 Brad My name is Brad Hefta-Gaub and my co-host is.
0:37 Scott Scott Porad.
0:38 Brad And today we have a very special guest with us, Paul Brown.
0:43 Brad Paul,
0:44 Brad how would you like to introduce yourself and tell us a little bit about what brought you here?
0:50 Paul Gosh.
0:52 Paul Well, I guess I'm fresh off 10 years at Salesforce.
0:57 Paul was a lot of fun to work there through some really interesting parts of the company's journey.
1:03 Paul When I joined 10 years ago,
1:06 Paul we were all first party with a little bit of Azure and maybe a little bit of AWS and some of that company knew about,
1:14 Paul some of that company didn't know about at the time, but got to, you know, ride shotgun and drive the stagecoach some and had a...
1:23 Paul really good time on the journey to being all in on public cloud and you know to a certain extent being all in on AI as well so saw that over the past couple years you know prior to that I was an entrepreneur did that twice didn't learn my lesson the first time I guess did a really brief stint at Amazon back in the early days of AWS
1:51 Paul In way, way back,
1:53 Paul I was a math professor,
1:55 Paul but I gave up that vow of poverty when the internet was young and went the entrepreneurial way.
2:06 Brad Love it. Welcome to the pod.
2:08 Brad Paul,
2:09 Brad you and I have known each other for a couple years and I know that we often spend time talking with each other on the skin track about various challenges that we face as upper management leadership in engineering organizations. And so I know that you've got lots of thoughts to share and lots of wisdom to share with us. But recently you posted on your link.
2:31 Brad LinkedIn about you personally diving into vibe coding and kind of trying to sort of see what this this whole beast is all about and I thought it was interesting what you had to share about that so why don't we get started what do you maybe give us a little bit of a what got you into the idea that you were going to explore this space you know kind of in a more intimate way and what have you learned
2:56 Paul Oh, gotcha. Well,
2:58 Paul I think, you know, I heard from a number of folks and not just technical folks that AI was magic.
3:06 Paul And I'll admit,
3:08 Paul I started out as a bit of a skeptic. I remember I went and asked ChatGPT in the early days some hiking question and got a completely ridiculous and wrong answer.
3:22 Paul You know, and had the Nelson months,
3:25 Paul you know, ha ha finger point,
3:26 Paul you know, see this isn't magic moment.
3:31 Paul But lots of other people did experience the magic, and I related one experience in that blog post on LinkedIn from a friend who had started a marketing analytics function at his company,
3:45 Paul you know, on the back of Python and SQL and other things emitted from ChatGPT.
3:51 Paul And it was wonderful for him because he had 100% more software,
3:56 Paul 100% more knowledge about his business.
3:58 Paul And maybe it's not all perfectly right,
4:00 Paul but 100% more than he had before.
4:03 Paul And that kind of, that inflection point between not having something and having something,
4:14 Paul you know, it makes me think about the old days.
4:17 Paul mainframes and mini computers and then the shift to personal computing and this is kind of that like is this a shift to personal software you know can we kind of all have whatever we want so I wanted to explore that and there were some things that I wanted so I've set about you know trying to build them and just sort of bumbled my way through it
4:42 Brad Yeah.
4:43 Brad Yeah,
4:43 Brad that's so cool.
4:44 Brad Actually, it's so interesting that you describe it as like the transition from mainframes to personal computers.
4:49 Brad Only reason I think there's something about the way you said that was like reminds me I've heard this recently on a couple podcasts,
4:55 Brad some some VCs and founders I listen to on podcasts,
4:58 Brad and one of them has been comparing really trying to say that what the what the product of
5:04 Brad the LLM chat interface is it's not really a chat bot what it is is a computer right and so thinking of that space of sort of like generative AI as as if it was a computer that you can ask to do anything that you want it to do I think that's an interesting framing I don't know if that's exactly how you you meant it but like for sure there's there feels like there's some parallels there
5:30 Paul Oh, you know, I think that's definitely there,
5:32 Paul you know, and had lots of discussions along those lines,
5:36 Paul like, at what point does the catwalk ripple,
5:40 Paul you know, eat the rest of user experience?
5:44 Paul You know, should I build a UI?
5:47 Paul Should I create a, you know, a more rigid but directed user experience or should I, you know, throw the door open?
5:58 Paul just have a conversation with the computer you
6:02 Brad Yeah.
6:03 Paul know the and the conversational aspect I think is really compelling especially if you're in a mood where you want to iterate quickly like if you know like if yes it's a binary thing if it's on off like I don't have a conversation with my light switch if it does the right thing it's very simple it's
6:23 Paul I guess it can't go wrong, well they can,
6:25 Paul right,
6:25 Paul but probably won't go wrong, but there are a lot of things that are more subtle, you know, and can be, you know, much more personal without having to do all the work of stacking up abstractions and figuring out where this button goes and that button goes and everything else.
6:45 Paul There you go.
6:47 Scott Can you tell us briefly,
6:50 Scott I guess, you know, you talked about this little in your blog post, you started out by trying to make a solver for the New York Times tips game.
7:02 Brad Yeah.
7:02 Scott This is a game I play from time to time.
7:07 Scott If you could answer this for us, and maybe it's not an interesting answer,
7:12 Scott but maybe there was a mindset or a frame of mind you went into trying to solve it, and then did you learn along the way that using an AI bot to solve it, they actually had to take a different approach to it?
7:23 Scott Like we have a frame of reference for how we've done computer programming all these years,
7:28 Scott and I would assume if you jumped into it, you jumped in with that frame of reference.
7:31 Scott Did it change at all or was that it?
7:33 Scott just to learn a new way.
7:35 Paul So it's a little bit of both.
7:37 Paul Like I think I took a practitioner's approach the first time. So I wrote a spec. The spec included a data model.
7:47 Paul The spec included an expectation for how the solver was actually going to be implemented.
7:52 Paul I worked through things enough that I knew that a backtracking style solver was what I wanted it to build.
8:00 Paul You know, I knew I wanted it to write in Rust,
8:03 Paul which with the language that I was interested in learning about that,
8:08 Paul you know, the lots of my programming chops atrophied, you know, over the time that I was in management at Salesforce.
8:16 Paul I read a lot of code, but I didn't write almost any.
8:21 Paul You know, and I got some of what I asked for.
8:24 Paul So the.
8:26 Paul You know, the AI did a good job of interpreting my data model,
8:29 Paul you know, I got a working Rust program at the end.
8:33 Paul There were things that I didn't get that I did initially ask for.
8:37 Paul Like I asked for an idiomatic functional implementation.
8:42 Paul And I got the, you know, all of the AI is actually their first version of the recursion past mutable collection.
8:50 Paul you know not remotely what I was after but yeah so so I think you know I did get questions from some friends afterwards which is like well why didn't just point the guys at the game and say write me a solver for this you know here's here's the input here's the output I desire let the magic happen
9:13 Paul You know, and that can actually work really well and it can actually work really poorly,
9:19 Paul you know, with with a couple of the other experiments like pulling avalanche data to add to Strava.
9:28 Paul There were a couple spots where that,
9:30 Paul you know, like letting the AI do all the work went sideways.
9:38 Paul You know, a couple of interesting ones,
9:40 Paul Claude kind of got itself in a twist trying to scrape out information like geo-geason boundaries and zones for avalanche forecasts and full information.
9:53 Paul And, you know, at some point I got tired of watching things scroll by and, you know, went and figured it out myself and told it what I wanted it to do.
10:02 Paul And then it was.
10:03 Paul You know, back to work after that.
10:05 Paul That was great.
10:08 Paul Gemini 3,
10:09 Paul when I went back to do some improvements, actually completely made up a Strava API feature.
10:16 Brad Ha, ha,
10:17 Brad ha.
10:18 Paul which which was awesome is it hey for so i asked it for you try to provide constrained prompts for these things give me three improvements like three areas where this can be better and i have an idea we should implement this x drop a signature header on the web hook and you do this hmac 256 signature and it's it's better it's more secure so i'm like hey great let's do that
10:44 Paul um and then the whole thing stopped working and well it stopped working because the web hook was returning 400 because this header wasn't being supplied let alone a correct signature and so then i asked like hey would you mind starting your source for the x value signature header because i couldn't find anything about it when i looked myself
11:08 Paul And like,
11:09 Paul oh,
11:09 Paul yeah, sorry.
11:10 Paul Apparently I made that up.
11:12 Paul Let's back it out.
11:15 Scott That is really interesting because this is something we've discussed many times, which is you have to have some expertise.
11:24 Scott You have to be the QA person in this stuff, and that makes it a little bit harder.
11:33 Scott It makes me worry about your friend who set up the whole marketing analytics program not knowing anything,
11:38 Scott because what if it's wrong?
11:41 Brad You know what I think, though?
11:43 Brad Let's explore this.
11:44 Brad Is it the QA person or is it the engineering manager?
11:49 Brad Like the more I.
11:51 Brad play with like paul your story is so on point like how i you know i've got hundreds of stories that are exactly the same and it feels to me like the role i'm almost always playing with with the ai coding agents is i feel like an engineering manager i feel like i'm helping give them make sure that the ticket is well enough described uh kind of
12:15 Brad a little bit like, do I know what you're good at doing that I don't have to get super specific about?
12:20 Brad Or do I know what you're bad at doing that I need to get more specific about?
12:23 Brad That feels like I'm managing an engineer.
12:25 Brad Like, you know, you know, different engineers are good or better at different things.
12:30 Brad And then I'm paying attention to it. And when it starts to spin out of control, I say,
12:34 Brad hey, stop, stop for a second.
12:36 Brad Give me a status report.
12:37 Brad What are you doing?
12:38 Brad What are you stuck on?
12:40 Brad What can I help
12:40 Scott Yep
12:41 Brad you with?
12:42 Brad And like literally those are the prompts I'm giving.
12:45 Brad Like literally the prompts are pause.
12:48 Brad I see you appear to be stuck.
12:49 Brad Please give me a summary report of what you're working on and what are your challenges and let me let me talk to you about it. And it comes back and it says, you know, oh, I I've been trying to figure out how to do geo fencing around the perimeter of the avalanche forecast zone,
13:04 Brad and I'm currently struggling on getting something that, you know, works and passes the test,
13:09 Brad you know.
13:10 Brad OK,
13:10 Brad let's talk about that.
13:12 Paul Yeah,
13:13 Paul one where I could like click the 100 emoji for that.
13:18 Paul Yeah, I mean, unknown unknowns are not limited to AI,
13:21 Paul right?
13:21 Paul And it is the engineering manager's instinct for, you know, the smell of, you know, something being unknown.
13:32 Paul Maybe we don't understand that. I'd like to dive in there.
13:35 Paul I'd like to hear more, you know, stop.
13:38 Paul I mean, I do think, by the way, I'd love to come back to your question too,
13:43 Paul Scott,
13:43 Paul which is I think like, what if it's wrong,
13:45 Paul you know, is an interesting one too,
13:48 Paul but I think it kind of loops back to what Brad was saying too.
13:53 Scott Back to the original question,
13:54 Scott it sounds like you started by treating the AI as kind of like a...
14:00 Scott programming team or a programmer that was going to follow your spec and then you evolved a little bit to just you you went from being a product manager who gave a the programmer a spec to a stakeholder who just said I want a thing that does this and it got hung up from time to time and you'd have to stop it or as Brad was saying start to interact with it so
14:27 Scott It changed a little,
14:28 Scott it sounds like, your process along the way, but it didn't, you still kind of treated it, well,
14:34 Scott no, that sounds like it definitely changed, I guess is what I'd say.
14:38 Paul Yeah, one,
14:39 Paul and,
14:39 Paul you know, I'm still experimenting, you know, I have more tools to play with now,
14:45 Paul you know, like Keyro won the $20 for you this month.
14:53 Paul lottery which in by the way it's like all the different little bits of opinionation and capabilities in all these different tools you know the features and sub features and different capabilities of Claude versus Gemini or anti-gravity versus
15:12 Paul um hero or you kind of the list goes on and on and there just so many like different interesting ideas and perspectives in there um and like steve yegi's beads is on my list of things to play with too which i haven't gotten a chance to try out yet
15:32 Scott The question for you, Paul,
15:33 Scott but also you, Brett, is it your view with all these tools and how they all have different
15:38 Scott looks you know different angles on it and the landscapes are evolving so rapidly you can see two ways that people approach this one is to say look i'm just picking this tool i'm going to call this the cold fusion way if you guys remember cold fusion which is i'm picking this tool it might not be the best tool but the i'm going to learn it i'm going to know it frontwards and backwards right versus
16:04 Scott you know you started out your butterfly approach then which is like we're going to jump from like you said twenty dollars a month here twenty dollars a month there what do you think if someone's just getting into this what do you think the right approach is to try a bunch of different things or to pick one and try to figure out how to make it work for you
16:18 Paul Do you want to go first, Brad, or I have an
16:21 Brad No,
16:21 Paul opinion.
16:21 Brad I've shared my opinion on this multiple times on the pod.
16:24 Paul Okay.
16:24 Brad So Scott better know what my opinion is, but I want to hear you. I want to hear from listening to you either,
16:31 Brad Scott.
16:32 Brad Okay,
16:32 Brad tell us, Paul,
16:33 Brad what's your opinion on this one?
16:35 Paul So I think like Markdown is the new code.
16:41 Paul Right. So those those specifications that you hone kind of that top level description, you know, if it expands out to an implementation plan or a set of documents,
16:52 Paul right,
16:52 Paul that's what's actually portable.
16:55 Paul And that first experiment that I took the same stack and gave it successively to Gemini and then Claude and then Codex and that was Gemini 2.5.
17:08 Paul pro got it first and then Gemini 3 got it later you know I had them critiquing each other's implementations I still do that I had CoPilot GitHub reviewing some code changes from Gemini today and then poking Gemini to ask what it thought of CoPilot's feedback but it's it's the natural language
17:30 Paul artifacts the top level artifacts that are you know the new code if you will and you know what we used to think of as the code you know is much less important right it's like the the output of the compiler from 40 years ago
17:46 Brad I like Paul's answer.
17:48 Brad My answer is similar but slightly different.
17:51 Brad And what I've been saying.
17:54 Brad uh scott to your question is absolutely i do not think people should be focusing on a tool and trying to get deep on that tool i think that is a fool's errand and i think that you it just is moving too fast and so getting good at one tool is the wrong approach um instead i think the right approach is get really good at
18:20 Brad working with all of them and understanding how to be that engineering manager like it's it's learn how to read read the code that it generates learn how to
18:35 Brad read the markdown that it generates,
18:37 Brad right?
18:37 Brad Like my current working style is I go into plan mode, like all the tools I'm using now all have plan mode.
18:43 Brad So I don't write the markdown anymore.
18:45 Brad I give them the problem statement and expect them to give me the markdown of their plan.
18:49 Brad Then I read that plan as if I'm an engineering manager to say, does this engineer understand the problem that I've given to them? And I critique their plan. So I critique the markdown.
18:59 Brad And then once I've had the conversation with the agent,
19:01 Brad the agent about their markdown and i'm satisfied with their markdown then i ask then i say build right and all of the all of the modern tools anti-gravity cursor you know all of them now have plan mode and
19:16 Brad And so that the skill that I'm focusing on, I must say this, please,
19:20 Brad please,
19:20 Brad please, if you're an engineer listening to this podcast,
19:22 Brad absolutely get good at describing problems and reading implementation plans. Now, this is something you already should have been good at before AI existed is reading implementation plans of your fellow developers and learning how to critique them. That is a skill that every engineer needs to have.
19:41 Brad have and it's that much more important now in the in the age of AI coding agents
19:47 Scott It's fascinating because advice I've given to engineers earlier in their career,
19:53 Scott Brad,
19:54 Scott when they say,
19:54 Scott how do I get ahead and so on and so forth,
19:56 Scott I say,
19:57 Scott look,
19:57 Scott get good at writing and implementation.
19:59 Paul You plan.
20:01 Brad Yep.
20:01 Scott Like once that's to me, that's like the key thing.
20:04 Scott If you can make a plan,
20:06 Scott if you can understand. So to write a plan, you got to know what is being asked of you and you got to understand how your system works so that you can know how to build it.
20:15 Scott And if you can write a plan,
20:16 Scott then you're at a fork in the road.
20:18 Scott You can get leverage from other people and inviting other people in essentially you're becoming a leader or manager or you can be you can write more and more sophisticated plans and become more and more.
20:27 Scott more of a climb up the engineering track,
20:32 Scott right?
20:32 Scott And it's interesting what you say there is it shifts from writing the plan to being able to evaluate the plan.
20:38 Paul Yeah.
20:39 Paul Yeah.
20:39 Paul Well, and to think about that, like if you were going to write a plan,
20:43 Paul before you could write a plan, you have to be able to also evaluate your own plan.
20:48 Paul Right.
20:48 Paul Like, how do you know that your plan's a good plan is you're effectively self-evaluating your plan.
20:53 Paul And to your point, Scott,
20:55 Paul like self-evaluating your plan,
20:56 Paul you have to understand,
20:58 Paul do you understand the thing you're being asked to do? So you've got to get really good at understanding how to ask things,
21:04 Paul how to understand what you're being asked for. Right.
21:06 Paul That's like that's one of the important skills. And then the other important skill that you described is understanding enough about your system.
21:14 Paul To be able to just to think about how to mutate that system.
21:18 Paul Right.
21:18 Scott Mm
21:18 Paul And
21:18 Scott -hmm.
21:19 Paul so this is this is like job one of the first time, the first week of every engineer on a new on a new team is they better focus on do I understand the system I'm about to start working on and how quickly can I get up to speed on learning this system, learning its learning its architecture,
21:36 Paul learning the data model.
21:40 Paul uh learning potentially learning its quirks and weaknesses and strengths and then i have the ability to start writing code right like if i don't understand the system i'm working on i can't think about it about how to mutate i can't reason about how to mutate it and so i you know more more than ever i'm thinking like that that
22:00 Paul That's how you that's how I believe you have to interact with this AI is it's like another developer on your team. And so how would you evaluate?
22:10 Paul How would you participate in assisting another developer building a feature on your code base?
22:15 Scott I want to riff for a second on understanding the code. In the last couple of weeks, someone in our company hooked up an AI chatbot into Slack,
22:25 Scott which...
22:27 Scott However they get it has awareness of our code base.
22:30 Scott And so now we're seeing people ask questions like this.
22:36 Scott Where can I find the name for the partner in invoices?
22:39 Scott And it responds with some answer like here's where it is in the coder. What are the definitions of user suppression levels and where can I find them?
22:46 Scott And that's pretty interesting.
22:48 Scott Both of those questions were asked not by engineers.
22:51 Scott by other people in the company who needed that information for something um but we're giving engineers you said okay you start on a new team you go and try to understand the code we've abstracted that away now we're abstracting it away from even the engineer needing to understand the code and now the folks outside of engineering can do it which is pretty cool okay let me let me let me pose another question deal
23:13 Scott Paul, you were a manager of engineers at Salesforce for a long time.
23:16 Scott There is, it seems like in the field, pockets of apprehension, if not downright resistance.
23:25 Scott And I can understand that.
23:27 Scott I think it was Upton Sinclair who once famously said it's hard to get someone to understand something when their salary depends upon not understanding it.
23:35 Scott And that's what a lot of these engineers feel like.
23:39 Scott What's your take on as engineering leaders,
23:42 Scott engineering managers,
23:43 Scott if we should be mandating, basically forcing people to use these tools?
23:51 Scott That's a question I've been struggling with for the last, say, three to six months.
23:56 Scott Should I just, I don't know how I would make my team do it, I can't force their fingers to type certain things, but I've been pondering that question.
24:04 Scott What's your take on that?
24:05 Paul Gotcha.
24:06 Paul Well, there's there's the Spotify memo,
24:08 Paul right,
24:09 Paul which is,
24:09 Scott Mm
24:10 Paul you
24:10 Scott -hmm.
24:10 Paul know, before you come and ask me for money or a headcount, you know, be prepared to provide an argument for why you couldn't get this done with with AI.
24:22 Paul So I think.
24:25 Paul You know, the resistance is due to a sense of, you know, disempowerment or some kind of hypothesis on that the AI is going to somehow supplant, you know, the engineer.
24:40 Paul And I think that's, I think that's misplaced, right?
24:44 Paul It's a new tool.
24:45 Paul It's a way to magnify your impact,
24:50 Paul right? To multiply your effort.
24:53 Paul Um,
24:53 Paul and I think people really need to, um,
24:58 Paul go into it with the mindset of how do I make this work versus why didn't it work when I tried something?
25:08 Paul Um,
25:09 Scott Mm
25:09 Paul because
25:09 Scott -hmm.
25:09 Paul it definitely can be made to work and the potential gains are significant.
25:15 Paul Um,
25:16 Paul and it's not that it's going to replace you, it's that it's going to multiply you.
25:19 Paul It's going to exponentiate you.
25:21 Paul you know if you're starting at zero you can get to one if you're starting at one two starting at two four um so i think you know this like you didn't like typing up giant yaml documents to feed to something great get the ai to do that right but the person who's in charge of figuring out what the ai should do you know how to connect that to the business and what's supposed to happen
25:48 Paul and that's you um so yeah grab grab that tool and get busy
25:54 Scott Another bit of resistance I get, here's a quote I just got this week in a Slack message.
25:58 Scott AI-generated code isn't good enough yet for complex business logic.
26:03 Paul They're just wrong,
26:04 Paul Scott.
26:05 Paul Yeah.
26:05 Paul They're just wrong.
26:07 Paul Would you mandate it? And if so,
26:08 Paul how?
26:08 Paul I, well,
26:09 Paul okay,
26:09 Paul so here's my quick answer.
26:11 Paul I have not mandated it in the teams that I'm working with,
26:14 Paul but I do say to every single person, both in team meetings and one-on-ones, you know, look, man,
26:20 Paul we got to move fast.
26:22 Paul We got to move fast.
26:24 Paul Right. I work in startups, so it's all about moving fast.
26:27 Paul We got limited runway.
26:27 Paul We got to like make something happen,
26:29 Paul you know, and so it's like, we got to move fast and I need you to be moving as fast as you possibly can.
26:35 Paul And, you know, I mean, there's, you know.
26:40 Paul a bunch of my teams that i work with like they see me doing it like i i'm a i'm a player coach right i'm i'm actually also writing code and they see that it's like i already was a fast developer and they just see me easily 10 times faster and so it's like look we got to move fast and so i feel like if you don't see other people being productive if you see other people and you don't see that they're being more productive with it
27:07 Paul And you genuinely are missing out on that, like you're missing that you're it's it's not apparent to you what is happening to the rest of the team making great strides on this.
27:18 Paul then you're probably not a good fit for the team already right like that you're already sort of signaling that you're not a good fit for the team right so i don't think it's about mandating it's more like this is what we're doing we're moving fast we're a team we gotta we gotta we got we're burning daylight let's move right so that's my take
27:38 Paul I agree with like burning daylight is a great.
27:43 Paul you know a great analogy like we've got some place to go we need to get there with the foods we have on the ground um you know i think it's useful to you know pivot back and forth between the why and the how like i was saying and when you find somebody with an objection like that i've chatted with some friends who work on like legacy c++ code base
28:03 Paul And the comment was like, oh, my God, no,
28:06 Paul like,
28:07 Paul don't don't let, you know, anything near that code.
28:10 Paul Why do you say that? Well,
28:13 Paul there are so many undocumented invariants and so much tribal knowledge. And so that actually like, wow,
28:23 Paul I would love to.
28:25 Paul itemize that technical debt i would love to find a way to burn it down like i've got you know 50 headcount like like let's throw some money at some of these tools and start doing those things to rein in you know those areas of unknown
28:45 Paul But I think just a little bit of back pivoting back and forth between, you know, why and how, why not? Well, how do we solve that?
28:53 Paul You know, and enough application to that will get you somewhere where you can be productive.
28:58 Paul Yeah.
28:59 Paul And I'll just say like, you know,
29:01 Paul we, I think that was.
29:05 Paul on one of the earlier episodes of the podcast that we did when um we talked to i don't remember who it was but they made the point like if you if you tried this you know at the time it was like if you tried this six months ago and you didn't like the results of it then try it again right and i like it's just getting so much better at hard stuff man i i had this bug i had to fix uh last week
29:31 Paul And it was like it was an obscure race condition.
29:35 Paul I it was hard to reproduce,
29:38 Paul you know, and but I knew the subsystem that it was in and I had data about like when it fails, this is the these are the signals that I have to know that it's failing.
29:50 Paul And I threw anti-gravity. I threw this at anti-gravity and basically it came back and it absolutely identified the race condition.
30:00 Paul It had full flow graphs saying here's exactly the race condition that will exist.
30:05 Paul If this timing happens this way,
30:07 Paul this is going to happen, you know, and it came back with a proposed fix and I was like,
30:14 Paul okay,
30:14 Paul I believe you, that's plausible.
30:17 Paul my code reviewers aren't going to accept it unless I give them a working example of forcing the race condition.
30:23 Paul So go back and modify the code,
30:25 Paul add in the appropriate like delays and timers to cause the race condition to happen.
30:30 Paul And then they'll believe me that we fixed it because this has been haunting us for like six months,
30:35 Paul you know, and it did it and it did it in like.
30:40 Paul 30 minutes and this is a bug that has been haunting us for six months and it's it's one of the most complex things that you can deal with like race conditions in asynchronous systems and it nailed it so
30:56 Scott That's pretty impressive.
30:59 Paul keep trying
31:01 Paul And don't be surprised when it gives you something completely hallucinated like making up an API call that doesn't exist because it's still going to do that.
31:11 Paul Yeah, interesting I'm somewhere under the hood is something stochastic so but
31:19 Paul Yeah, yeah, it's still probabilistic. It's not actually deterministic.
31:23 Scott I have a technical question for you Brad
31:27 Scott You know, you were talking a few minutes ago about how you were interacting with I don't know which model it was and it was telling you what it was thinking.
31:33 Scott I actually don't know how it does that part of it,
31:35 Scott how it how it has awareness of what it's what it's doing.
31:39 Paul This this I don't know how much you've you've looked into this yourself,
31:42 Paul Paul,
31:42 Paul and jumping into this. So I feel like I'm a little bit of a naive outsider on this, but this is my this is my understanding. So one of the biggest advances that I've seen that's happened in the last, I'll say year,
31:55 Paul is that.
31:58 Paul This is what we talk about the latest models are.
32:04 Paul agentic friendly right that's kind
32:06 Paul Hmm
32:06 Paul of the language that you hear in the in the frontier model development community and the sort of like vc space and sort of like what are we doing here and so they say basically like the models are more agentic friendly and what i understand that to be and you can see it in cursor and you can see it in anti-grab you can basically see it in these ides is they started like exposing this i think to those power users like us that wanted to like see it
32:30 Paul see it thinking and what it is is basically that the llms that are agentic friendly they have they they basically you can you can hook them with if you need to run a tool this is a tool that does the following right so for example
32:51 Paul In essentially the setup of the LLM, they say,
32:55 Paul here's a tool that will list code for you.
32:57 Paul Here's a tool that will do a diff for you. Here's a tool that and they basically pre give the LLM access to a bunch of tools and the tool and the LLMs are good at going,
33:09 Paul ah,
33:09 Paul I understand the I doesn't really understand it, but it's like.
33:13 Paul If I get to the stream of tokens that say,
33:16 Paul I should compare two files,
33:19 Paul then
33:19 Paul Mm-hmm.
33:20 Paul run this tool and look at the results and expect it to be this way. And so it's this combination of the LLMs happen to be agentic friendly in that their training and their post training allows them to be good at.
33:39 Paul knowing to instantiate a tool and knowing how to read the results of that tool and in the ides nowadays they're basically blurring that stuff past you really quickly as it's working so you'll literally like i just like uh i've got this race condition bug i know it's in this code right
34:00 Paul It says, you'll see, like blink on the screen real quick,
34:03 Paul the user wants me to find a fix to a asynchronous race condition bug.
34:09 Paul I am going to analyze the code in the modules that they said. Let me search for this keyword that the user used.
34:15 Paul I'm now searching for this keyword.
34:17 Paul I've now found it in this file.
34:18 Paul Let me read that file.
34:19 Paul I am reading that file.
34:20 Paul I'm comparing that file to the last version that they said the race condition didn't exist in.
34:25 Paul You know, it's basically spinning all that.
34:27 Paul And it's just the model kind of like eating its own tail, feeding
34:31 Scott It's
34:31 Paul on it.
34:32 Scott like narrating itself
34:32 Paul It's narrating itself.
34:34 Paul Yeah.
34:34 Paul So
34:35 Paul Yeah,
34:35 Paul that's
34:35 Paul just fascinating to me.
34:36 Paul what I mean when I say it's thinking through.
34:39 Paul And Paul,
34:40 Paul you mentioned that same thing. You could see when your avalanche app was trying to do it, you could see it was spinning out on trying to solve a piece of the problem.
34:47 Paul Yeah,
34:48 Paul I'm doing this, I'm doing that.
34:50 Paul And there's that aspect of recursion, you know, back in the first days of, you know, the generative approaches,
34:58 Paul right,
34:59 Paul that was strictly input and output.
35:01 Paul And then people started kind of looping it back. It's like, OK, well,
35:06 Paul generate a plan based on the prompt and then kind of work your way down, generate,
35:14 Paul generate this like gradual escalation of capability until you get to something that you know how to do.
35:22 Paul Like, OK, I know how to read a file and I'll look up something in a database.
35:27 Paul um you know those those things can kind of take you sideways right if you don't have the right set of tools at your disposal you can do something like read the entire database instead of writing a query to get the information that you need um but yeah the that that whole space and the fact that it works sort of interesting and really magic
35:53 Paul And there's so many cool,
35:56 Paul like, I have yet to spend much time exploring multimodal AI and really curious about things like ternary LLMs, right,
36:10 Paul the possibility to compress this sort of approach to problem solving down.
36:18 Paul To places where you don't need a GPU anymore and it's suitable for you put in your light switch.
36:23 Paul Yeah, smart light switch.
36:24 Paul Yeah, totally
36:25 Paul So yeah,
36:27 Paul I think there's so much cool stuff out there.
36:31 Paul totally yeah
36:33 Paul Well,
36:34 Paul Brad,
36:34 Paul should we end with our AI game?
36:38 Paul I'm going to bring up Hacker News.
36:39 Paul Don't look at don't look don't look ball
36:42 Paul Oh,
36:42 Paul It only works, the game only works if you don't look at Hacker News yourself.
36:46 Paul okay.
36:47 Paul Yeah, not that kind of looking.
36:49 Paul Okay,
36:49 Paul I'm not looking.
36:50 Paul I am looking at Hacker News.
36:52 Paul How many of the top 10 articles are about AI?
36:55 Paul Oh, 10.
36:58 Paul I'm taking the under.
36:59 Paul I'm taking the under on that.
37:01 Paul Taking
37:01 Paul Let's
37:01 Paul the under.
37:01 Paul see.
37:01 Paul Well,
37:02 Paul I'm trying to think like what has happened recently.
37:06 Paul You know, there's there's been no big cloud flare outage sky volute is weeks ago.
37:12 Paul Netflix is buying Warner Brothers.
37:15 Paul That's probably not going to build on Hacker News unless there's some show that people won't be able to watch that they really like.
37:20 Paul Yeah, I think my guess is still going to be high.
37:23 Paul You think 10 is wrong?
37:24 Paul That's too high.
37:25 Paul I'm going to go with – wait,
37:27 Paul I'm going to go – here's my guess.
37:28 Paul I'm taking the under.
37:30 Paul But
37:30 Scott Yep.
37:30 Paul I'm going to be even old,
37:31 Paul so we'll see how much this read. I'm going to go four.
37:34 Paul Four of the top ten articles are about AI.
37:37 Scott Okay,
37:37 Scott number one,
37:38 Scott Netflix to acquire Warner Brothers.
37:41 Paul Look at that.
37:43 Scott Number two,
37:44 Scott Loud Flare outage on December 5th.
37:47 Paul There you go.
37:48 Paul So far, no AI.
37:51 Scott Number three, Gemini 3 Pro,
37:53 Scott the frontier of vision AI.
37:55 Paul There we go. That's
37:56 Paul an Cool,
37:56 Paul AI.
37:56 Paul That's
37:56 Paul cool.
37:56 Paul one.
37:58 Scott Number four,
37:59 Scott item potency keys for exactly once processing.
38:03 Paul No, not AI might
38:04 Paul Yeah.
38:06 Scott Number five,
38:07 Scott perpetual futures.
38:09 Scott I don't know what it's about,
38:10 Scott but the URL is bitsaboutmoney.com.
38:14 Scott So I'm going to say no AI.
38:16 Paul be crypto related.
38:17 Scott Could be.
38:18 Scott Number six,
38:19 Scott Frank Geary has died.
38:22 Paul Age 96.
38:23 Scott Not a party.
38:25 Paul I hadn't heard that, but there you go.
38:26 Scott pardon me number seven feels buzz in css that's almost like the opposite of ai
38:34 Paul BizBuzz in CSS,
38:35 Paul that's awesome.
38:37 Paul That's really cool.
38:37 Paul I want to see that program.
38:39 Paul So I have to comment though so the last time I got a chance to go back to programming which was nearly 20 years ago
38:48 Paul Fizz Buzz was all the rage,
38:51 Paul you know, wrote Fizz Buzz in Haskell before writing some other stuff in Haskell, but it's just hilarious to me that Fizz Buzz is still out there.
39:01 Paul By the way, I think we both are at this point. There's no chance that it's even four,
39:06 Paul right?
39:06 Paul Because you're at you're at seven and we got one.
39:08 Paul Yeah, yeah, the price is wrong.
39:10 Scott Yep.
39:11 Scott Number eight.
39:13 Scott useful for you Paul patterns for defensive programming in Rust
39:18 Paul Wow,
39:19 Paul like it?
39:20 Paul I thought, wait, Jose,
39:21 Paul I thought the whole point of Rust was you didn't need defensive programming because the language guaranteed you couldn't screw up.
39:28 Paul No,
39:28 Paul no, no. The language with the right idiom and flag will do that. But it's like Scala,
39:34 Paul right?
39:34 Paul There's plenty of rope to tie yourself up if you want.
39:39 Paul All
39:39 Scott Where
39:40 Paul right,
39:40 Scott are you?
39:40 Paul number 10.
39:41 Scott Oh, we're on number nine.
39:42 Paul Okay,
39:43 Paul number nine.
39:44 Scott Framework sponsors catchy OS.
39:47 Scott I had to look that I had to click on that link because I didn't even know what that meant.
39:51 Scott Hey,
39:52 Scott catchy OS community,
39:53 Scott we have some massive news to share today framework,
39:56 Scott the company behind the modular and repairable laptop.
39:59 Paul Lucian is now sponsoring Catchy OS.
40:03 Brad Okay.
40:03 Paul So I don't really know what that's about.
40:05 Paul Number 10 should almost be the entire theme of this entire podcast in its entirety from since we started,
40:15 Paul Brad.
40:15 Brad Okay.
40:16 Paul Most technical problems are people problems.
40:20 Brad Yep.
40:23 Brad Accurate.
40:25 Brad Maybe AI,
40:26 Brad there might be some AI in there.
40:28 Paul A blog post by a guy named Joe Schrag,
40:33 Paul who I've never heard of before,
40:34 Paul writes about that.
40:38 Paul So there you go.
40:40 Paul That's the AI game for today.
40:42 Paul Apparently,
40:42 Brad my only
40:43 Paul AIs go and bust.
40:43 Brad one out of ten well you're you i i should have i definitely should not have taken your money on this one because i i i knew right out of the gate that you were a rookie at this game when you said 10 like it's
40:57 Paul Oh, it's not based on the game.
41:00 Paul That's okay.
41:03 Brad The rules are clear.
41:04 Brad You guess how many articles in the top 10 are about AI.
41:09 Brad Well, Paul,
41:10 Brad thanks for coming on the pod.
41:13 Brad Thanks for talking to us about this.
41:15 Brad I actually feel like...
41:16 Brad uh i'm you know in in the in the camp of like using ai to speed up developers and to to build software faster and more reliably i am quite bullish and i love that you kind of decided to dive in and kind of like learn for yourself like really kind of test it and experience it and i love that you kind of
41:43 Brad came with these, you know, the approach that you took.
41:46 Brad And it's pretty exciting that you're out there attacking this.
41:50 Brad And so thanks for sharing your experience.
41:55 Paul Thank you. Thank you for having me. Thank you for having me on. I mean, this is just so fun.
42:03 Paul And I feel like, you know, there's kind of some of that giddy joy of.
42:10 Paul making software again.
42:13 Paul So yeah, thank you very much.
42:15 Brad Cool. So how do we summarize this?
42:17 Brad Everybody,
42:18 Brad if you're skeptical about AI code,
42:20 Brad just discover the giddy joy of writing software again.
42:24 Brad That's the words of wisdom from Paul Brown.
42:31 Paul Right.
42:31 Paul And then smash rate and subscribe.
42:33 Brad Smash that subscribe button.
42:35 Brad Rate and review.
42:37 Brad And tune in next week to another episode of the How Many CTOs Does It Take podcast.
42:42 Paul Bye-bye.
42:44 Brad This episode of How Many CTOs Does It Take podcast was edited by Saffron Efigab.
42:49 Brad Our theme song was AI-generated by Brad.
42:53 Scott Please follow us on all the social media platforms at at HowManyCTOsPod.
42:58 Scott You can find us on YouTube,
43:00 Scott LinkedIn,
43:01 Scott Instagram,
43:01 Scott Facebook,
43:02 Scott Threads, X,
43:02 Scott TikTok and Patreon or at our website at HowManyCTOsPod.com.
43:08 Scott And remember to rate, review and subscribe on the podcast platform of your choice.
43:13 Brad How many CTOs
43:15 Scott Thanks for listening.