Transcript: The K-Shape of AI Development: Managing Digital Interns
0:00 Brad The people that are going to excel are the engineers who have a team leader,
0:05 Brad tech leader mindset.
0:27 Scott Welcome back, everybody.
0:29 Scott We know you've been waiting for another episode of how many CTOs does it take?
0:35 Scott And in this occasion,
0:37 Scott this instance,
0:38 Scott my name is Scott Porad, and this is my trusty co-host.
0:41 Brad Brad Hefta-Gaub,
0:42 Scott But the question today is not how many CTOs does it take? The question is how many tokens does it take?
0:50 Brad how many tokens does it take?
0:54 Scott Brad, we were just chatting before we got recording here and I said we should be recording this.
1:00 Brad Yes, we should be.
1:01 Scott You got your cursor wrapped.
1:05 Brad My year
1:06 Scott I
1:06 Brad -end got review
1:06 Scott a
1:06 Brad is
1:06 Scott Spotify
1:07 Brad better.
1:07 Scott wrapped.
1:09 Scott You got a cursor wrapped.
1:11 Scott My Spotify wrapped says my Spotify age is 32.
1:14 Scott Did you get a Spotify I age?
1:15 Brad didn't get it.
1:16 Brad No.
1:18 Brad I don't use Spotify.
1:19 Brad My Spotify account is like the family account,
1:22 Brad so it's like random stuff that my wife and kids listen to.
1:26 Scott The sad thing about having your Spotify age be 32 when you're my age is at first you think, oh,
1:36 Scott that's cool.
1:37 Scott I'm listening to music 20 years younger than my age.
1:42 Scott And then you think, wait a second, 32 isn't young?
1:47 Brad You're a millennial. Your music tastes are millennial.
1:52 Brad Oh, man,
1:53 Brad that's
1:53 Scott Well,
1:53 Brad funny.
1:54 Scott tell me about your cursor wrapped.
1:58 Brad Well,
1:58 Brad I,
1:58 Brad you know,
2:00 Brad I.
2:01 Brad I was not necessarily proud of this. I was kind of like,
2:04 Brad wow,
2:05 Brad that doesn't that seems like a smaller number than I was expecting.
2:08 Brad But then I shared it with some of the devs that I work with and they were all like, what the hell?
2:13 Brad Anyway,
2:13 Brad 2.9 billion tokens in 2025.
2:16 Brad That's what I personally used within Cursor alone.
2:20 Brad 2.9 billion
2:20 Scott What
2:21 Brad tokens.
2:21 Scott is the cost of a token?
2:24 Brad Ah.
2:25 Brad That's a great question.
2:26 Brad And I'm an early enough adopter of Cursor that I'm on the legacy plan.
2:32 Brad I'm not going to call it the grandfathered plan.
2:35 Brad I'm on the legacy plan.
2:36 Brad And so I'm only paying 20 bucks a month.
2:39 Brad And like everyone else has to pay per token,
2:42 Brad but I'm like grandfathered in or excuse me, legacyed in.
2:46 Scott I wonder what the actual cost of a token is.
2:50 Brad Yeah.
2:52 Brad Yeah.
2:52 Scott Probably $10.
2:54 Brad No.
2:55 Scott I'm just kidding.
2:56 Scott I'm just kidding. But these companies are spending so much money.
2:59 Scott I read today that Microsoft spent $88 billion on AI last year.
3:05 Brad Well,
3:05 Brad that sounds about right.
3:06 Brad But then that means my 2.9 billion tokens are not nearly that much. They spent a lot more than that.
3:15 Scott Yeah.
3:15 Brad That's a lot.
3:17 Scott Well, that's cool.
3:17 Brad So, yeah,
3:19 Brad so I think there's a little, it's funny because I feel like even some of my top developers,
3:23 Brad so like one of the top developers I work with after I posted my $2.9 billion,
3:27 Brad he was like, he responded saying,
3:30 Brad oh no,
3:30 Brad I need to up my level because he'd only had 700 million tokens.
3:34 Brad So he was a little bit like,
3:35 Brad wow,
3:36 Brad like, you know, what's the map on that?
3:38 Scott I
3:38 Brad Four times more
3:38 Scott mean, this is an interesting thing because there's a lot of different ways one can.
3:43 Scott To interpret token usage.
3:45 Scott It's kind of like clocks.
3:48 Brad uh yeah maybe maybe it's a little bit like clocks i mean i would tend to agree that like as as you know you and i love to talk about engineering efficiency and and you know every time we talk about what's the right metric i always love to say it's clocks of course right because it's the worst measurement ever so i wouldn't be so i think it's a legitimate to say like uh what did you do with those tokens brad right like were there you know
4:14 Brad It's not a bad case to make that throwing around token numbers is a little bit like throwing around lines of code.
4:22 Brad So I think that's a fair point.
4:27 Brad I would say that like, you know, and we kind of hinted at this in our last episode of 2025,
4:35 Brad like, where do we think, what do I, what my opinion about where I think AI coding is going in 2026 and what, you know, I don't want to turn this into, it's not a predictions, you know, I don't want to say a predictions episode or anything like that, but I think that we're going to see a dramatic.
4:51 Brad um you know uh a k-shaped not a k-shaped economy but like a k-shaped reaction to ai coding tools right the the halves are going to grow faster and the have nots are going to fall further behind faster um and i think it's going to come down to that that central question of what you're you're scratching at which is it doesn't matter how many tokens you use it matters how
5:16 Brad effectively used the tokens that you use.
5:19 Scott I agree.
5:21 Scott And it might actually be an effective jumping off point for something that might actually be useful and practical to the 39 people who listen to this show, which is what are the things that the effective people are doing or are going to do that?
5:41 Scott will make them continue to be effective or more effective.
5:44 Scott And I have one thing in my list. I'm curious to know,
5:47 Scott you are much more of the power user here than I am. So I'm curious to know what you think those things are.
5:51 Brad You want to share your item that you think is the leading indicator of this?
5:56 Scott I see.
5:57 Scott two well actually i said one thing but i actually think it's two things the first is engineers are going to figure continue to figure out how to take large tasks and break them into smaller tasks that the agents can do one by one and i just i will come back to this like 50 times in the next year there was just this thing that kent beck it didn't just happen he put it on
6:23 Scott Probably LinkedIn is where I saw it, where he said, you know, 90% of my skills are becoming obsolete, but the remaining 10% are 10,000 times more valuable.
6:31 Brad Yep.
6:31 Scott And knowing how to break the work down to describe each step to the agent is one of those skills,
6:39 Scott which jibes with something I've said.
6:45 Scott Many times in this podcast,
6:46 Scott I'm sure,
6:47 Scott which is the advice I always give to engineers about developing their career is get to the point where you can write an implementation plan or text back or whatever you want to call it so that other people can understand your design and start collaborating with you on it is to me the most important skill to develop because then you're at a crossroads in your career. You can decide if you want to be a manager or if you want to stick with.
7:12 Scott being an IC,
7:13 Scott but until you can actually take requirements and translate them into work for someone to do,
7:18 Scott that's where you've got to get to.
7:20 Scott So that fits with one of the things I'm seeing is being able to break that work down.
7:26 Scott The other thing I'm seeing is AI is accelerating things.
7:32 Scott It's not making it free.
7:34 Brad Here
7:35 Scott And what do I mean by that? I'll give you two examples.
7:38 Scott A few examples. I'll start with one. I've probably talked about this in the podcast before because it came from earlier this year,
7:44 Scott an engineer who updated a lot of code, but it saved him tons of time.
7:49 Scott But he said it took him like four or five hours to get the prompt right, you
7:53 Brad Yeah, yeah
7:54 Scott know, of experimentation. And then it saved him probably weeks of time.
7:58 Scott We had another case that came through recently.
8:03 Scott an engineer they had about a little less than 2 000 tests and these tests were flaky and low quality for a variety of reasons and i mean he spent he spent an afternoon it was four or five hours he spent an afternoon setting up a process a set of steps for the agent to go through first by getting some third-party criteria for what
8:28 Scott what a good test looked like then evaluating each test and giving it a letter grade then asking the robot to go through and I've been doing a bunch of steps and ultimately having it refactor some of the things at the bottom or at the not at the bottom at the end again multi-step process learning how to take these big tasks and it took them a bunch of time and then an engineer I just talked to recently this I thought this was really interesting he created
8:55 Scott created an entire GitHub repo which consisted of mostly MD files.
8:59 Brad Mm-hmm.
9:01 Scott Just context.
9:02 Brad Yep, yep.
9:03 Brad This is a huge,
9:05 Brad by the way, you know, you talked about the notion of like what our best practice is.
9:08 Brad And so for sure, great documentation in an MD file in your directory trees,
9:15 Brad that is gold for the agents. They need that.
9:20 Brad Give them more of that.
9:22 Scott They needed this was not even really in the directory.
9:27 Scott free of the code it was separate he was trying to get you he was he is trying to get i'm just looking for here i had i looked at it earlier today he was trying to get um
9:40 Scott The robot to containerize something that's currently running on bare metal,
9:44 Scott like
9:45 Brad Uh-huh.
9:45 Scott an older application.
9:47 Scott And he set up a number of, and can we just tell you what these files are? He's got AI agent and Quickster Yep. MD implementation plan summary MD,
9:57 Brad Yep.
9:57 Scott containerization implementation plan MD,
9:59 Brad Yep.
10:00 Scott implementation plan checklist,
10:02 Scott containerization goals MD,
10:04 Scott migration guide MD.
10:05 Scott The previous containerization attempts MD,
10:08 Scott which because lessons learned from previous attempts,
10:10 Scott Yep. relevant confluence pages MD.
10:13 Brad Yep.
10:14 Scott kids have other repositories like
10:16 Brad Yep.
10:16 Scott and then you know just all just pages pages just describing all this context and what to do he's been working on this for like three or four days
10:24 Brad Yeah,
10:24 Brad yeah, totally,
10:24 Brad totally.
10:25 Brad So here's my, I'm going to just going to come right out and say the magic words that matter for all of this. And you know, I already said K-shape, K-shaped economy when it comes to dealing with AI coding agents.
10:41 Brad The haves are going to excel.
10:43 Brad The have-nots are going to fail faster and faster.
10:45 Brad And it all comes down to not only what you said about the concept of being able to really break down work,
10:52 Brad but I believe it's all about being a tech lead or an engineering manager.
10:59 Brad The people that are going to excel are the engineers who have a team leader,
11:05 Brad tech leader mindset.
11:09 Scott Mm
11:09 Brad But
11:09 Scott hmm.
11:09 Brad instead of leading other developers,
11:11 Brad they are leading the robots.
11:13 Scott Mm And hmm.
11:13 Brad so that's the money shot.
11:15 Brad That's the thing that matters the most.
11:17 Brad And so I will say I think the reason that I'm going to make the case that my 2.9 billion tokens are incredibly effective tokens because I was operating at a, you know.
11:29 Brad tech lead engineering manager architect level with with the robots for those 2.9 billion tokens so imagine that that was effectively you know 300 junior engineers that I had working for me and I managed them like junior engineers
11:45 Scott Yeah, one of the people on our team said you have to think about the agents as really eager interns.
11:52 Brad Yeah. And so what does that mean?
11:54 Brad So let's kind of go to the next level,
11:57 Brad right?
11:57 Brad So one,
11:58 Brad it means being really good at describing the problem space,
12:01 Brad right?
12:02 Brad Like you said,
12:03 Brad and I agree with you that like being able to break down the problem,
12:06 Brad explain the problem,
12:08 Brad capture the statement of the problem,
12:11 Brad that's what your engineer that's created all these MD files for this containerization project,
12:16 Brad that's what they've done.
12:17 Brad Right.
12:18 Brad Now, that's only the start, though,
12:20 Brad because if I just do that, but I am not good at evaluating progress and output.
12:33 Brad Right.
12:33 Scott Mm-hmm.
12:33 Brad The other thing that a tech lead or an engineering manager does is they have an understanding of what.
12:40 Brad of what right-sized progress looks like and what right-sized output looks like,
12:46 Scott Mm
12:46 Brad right?
12:46 Scott -hmm.
12:46 Brad So they can evaluate the technical output to say this is correct or this is not correct.
12:52 Brad But more importantly,
12:54 Brad or equally importantly,
12:56 Brad they're able to evaluate the progress against the goal as it's happening and course correct.
13:04 Brad Right. So what's the classic story of the really good engineering managers you've worked with in your career?
13:12 Brad They're the ones that get.
13:15 Brad compelling results out of all of their engineers.
13:18 Brad And they're never coming to you saying like,
13:21 Brad oh yeah,
13:22 Brad this engineer,
13:23 Brad you know, please get them off my team.
13:25 Brad They suck. They can't do their job, right?
13:28 Brad You don't ever hear that from the really good engineering managers,
13:30 Brad right?
13:31 Brad They're in there figuring out how to make even the worst performers perform well.
13:36 Brad And the reason they're able to do that is because they know what
13:42 Brad They know not only what good results look like, but they know what it looks like when the engineer is struggling and
13:50 Scott Mm-hmm.
13:51 Brad the AI bots struggle and they continue to struggle and they're going to continue to struggle.
13:58 Brad They're getting better, but they struggle.
14:01 Brad And so.
14:03 Brad This is why I think there's going to be a case-shaped reaction to more and more use of AI agents, and this is why measuring the number of tokens is a dangerous thing.
14:14 Scott Mm
14:14 Brad You'll
14:14 Scott -hmm.
14:14 Brad burn through a bunch of tokens in a world where you let the bots go off the rails,
14:21 Brad you're not good at evaluating their progress.
14:26 Brad They churn out.
14:28 Brad Slop, right?
14:29 Brad We all hear
14:29 Scott Mm
14:30 Brad the term AI
14:30 Scott -hmm.
14:30 Brad slop,
14:31 Brad right?
14:32 Brad And so whenever I see somebody on LinkedIn or on Twitter basically saying, oh,
14:36 Brad AI coding sucks,
14:37 Brad it just generates slop.
14:39 Brad My response to that is,
14:41 Brad I bet that person would be a horrible engineering manager.
14:46 Brad Like they only know how to evaluate their own work.
14:49 Brad They don't know how to evaluate other engineers' work.
14:52 Scott Yeah, well,
14:53 Scott there's two parts there.
14:54 Scott It's not just evaluating the work. I think the words I want to use to say evaluate the process. And what I mean by that is like, as an engineering manager,
15:03 Scott if I had an engineer.
15:05 Scott Who, you know, went for a couple of days of the easiest way to describe this would be if their tickets stayed in the same column on the board for a couple of days.
15:14 Scott It's like a sign. Actually,
15:16 Scott not even a couple of days, a day or two, you know, two stand ups in a row and like that ticket's still there.
15:21 Scott It's like, OK, that person's stuff. Something's wrong.
15:24 Brad Yep.
15:25 Scott Actually,
15:25 Scott I don't know what's wrong.
15:27 Scott Something's wrong. Right.
15:28 Brad Yep.
15:30 Scott And so it's not whether the work product that person was creating was good or not, but they're actual like they weren't making any progress.
15:38 Brad Yeah, exactly.
15:39 Brad And I think that that what I'm saying is that is a 100 percent happening with all the AI coding agents. And if you let them get over their over their skis, then and you don't and you don't course correct quickly,
15:52 Brad then they're burning tokens,
15:54 Brad they're wasting their time, they're wasting time,
15:56 Brad they're burning tokens and they're wasting your.
15:57 Brad in your time and they're increasing the likelihood that they're going to produce slop yep
16:02 Scott Okay,
16:02 Scott I have a question for you, something I have never tried or never even thought to have tried until now.
16:07 Scott If you have an engineer who's struggling with something,
16:11 Scott you say to them, you sit down with them, you talk through the problem, you can point in the right direction,
16:15 Scott you say work on this, you say let's check in after lunch,
16:18 Scott right?
16:19 Scott Let's see how it's going.
16:20 Scott Can I say to an agent,
16:23 Scott Like pick a time frame, 15 minutes,
16:25 Brad Yeah.
16:25 Scott 30 minutes.
16:26 Scott Hey,
16:26 Scott do this thing and report back to me. If you're not done in 30 minutes, just stop and
16:32 Brad Yeah.
16:32 Scott let's discuss.
16:34 Brad Yeah.
16:35 Brad So my quick answer to that is the tooling has been changing and evolving in such a way that the.
16:44 Brad I would argue that the development tool set companies,
16:47 Brad Cursor being a good example of this,
16:51 Brad they have recognized this challenge that you and I have talked about, which is that the agents can get off the rails if you give them too big of a task.
16:58 Brad And so they've been changing their tooling to do to essentially add those checkpoints.
17:03 Brad And you might recall like and if you go back into our old episodes, you might remember that I've said things like early on, like, you know.
17:10 Brad Nine months ago,
17:11 Brad one of my the first prompt that I would always give,
17:15 Brad I would say, do not write any code until you've told me your plan. Like I had explicitly say that to the agents and now cursor has plan mode. So you use plan mode instead.
17:27 Brad And so that's one of the examples of the guardrails that the tooling companies have put in to effectively give you that functionality that you're talking about.
17:36 Brad Like, hey,
17:37 Brad give me your thought,
17:38 Brad give me your implementation plan,
17:39 Brad check in with me before you move on.
17:42 Brad It's not necessary.
17:43 Brad I've not yet seen Cursor implement anything where the agent stops when it gets stuck and says, hey,
17:53 Brad I'm stuck.
17:53 Brad stuck. I would not be surprised if that doesn't show up soon because I'm confident that all of us are running into it.
18:01 Brad But as a simple example of this, I literally am.
18:05 Brad currently in the midst of burning some tokens on trying to solve this obscure like safari bug where it's like safari's use of audio and video is is is bad it's it's quite bad and um so there's some bug i'm trying to solve that has to do with you know default mac default audio device in safari person unplugged their airpods and it you
18:31 Brad goes crazy right and i've been working on this for a couple days which which effectively means i'm burning a you know millions of tokens on it and um the agent keeps coming back to solutions that i have already tried and it failed and it keeps and it keeps circling back right um and so this is a good example where i i fundamentally feel that if you're a developer who's not
18:56 Brad who would struggle at this themselves then you're going to be completely perplexed and you're going to be completely stymied by the agent running off and getting stuck like if you don't know how to unstuck yourself right then you're not going to know how to unstuck the agent right um
19:13 Scott Yeah, yeah.
19:13 Brad uh so
19:16 Brad The basic answer is yes,
19:18 Brad you can certainly include in your prompts.
19:20 Brad Hey,
19:21 Brad come back to me.
19:22 Brad Another thing that you can do, depending on the tools that you're using, like all of the tools nowadays are kind of showing their thought process.
19:30 Brad So if you're paying attention and you're watching the agent work,
19:32 Brad if you're sitting back and watching the agent work,
19:34 Brad you can see it thinking about what it's going to say next or try next,
19:40 Brad and they all have a stop button.
19:42 Brad button you know and if you're watching and it's basically you're going to start spinning off to do the same thing you got to hit that stop button and say and it's exactly what you would do with a junior engineer right you're checking in at stand-up and you're like hey you haven't made any progress on the ticket talk to me what's going on how are you what are you trying today you know and if they're basically
19:59 Brad telling you something that they talked about yesterday you're like that didn't work yesterday let's let's take this out of stand-up and let's go do a one-on-one real quick and we're going to try to figure out why you're stuck on this
20:10 Scott okay
20:10 Brad yes
20:11 Scott so to just recap and reset here a little bit if we're seeing k-shape for
20:17 Brad the yep
20:18 Scott upper branch decay i said breaking down tasks
20:22 Scott and paying attention to progress you sort of encapsulated that in the saying you sort of tech leader engineering manager skills what
20:29 Brad Yes.
20:30 Scott else do you think the people who are on the upper branch of the k are doing yeah
20:35 Brad Well,
20:35 Brad those two levers are pretty important,
20:37 Brad right?
20:37 Scott yeah
20:38 Brad They're pretty important.
20:39 Scott if all they're doing is that then they're probably pretty good
20:42 Brad Yeah,
20:42 Brad I do think that and we talked a little bit about this a couple episodes ago in the door report.
20:46 Brad I do think that there are very real.
20:49 Brad environmental attributes that make a difference.
20:53 Brad So, you know, kind of the do you have already good processes for code reviews?
21:00 Brad Do you already have good processes for change management,
21:02 Brad CICD?
21:03 Brad I personally believe that I'm a big strong believer.
21:07 Brad I'm a strong believer in linting, type checking.
21:14 Brad quality test harnesses right like the and so if you've got those things in place right then you can be a little bit more free to let the agents run and
21:25 Scott Mm-hmm.
21:27 Brad Because they can't do damage, right?
21:28 Brad Because you have other guardrails in place to make sure that they don't do damage.
21:33 Scott Mm-hmm.
21:34 Brad So I feel like that's another important point to make that, you know, how many times have you gone into an organization,
21:40 Brad right? You know, as a fractional CTO, you show up and you find out that like they don't have.
21:45 Brad They don't even have like high quality version control best practices or they don't have, you know, unit test best practices or they don't have, you know, linters or things like that.
21:55 Scott It's been a long time since I've seen no version control.
21:59 Brad Well,
22:00 Brad yeah,
22:00 Brad I mean.
22:03 Brad there's all kinds of different versions of weak version control right
22:06 Scott It's
22:06 Brad like
22:07 Scott true.
22:07 Brad your branching strategy can be you know can yeah i got it under version control but literally it's just like when we commit we commit it like put you know we don't we don't we don't know what we release we're we're saving it all in version control but we have no tagging to know what we've released or something like that
22:25 Scott Okay,
22:26 Scott here's another thought for the topic,
22:27 Scott okay?
22:28 Scott I think.
22:29 Scott A certain swashbuckler vibe.
22:33 Brad YOLO.
22:35 Scott I guess YOLO, maybe I'm not exactly sure.
22:37 Scott But there's a certain element where you have to not be afraid of failing.
22:44 Brad Agreed.
22:45 Scott Because I'm sure I would imagine for every 10 things you try to do with your junior intern robot.
22:55 Scott Some percentage of them, five of them, five of the 10 don't succeed very well, something like that. I don't
23:01 Brad throw
23:01 Scott know.
23:01 Brad them out oh
23:03 Scott Throw them out,
23:03 Scott of course. So it's a different attitude as an engineer.
23:07 Brad totally i think that again like so so so let me just play that back to you because i i think i agree with you very much that um if you have a strong competency in
23:22 Brad um evaluating correctness and evaluating
23:25 Scott Mm
23:25 Brad progress
23:25 Scott -hmm.
23:26 Brad and you combine that with a willingness to
23:34 Brad You know, the bots aren't going to complain that you gave them a bunch of tasks that you then just closed their PR without merging. You know, they're not going to get all sad because you didn't merge their PR,
23:46 Brad right?
23:46 Brad So if you're basically willing to give the bots tons of work to do and you have the ability to evaluate that.
23:52 Brad So like literally I just heard this interview with the CEO of Cursor, not just, but maybe it was a couple months ago, but he was basically talking about like.
24:00 Brad They realized that the job of the engineer has moved into it's all about code review,
24:04 Brad right?
24:05 Scott Mm
24:05 Brad So like,
24:05 Scott -hmm.
24:06 Brad when was the last time we looked at what a code review really looks like these days? And it's like, we all think of the GitHub, kind of the GitHub,
24:12 Brad green lines, red lines,
24:14 Brad you know, split view,
24:15 Brad unified view,
24:16 Brad but has have we have really stepped back and asked the question, what's the easiest way to evaluate and adjudicate quality code? And so they're doing some cool innovation in that space.
24:26 Brad case because they recognize that like how you know it's it's all about evaluation of evaluating the quality of the output right that's a huge piece of this um That's i
24:35 Scott CEO Krishnamay said that, but I feel like I said that first.
24:40 Brad mean it's not a it's it's i'm just gonna say it's it's a pretty straightforward idea
24:47 Scott You know, here's a question.
24:50 Scott It's a little bit of a change of topic,
24:53 Scott but it was already hard enough to be a good software engineer.
24:58 Scott Is this actually making it this, when I say this, I mean, is AI or AI-assisted coding actually making it harder?
25:06 Brad Man, I'm going to take the, I don't know, say the over or the under.
25:10 Brad I'm going to take the, look, I've already said this. I think it's going to be K-shaped. I think that for the people that are good at this job.
25:19 Brad they are just gonna crush it and
25:21 Scott You're really good.
25:23 Scott Yeah,
25:23 Brad the
25:23 Scott so you're
25:24 Brad people that are great and
25:25 Scott with me.
25:25 Brad the people that are weak are going to get crushed so uh here's a quick story like one of the teams i'm working with like like first time ever halfway through the sprint one of the engineers is like
25:38 Brad fuck this let's ship tomorrow like they're like let's start like we got so much shit done in the first half of the sprint let's let's let's like ship now you
25:48 Scott Yeah
25:48 Brad know and it's this sense it when you said swashbuckler that's kind of what i i'm hearing there like i've got another team that i work with where you know this one of the the team of one engineer is just like crushing it and he's like you know hey like like
26:03 Brad Like, I'm getting stuff done in an hour of what would have taken me a week before.
26:08 Scott Well, you know, it's interesting when I said squash buckler, there was an assumption in that statement that maybe I have wrong.
26:17 Brad Oh,
26:17 Scott My
26:17 Brad say it.
26:17 Scott assumption.
26:19 Scott So what I meant, my assumption was that engineers are choosing to use AI for some things, meaning use an agent to write some things and choosing to write it themselves for other things.
26:32 Scott And where the swashbucklerness came in there was saying,
26:35 Scott well,
26:35 Scott I'm going to go see if the agent can do this one
26:37 Brad No.
26:37 Scott instead of me.
26:38 Scott But you're saying 100% agent.
26:41 Brad 100% of the code is my top engineers.
26:44 Brad They are no longer writing code.
26:47 Brad 100% of the lines of code that are being checked in by my top engineers are not written by the developers.
26:54 Brad It is being managed and evaluated by the developers.
26:59 Scott Yeah
27:04 Brad I defy anyone to tell me otherwise.
27:06 Brad If you're still typing brackets and fours and ifs, you are getting left behind.
27:15 Scott So this raises an interesting question that came up this week.
27:22 Scott my company.
27:24 Scott I have just for some context,
27:26 Scott I have a weekly engineering roundtable that I host.
27:32 Scott And what that means is it started out as an office hours, the idea where people could come and ask me questions or whatever,
27:38 Scott but that didn't really ever work very well.
27:40 Scott So I mean, sometimes somebody would come and sometimes they'd have a question, but lots of times they just wanted to hear what other people wanted to have.
27:47 Scott that we're going to ask and so i changed it up a little bit and said okay we're just going to have this meeting and um anybody can bring a topic or i'm going to try to come with a topic or two every week and i'm just going to throw it out there and we'll just discuss there's no no we're not trying to accomplish anything we're just discussing and one of the topics that was discussed this week it's every tuesday it was discussed this week was in a world where
28:12 Scott The agents are writing most of the code.
28:14 Scott Does the quality of the code matter?
28:18 Scott like it doesn't matter if it's a big ball of mud if the agent knows how to change the ball of mud i mean the reason we don't like it to be a ball of mud i was more on the side of that it doesn't matter i think it does matter but i was more on the side that it doesn't matter because i was like look most of the reason we don't like spaghetti code is because it's difficult to change and difficult to understand but if the robot can understand it and can change it then what's the matter if
28:43 Brad Yeah.
28:43 Scott it's
28:44 Scott messy and and we don't like it because when we want to make big changes to it it's really difficult but if the robot can make big changes to
28:52 Brad Yeah.
28:52 Scott it like it's just still one prompt for us or whatever figuratively speaking and so
28:57 Scott It doesn't matter.
28:58 Scott This was intense debate,
29:00 Scott by the way, and I think maybe you're – I took that it doesn't matter.
29:03 Scott It sounds like you're taking that it matters.
29:05 Brad Yeah,
29:05 Brad I'm on the it matters.
29:06 Brad And I'm on the it matters for two reasons.
29:10 Brad The first simple reason is that as I just proclaimed that the top of the K are going to be all the people that are really good at evaluating quality and evaluating work product. And so it's still human beings that are accountable for the.
29:24 Brad For the code quality,
29:25 Brad I am confident that the answer is it's all about how quickly you can evaluate the content.
29:29 Brad So it does matter that it's be readable by you,
29:32 Brad the evaluator.
29:33 Brad Right.
29:34 Brad So that's reason number one. And reason number two, I think that in 2026, even I suspect between now and 2028,
29:45 Brad like,
29:46 Brad no, the robots aren't good at reading the big ball of mud.
29:50 Brad And they aren't good at the bigger projects. It's not there.
29:53 Brad Like all of our data is showing that we're starting to kind of.
29:57 Brad geek out right now like there's there's we've hit a plateau in sort of like frontier model performance um and um it's still quite good but it's kind of plateauing right now and i'm not seeing like a major breakthrough on the horizon so what that means is you've got something that's really quite good at writing things very quickly and now it all comes down to how quickly you can evaluate it so i'm going to say code quality matters
30:24 Scott Okay, so as I said a minute ago,
30:26 Scott just for the record, I'm not fully in team.
30:28 Scott It doesn't matter.
30:30 Scott But theoretically,
30:31 Brad That's your way
30:31 Scott I entertain the idea.
30:33 Brad of saying uncle?
30:34 Scott No, no, I'm just saying I want to be clear, but I do want to push harder.
30:38 Scott I do want to push harder against team. It does matter with this.
30:43 Scott Imagine,
30:43 Scott theoretically, you've got perfect and complete.
30:49 Scott test automation.
30:50 Scott Does it matter?
30:51 Brad If the code is quality,
30:53 Brad if the test automation
30:53 Scott Yeah.
30:54 Brad catches it.
30:55 Scott Yeah.
30:57 Brad Well,
30:57 Brad I mean,
30:58 Brad I've also asked the AI to go write tests for me.
31:03 Brad and i love that it can do that and it often does but guess what you better code review those tests i read i've had the ai right test that basically passed and tested the exact opposite thing that it was supposed to do it's like it's a little bit like i mean that but by the way you see that with with human engineers too right like oh i couldn't get the test to work so i changed the test when you look at it what you changed the test to actually be
31:29 Brad passing the fail case,
31:31 Brad not the...
31:32 Scott you're reminding me of this war story moment that happened a few years ago where i was looking at some code just about five years ago i was looking at some code that had been written long before i arrived this wasn't even at the company i met and i was prior company i was looking at this code that was there and i'm just i'm just staring at it and it's supposed to be doing one thing and i'm like
31:55 Scott I just think this is doing exactly the opposite of what it's supposed to be doing and I asked another person to come look at it and I was like could you just look at this like I just think this is 100% backwards and they were like yeah that's that's what's happening and I was like how how did this this code has been here for years and it's literally been like doing minus when it should be doing plus I mean like that was how simple it was right
32:22 Scott Right?
32:22 Scott And I was like, how did nobody notice this?
32:25 Scott And how did test pass? How did this whole thing get through?
32:30 Brad And what was the answer?
32:32 Scott Who knows?
32:33 Scott But I just remember staring at it.
32:34 Scott I just remember not being able
32:35 Brad There
32:35 Scott to believe
32:36 Brad were tests.
32:36 Scott my eyes.
32:36 Brad There were tests and it was passing the test.
32:39 Brad The tests were exactly wrong.
32:41 Scott Yeah, probably.
32:42 Scott I just remember staring at the code and just like not believing my own eyes, just being like.
32:47 Scott That can't be the way it is, right?
32:50 Brad I will
32:51 Scott Anyway.
32:52 Brad say, like, I do think.
32:55 Brad I you noticed that I said look you got to have you know good engineering practices that would be good for your humans as
33:01 Scott Mm
33:01 Brad well
33:02 Scott -hmm.
33:02 Brad and let the bots live in that space and I listed linting and type checking and and I will say that you know I uh Paul Brown uh one of the guests on our podcast recently uh he and I were talking about this months ago and he was kind of like
33:16 Brad Who cares about Lent? Why do you care about Lent? If the bot is able to write the code without Lent, then what do you care?
33:24 Brad And so I think it's fair that there's going to be a valid debate that's going to happen.
33:30 Brad I'm currently standing on the ground pretty firmly.
33:34 Brad I feel pretty confident.
33:35 Brad I guess I'm going to say this is a strong opinion,
33:38 Brad loosely held.
33:40 Brad And then if you can show me some data that I'm...
33:42 Brad Be happy to change my mind on this.
33:44 Brad But I think that it all comes down to evaluation of the code that's being written.
33:50 Brad And so we need to stack the deck in our favor as the human evaluators to
33:57 Scott Every
33:57 Brad evaluate.
33:57 Scott way we can.
33:58 Scott readability is one of those pieces all of it is all they're all parts of it
34:03 Brad By the way, my 2.9 billion tokens doesn't include all of the co-pilot code review requests that I click on.
34:09 Brad So now on, like all the all the teams I work on, it's like it's very clear that our practice is, you know, whoever writes the code.
34:19 Brad writes the code whether they use an agent to write or do they encrypt and decrypt the keyboard themselves the very first thing that happens is everybody clicks code review by co-pilot and then i can't count that like this you know engineers that i know that would have written the code that way written the code as best they could have
34:38 Brad the co-pilot comes back and gives some feedback and they're like oh that is actually a good thing let me consider that let me go change that code and so before we even have a human code review it it's like i won't even look at code that hasn't had co-pilot look at it now and then i will go look at the co-pilot comments and see like well were those valid things did the engineer did engineer make those changes and that gives me a sense of like that that's another piece of evaluating code quality
35:03 Brad Right. Like if the original implementation had a lot of sort of fundamental errors that were caught by the AI code review agent,
35:12 Brad I'm going to pay more close attention to the final product.
35:16 Scott I want to add one thing to this.
35:19 Scott We're talking about the ability to evaluate.
35:21 Scott We're talking about the intern,
35:23 Scott the eager intern and all these different practices.
35:26 Scott I'll throw one more little practice tactic into the sauce,
35:31 Scott small PRs.
35:33 Brad Yeah.
35:34 Scott I bet the agents are inclined.
35:36 Scott You can let the agent make large PRs, but we know I'm sure there's science behind this, but I just know it from.
35:46 Scott working with people if your pr is more than 300 lines it's hard for somebody to reason about what the heck's going on there when i when i see those prs that are 68 files and thousands of lines i'm like this is forget this right
36:02 Brad yeah totally totally well there's there's two things about that first of all let me say uh spiritually i'm 100 aligned with what you just said agree 100 um and i'm also a big fan of the 300
36:12 Brad uh lines uh benchmark as a reasonable thing i will say that i have adjusted my mindset around that where it's 300 lines of functional code and
36:22 Scott Mm
36:22 Brad if
36:22 Scott -hmm.
36:23 Brad you've got if that 300 lines of functional code includes you know 1500 lines of of new tests written uh in the pr then i then i can i still consider that to be passing but you're right that i just got done saying how important it is to code review the tests so
36:37 Scott Yeah.
36:37 Scott Yeah, team, it matters.
36:39 Scott You need to code review the tips.
36:41 Brad Yeah, that's that's a fair point.
36:43 Brad That's a fair point. But by the way, by the way, I was going to say with one last point about your
36:47 Scott Please
36:47 Brad shape PRs, another good friend of mine who's like he's not a software developer,
36:53 Brad but he's a, you know, you know, an engineer mindset and he's been working on a project, you know, two guys in a garage type thing.
37:00 Brad And his co-founder was like ranting to him like you can't give me these 3000 line PRs and he's like, I don't.
37:07 Brad I don't know why not and I was like tell me what was in that 3000 line PR and it's like oh yeah I you know kept asking the agent for more and more and more and more and more and I was like okay that was your that was your mistake like you can't ask your co-founder to code review something that has 12 concepts in it that's that's where that was your original sin so
37:28 Scott Brad, it is a crazy new world.
37:33 Brad Yep, yep
37:34 Scott It's tectonic.
37:36 Scott It is in my entire career has been one way.
37:40 Scott If I think about the major changes in my career,
37:44 Scott which I'm not going to collect every one of them up here, but no SQL databases were a huge change.
37:51 Scott Cloud was a huge change.
37:53 Scott Um,
37:54 Scott I,
37:54 Brad The internet.
37:55 Brad I
37:55 Scott well,
37:56 Scott the internet, yeah,
37:56 Scott that was, that kind of started on the internet.
37:58 Scott I didn't really do pre-internet software.
38:01 Scott This just feels bigger than all of them.
38:04 Scott It
38:05 Brad agree.
38:05 Scott just feels bigger.
38:07 Brad I agree.
38:08 Scott It's changing the craft.
38:11 Brad I agree.
38:13 Brad I agree.
38:14 Scott It's crazy. It's blowing my mind.
38:17 Brad Well, there you go.
38:18 Brad 2026,
38:19 Brad K-shaped economy for AI coding.
38:22 Scott All right.
38:23 Scott Well,
38:23 Scott smash,
38:24 Scott rate,
38:25 Scott review,
38:26 Scott subscribe,
38:26 Scott do all that stuff, my friends.
38:28 Brad Yep.
38:28 Scott And Brad,
38:29 Scott we'll see you here next week.
38:30 Brad See you here next week.
38:31 Scott All right. Bye, everyone.
38:32 Brad Bye.
38:34 Brad This episode of How Many CTOs Does It Take Podcast was edited by Saffron Epigow.
38:39 Brad Our theme song was AI generated by Brad.
38:42 Brad Please follow us on all the social media platforms at at how many CTOs pod you can find us on YouTube LinkedIn Instagram Facebook threads X tick-tock and patreon or at our website at how many CTOs pod.com and remember to rate review and subscribe on the podcast platform of your choice thanks for listening