The K-Shape of AI Development: Managing Digital Interns
In this episode of "How Many CTOs Does It Take?" podcast, hosts Scott Porad and Brad Hefta-Gaub yet again delve into the transformative impact of AI on software engineering. Brad shares his 2025 Cursor Year in Review, revealing his record 2.9 billion tokens used, prompting a discussion about the efficiency and cost of using AI tools. They discuss the emergence of a K-shaped economy in AI coding and what attributes lead to successful AI-assisted coding. They emphasize the importance of breaking down tasks, evaluating progress, and adopting a tech leader mindset. The hosts debate whether code quality still matters in a world dominated by AI-generated code and conclude with reflections on the rapid evolution of the software development landscape brought about by AI.
Resources:
- How Many CTOs Pod: https://howmanyctospod.com
- Scott Porad: https://www.linkedin.com/in/scottporad/
- Brad Hefta-Gaub: https://www.linkedin.com/in/bradheftagaub/
#TechPodcast #EngineeringPodcast #DevTalks #PodcastForDevs #HowManyCTOs #Podcast #CTOs #CTOPodcast #ChiefTechnologyOfficer #Technology #Engineering #SoftwareDevelopment #SoftwareEngineering #TechLeadership #EngineeringLeadership #EngineeringCulture #TechDebates #AI #Cursor #AIAssistedCoding #AIAgents #AIProgramming #AIFuture #AIRevolution
Brad: The people that are going to excel are the engineers who have a team leader,
tech leader mindset.
Scott: Welcome back, everybody.
We know you've been waiting for another episode of how many CTOs does it take?
And in this occasion,
this instance,
my name is Scott Porad, and this is my trusty co-host.
Brad: Brad Hefta-Gaub,
Scott: But the question today is not how many CTOs does it take? The question is how many tokens does it take?
Brad: how many tokens does it take?
Scott: Brad, we were just chatting before we got recording here and I said we should be recording this.
Brad: Yes, we should be.
Scott: You got your cursor wrapped.
Brad: My year
Scott: I
Brad: -end got review
Scott: a
Brad: is
Scott: Spotify
Brad: better.
Scott: wrapped.
You got a cursor wrapped.
My Spotify wrapped says my Spotify age is 32.
Did you get a Spotify I age?
Brad: didn't get it.
No.
I don't use Spotify.
My Spotify account is like the family account,
so it's like random stuff that my wife and kids listen to.
Scott: The sad thing about having your Spotify age be 32 when you're my age is at first you think, oh,
that's cool.
I'm listening to music 20 years younger than my age.
And then you think, wait a second, 32 isn't young?
Brad: You're a millennial. Your music tastes are millennial.
Oh, man,
that's
Scott: Well,
Brad: funny.
Scott: tell me about your cursor wrapped.
Brad: Well,
I,
you know,
I.
I was not necessarily proud of this. I was kind of like,
wow,
that doesn't that seems like a smaller number than I was expecting.
But then I shared it with some of the devs that I work with and they were all like, what the hell?
Anyway,
2.9 billion tokens in 2025.
That's what I personally used within Cursor alone.
2.9 billion
Scott: What
Brad: tokens.
Scott: is the cost of a token?
Brad: Ah.
That's a great question.
And I'm an early enough adopter of Cursor that I'm on the legacy plan.
I'm not going to call it the grandfathered plan.
I'm on the legacy plan.
And so I'm only paying 20 bucks a month.
And like everyone else has to pay per token,
but I'm like grandfathered in or excuse me, legacyed in.
Scott: I wonder what the actual cost of a token is.
Brad: Yeah.
Yeah.
Scott: Probably $10.
Brad: No.
Scott: I'm just kidding.
I'm just kidding. But these companies are spending so much money.
I read today that Microsoft spent $88 billion on AI last year.
Brad: Well,
that sounds about right.
But then that means my 2.9 billion tokens are not nearly that much. They spent a lot more than that.
Scott: Yeah.
Brad: That's a lot.
Scott: Well, that's cool.
Brad: So, yeah,
so I think there's a little, it's funny because I feel like even some of my top developers,
so like one of the top developers I work with after I posted my $2.9 billion,
he was like, he responded saying,
oh no,
I need to up my level because he'd only had 700 million tokens.
So he was a little bit like,
wow,
like, you know, what's the map on that?
Scott: I
Brad: Four times more
Scott: mean, this is an interesting thing because there's a lot of different ways one can.
To interpret token usage.
It's kind of like clocks.
Brad: uh yeah maybe maybe it's a little bit like clocks i mean i would tend to agree that like as as you know you and i love to talk about engineering efficiency and and you know every time we talk about what's the right metric i always love to say it's clocks of course right because it's the worst measurement ever so i wouldn't be so i think it's a legitimate to say like uh what did you do with those tokens brad right like were there you know
It's not a bad case to make that throwing around token numbers is a little bit like throwing around lines of code.
So I think that's a fair point.
I would say that like, you know, and we kind of hinted at this in our last episode of 2025,
like, where do we think, what do I, what my opinion about where I think AI coding is going in 2026 and what, you know, I don't want to turn this into, it's not a predictions, you know, I don't want to say a predictions episode or anything like that, but I think that we're going to see a dramatic.
um you know uh a k-shaped not a k-shaped economy but like a k-shaped reaction to ai coding tools right the the halves are going to grow faster and the have nots are going to fall further behind faster um and i think it's going to come down to that that central question of what you're you're scratching at which is it doesn't matter how many tokens you use it matters how
effectively used the tokens that you use.
Scott: I agree.
And it might actually be an effective jumping off point for something that might actually be useful and practical to the 39 people who listen to this show, which is what are the things that the effective people are doing or are going to do that?
will make them continue to be effective or more effective.
And I have one thing in my list. I'm curious to know,
you are much more of the power user here than I am. So I'm curious to know what you think those things are.
Brad: You want to share your item that you think is the leading indicator of this?
Scott: I see.
two well actually i said one thing but i actually think it's two things the first is engineers are going to figure continue to figure out how to take large tasks and break them into smaller tasks that the agents can do one by one and i just i will come back to this like 50 times in the next year there was just this thing that kent beck it didn't just happen he put it on
Probably LinkedIn is where I saw it, where he said, you know, 90% of my skills are becoming obsolete, but the remaining 10% are 10,000 times more valuable.
Brad: Yep.
Scott: And knowing how to break the work down to describe each step to the agent is one of those skills,
which jibes with something I've said.
Many times in this podcast,
I'm sure,
which is the advice I always give to engineers about developing their career is get to the point where you can write an implementation plan or text back or whatever you want to call it so that other people can understand your design and start collaborating with you on it is to me the most important skill to develop because then you're at a crossroads in your career. You can decide if you want to be a manager or if you want to stick with.
being an IC,
but until you can actually take requirements and translate them into work for someone to do,
that's where you've got to get to.
So that fits with one of the things I'm seeing is being able to break that work down.
The other thing I'm seeing is AI is accelerating things.
It's not making it free.
Brad: Here
Scott: And what do I mean by that? I'll give you two examples.
A few examples. I'll start with one. I've probably talked about this in the podcast before because it came from earlier this year,
an engineer who updated a lot of code, but it saved him tons of time.
But he said it took him like four or five hours to get the prompt right, you
Brad: Yeah, yeah
Scott: know, of experimentation. And then it saved him probably weeks of time.
We had another case that came through recently.
an engineer they had about a little less than 2 000 tests and these tests were flaky and low quality for a variety of reasons and i mean he spent he spent an afternoon it was four or five hours he spent an afternoon setting up a process a set of steps for the agent to go through first by getting some third-party criteria for what
what a good test looked like then evaluating each test and giving it a letter grade then asking the robot to go through and I've been doing a bunch of steps and ultimately having it refactor some of the things at the bottom or at the not at the bottom at the end again multi-step process learning how to take these big tasks and it took them a bunch of time and then an engineer I just talked to recently this I thought this was really interesting he created
created an entire GitHub repo which consisted of mostly MD files.
Brad: Mm-hmm.
Scott: Just context.
Brad: Yep, yep.
This is a huge,
by the way, you know, you talked about the notion of like what our best practice is.
And so for sure, great documentation in an MD file in your directory trees,
that is gold for the agents. They need that.
Give them more of that.
Scott: They needed this was not even really in the directory.
free of the code it was separate he was trying to get you he was he is trying to get i'm just looking for here i had i looked at it earlier today he was trying to get um
The robot to containerize something that's currently running on bare metal,
like
Brad: Uh-huh.
Scott: an older application.
And he set up a number of, and can we just tell you what these files are? He's got AI agent and Quickster Yep. MD implementation plan summary MD,
Brad: Yep.
Scott: containerization implementation plan MD,
Brad: Yep.
Scott: implementation plan checklist,
containerization goals MD,
migration guide MD.
The previous containerization attempts MD,
which because lessons learned from previous attempts,
Yep. relevant confluence pages MD.
Brad: Yep.
Scott: kids have other repositories like
Brad: Yep.
Scott: and then you know just all just pages pages just describing all this context and what to do he's been working on this for like three or four days
Brad: Yeah,
yeah, totally,
totally.
So here's my, I'm going to just going to come right out and say the magic words that matter for all of this. And you know, I already said K-shape, K-shaped economy when it comes to dealing with AI coding agents.
The haves are going to excel.
The have-nots are going to fail faster and faster.
And it all comes down to not only what you said about the concept of being able to really break down work,
but I believe it's all about being a tech lead or an engineering manager.
The people that are going to excel are the engineers who have a team leader,
tech leader mindset.
Scott: Mm
Brad: But
Scott: hmm.
Brad: instead of leading other developers,
they are leading the robots.
Scott: Mm And hmm.
Brad: so that's the money shot.
That's the thing that matters the most.
And so I will say I think the reason that I'm going to make the case that my 2.9 billion tokens are incredibly effective tokens because I was operating at a, you know.
tech lead engineering manager architect level with with the robots for those 2.9 billion tokens so imagine that that was effectively you know 300 junior engineers that I had working for me and I managed them like junior engineers
Scott: Yeah, one of the people on our team said you have to think about the agents as really eager interns.
Brad: Yeah. And so what does that mean?
So let's kind of go to the next level,
right?
So one,
it means being really good at describing the problem space,
right?
Like you said,
and I agree with you that like being able to break down the problem,
explain the problem,
capture the statement of the problem,
that's what your engineer that's created all these MD files for this containerization project,
that's what they've done.
Right.
Now, that's only the start, though,
because if I just do that, but I am not good at evaluating progress and output.
Right.
Scott: Mm-hmm.
Brad: The other thing that a tech lead or an engineering manager does is they have an understanding of what.
of what right-sized progress looks like and what right-sized output looks like,
Scott: Mm
Brad: right?
Scott: -hmm.
Brad: So they can evaluate the technical output to say this is correct or this is not correct.
But more importantly,
or equally importantly,
they're able to evaluate the progress against the goal as it's happening and course correct.
Right. So what's the classic story of the really good engineering managers you've worked with in your career?
They're the ones that get.
compelling results out of all of their engineers.
And they're never coming to you saying like,
oh yeah,
this engineer,
you know, please get them off my team.
They suck. They can't do their job, right?
You don't ever hear that from the really good engineering managers,
right?
They're in there figuring out how to make even the worst performers perform well.
And the reason they're able to do that is because they know what
They know not only what good results look like, but they know what it looks like when the engineer is struggling and
Scott: Mm-hmm.
Brad: the AI bots struggle and they continue to struggle and they're going to continue to struggle.
They're getting better, but they struggle.
And so.
This is why I think there's going to be a case-shaped reaction to more and more use of AI agents, and this is why measuring the number of tokens is a dangerous thing.
Scott: Mm
Brad: You'll
Scott: -hmm.
Brad: burn through a bunch of tokens in a world where you let the bots go off the rails,
you're not good at evaluating their progress.
They churn out.
Slop, right?
We all hear
Scott: Mm
Brad: the term AI
Scott: -hmm.
Brad: slop,
right?
And so whenever I see somebody on LinkedIn or on Twitter basically saying, oh,
AI coding sucks,
it just generates slop.
My response to that is,
I bet that person would be a horrible engineering manager.
Like they only know how to evaluate their own work.
They don't know how to evaluate other engineers' work.
Scott: Yeah, well,
there's two parts there.
It's not just evaluating the work. I think the words I want to use to say evaluate the process. And what I mean by that is like, as an engineering manager,
if I had an engineer.
Who, you know, went for a couple of days of the easiest way to describe this would be if their tickets stayed in the same column on the board for a couple of days.
It's like a sign. Actually,
not even a couple of days, a day or two, you know, two stand ups in a row and like that ticket's still there.
It's like, OK, that person's stuff. Something's wrong.
Brad: Yep.
Scott: Actually,
I don't know what's wrong.
Something's wrong. Right.
Brad: Yep.
Scott: And so it's not whether the work product that person was creating was good or not, but they're actual like they weren't making any progress.
Brad: Yeah, exactly.
And I think that that what I'm saying is that is a 100 percent happening with all the AI coding agents. And if you let them get over their over their skis, then and you don't and you don't course correct quickly,
then they're burning tokens,
they're wasting their time, they're wasting time,
they're burning tokens and they're wasting your.
in your time and they're increasing the likelihood that they're going to produce slop yep
Scott: Okay,
I have a question for you, something I have never tried or never even thought to have tried until now.
If you have an engineer who's struggling with something,
you say to them, you sit down with them, you talk through the problem, you can point in the right direction,
you say work on this, you say let's check in after lunch,
right?
Let's see how it's going.
Can I say to an agent,
Like pick a time frame, 15 minutes,
Brad: Yeah.
Scott: 30 minutes.
Hey,
do this thing and report back to me. If you're not done in 30 minutes, just stop and
Brad: Yeah.
Scott: let's discuss.
Brad: Yeah.
So my quick answer to that is the tooling has been changing and evolving in such a way that the.
I would argue that the development tool set companies,
Cursor being a good example of this,
they have recognized this challenge that you and I have talked about, which is that the agents can get off the rails if you give them too big of a task.
And so they've been changing their tooling to do to essentially add those checkpoints.
And you might recall like and if you go back into our old episodes, you might remember that I've said things like early on, like, you know.
Nine months ago,
one of my the first prompt that I would always give,
I would say, do not write any code until you've told me your plan. Like I had explicitly say that to the agents and now cursor has plan mode. So you use plan mode instead.
And so that's one of the examples of the guardrails that the tooling companies have put in to effectively give you that functionality that you're talking about.
Like, hey,
give me your thought,
give me your implementation plan,
check in with me before you move on.
It's not necessary.
I've not yet seen Cursor implement anything where the agent stops when it gets stuck and says, hey,
I'm stuck.
stuck. I would not be surprised if that doesn't show up soon because I'm confident that all of us are running into it.
But as a simple example of this, I literally am.
currently in the midst of burning some tokens on trying to solve this obscure like safari bug where it's like safari's use of audio and video is is is bad it's it's quite bad and um so there's some bug i'm trying to solve that has to do with you know default mac default audio device in safari person unplugged their airpods and it you
goes crazy right and i've been working on this for a couple days which which effectively means i'm burning a you know millions of tokens on it and um the agent keeps coming back to solutions that i have already tried and it failed and it keeps and it keeps circling back right um and so this is a good example where i i fundamentally feel that if you're a developer who's not
who would struggle at this themselves then you're going to be completely perplexed and you're going to be completely stymied by the agent running off and getting stuck like if you don't know how to unstuck yourself right then you're not going to know how to unstuck the agent right um
Scott: Yeah, yeah.
Brad: uh so
The basic answer is yes,
you can certainly include in your prompts.
Hey,
come back to me.
Another thing that you can do, depending on the tools that you're using, like all of the tools nowadays are kind of showing their thought process.
So if you're paying attention and you're watching the agent work,
if you're sitting back and watching the agent work,
you can see it thinking about what it's going to say next or try next,
and they all have a stop button.
button you know and if you're watching and it's basically you're going to start spinning off to do the same thing you got to hit that stop button and say and it's exactly what you would do with a junior engineer right you're checking in at stand-up and you're like hey you haven't made any progress on the ticket talk to me what's going on how are you what are you trying today you know and if they're basically
telling you something that they talked about yesterday you're like that didn't work yesterday let's let's take this out of stand-up and let's go do a one-on-one real quick and we're going to try to figure out why you're stuck on this
Scott: okay
Brad: yes
Scott: so to just recap and reset here a little bit if we're seeing k-shape for
Brad: the yep
Scott: upper branch decay i said breaking down tasks
and paying attention to progress you sort of encapsulated that in the saying you sort of tech leader engineering manager skills what
Brad: Yes.
Scott: else do you think the people who are on the upper branch of the k are doing yeah
Brad: Well,
those two levers are pretty important,
right?
Scott: yeah
Brad: They're pretty important.
Scott: if all they're doing is that then they're probably pretty good
Brad: Yeah,
I do think that and we talked a little bit about this a couple episodes ago in the door report.
I do think that there are very real.
environmental attributes that make a difference.
So, you know, kind of the do you have already good processes for code reviews?
Do you already have good processes for change management,
CICD?
I personally believe that I'm a big strong believer.
I'm a strong believer in linting, type checking.
quality test harnesses right like the and so if you've got those things in place right then you can be a little bit more free to let the agents run and
Scott: Mm-hmm.
Brad: Because they can't do damage, right?
Because you have other guardrails in place to make sure that they don't do damage.
Scott: Mm-hmm.
Brad: So I feel like that's another important point to make that, you know, how many times have you gone into an organization,
right? You know, as a fractional CTO, you show up and you find out that like they don't have.
They don't even have like high quality version control best practices or they don't have, you know, unit test best practices or they don't have, you know, linters or things like that.
Scott: It's been a long time since I've seen no version control.
Brad: Well,
yeah,
I mean.
there's all kinds of different versions of weak version control right
Scott: It's
Brad: like
Scott: true.
Brad: your branching strategy can be you know can yeah i got it under version control but literally it's just like when we commit we commit it like put you know we don't we don't we don't know what we release we're we're saving it all in version control but we have no tagging to know what we've released or something like that
Scott: Okay,
here's another thought for the topic,
okay?
I think.
A certain swashbuckler vibe.
Brad: YOLO.
Scott: I guess YOLO, maybe I'm not exactly sure.
But there's a certain element where you have to not be afraid of failing.
Brad: Agreed.
Scott: Because I'm sure I would imagine for every 10 things you try to do with your junior intern robot.
Some percentage of them, five of them, five of the 10 don't succeed very well, something like that. I don't
Brad: throw
Scott: know.
Brad: them out oh
Scott: Throw them out,
of course. So it's a different attitude as an engineer.
Brad: totally i think that again like so so so let me just play that back to you because i i think i agree with you very much that um if you have a strong competency in
um evaluating correctness and evaluating
Scott: Mm
Brad: progress
Scott: -hmm.
Brad: and you combine that with a willingness to
You know, the bots aren't going to complain that you gave them a bunch of tasks that you then just closed their PR without merging. You know, they're not going to get all sad because you didn't merge their PR,
right?
So if you're basically willing to give the bots tons of work to do and you have the ability to evaluate that.
So like literally I just heard this interview with the CEO of Cursor, not just, but maybe it was a couple months ago, but he was basically talking about like.
They realized that the job of the engineer has moved into it's all about code review,
right?
Scott: Mm
Brad: So like,
Scott: -hmm.
Brad: when was the last time we looked at what a code review really looks like these days? And it's like, we all think of the GitHub, kind of the GitHub,
green lines, red lines,
you know, split view,
unified view,
but has have we have really stepped back and asked the question, what's the easiest way to evaluate and adjudicate quality code? And so they're doing some cool innovation in that space.
case because they recognize that like how you know it's it's all about evaluation of evaluating the quality of the output right that's a huge piece of this um That's i
Scott: CEO Krishnamay said that, but I feel like I said that first.
Brad: mean it's not a it's it's i'm just gonna say it's it's a pretty straightforward idea
Scott: You know, here's a question.
It's a little bit of a change of topic,
but it was already hard enough to be a good software engineer.
Is this actually making it this, when I say this, I mean, is AI or AI-assisted coding actually making it harder?
Brad: Man, I'm going to take the, I don't know, say the over or the under.
I'm going to take the, look, I've already said this. I think it's going to be K-shaped. I think that for the people that are good at this job.
they are just gonna crush it and
Scott: You're really good.
Yeah,
Brad: the
Scott: so you're
Brad: people that are great and
Scott: with me.
Brad: the people that are weak are going to get crushed so uh here's a quick story like one of the teams i'm working with like like first time ever halfway through the sprint one of the engineers is like
fuck this let's ship tomorrow like they're like let's start like we got so much shit done in the first half of the sprint let's let's let's like ship now you
Scott: Yeah
Brad: know and it's this sense it when you said swashbuckler that's kind of what i i'm hearing there like i've got another team that i work with where you know this one of the the team of one engineer is just like crushing it and he's like you know hey like like
Like, I'm getting stuff done in an hour of what would have taken me a week before.
Scott: Well, you know, it's interesting when I said squash buckler, there was an assumption in that statement that maybe I have wrong.
Brad: Oh,
Scott: My
Brad: say it.
Scott: assumption.
So what I meant, my assumption was that engineers are choosing to use AI for some things, meaning use an agent to write some things and choosing to write it themselves for other things.
And where the swashbucklerness came in there was saying,
well,
I'm going to go see if the agent can do this one
Brad: No.
Scott: instead of me.
But you're saying 100% agent.
Brad: 100% of the code is my top engineers.
They are no longer writing code.
100% of the lines of code that are being checked in by my top engineers are not written by the developers.
It is being managed and evaluated by the developers.
Scott: Yeah
Brad: I defy anyone to tell me otherwise.
If you're still typing brackets and fours and ifs, you are getting left behind.
Scott: So this raises an interesting question that came up this week.
my company.
I have just for some context,
I have a weekly engineering roundtable that I host.
And what that means is it started out as an office hours, the idea where people could come and ask me questions or whatever,
but that didn't really ever work very well.
So I mean, sometimes somebody would come and sometimes they'd have a question, but lots of times they just wanted to hear what other people wanted to have.
that we're going to ask and so i changed it up a little bit and said okay we're just going to have this meeting and um anybody can bring a topic or i'm going to try to come with a topic or two every week and i'm just going to throw it out there and we'll just discuss there's no no we're not trying to accomplish anything we're just discussing and one of the topics that was discussed this week it's every tuesday it was discussed this week was in a world where
The agents are writing most of the code.
Does the quality of the code matter?
like it doesn't matter if it's a big ball of mud if the agent knows how to change the ball of mud i mean the reason we don't like it to be a ball of mud i was more on the side of that it doesn't matter i think it does matter but i was more on the side that it doesn't matter because i was like look most of the reason we don't like spaghetti code is because it's difficult to change and difficult to understand but if the robot can understand it and can change it then what's the matter if
Brad: Yeah.
Scott: it's
messy and and we don't like it because when we want to make big changes to it it's really difficult but if the robot can make big changes to
Brad: Yeah.
Scott: it like it's just still one prompt for us or whatever figuratively speaking and so
It doesn't matter.
This was intense debate,
by the way, and I think maybe you're – I took that it doesn't matter.
It sounds like you're taking that it matters.
Brad: Yeah,
I'm on the it matters.
And I'm on the it matters for two reasons.
The first simple reason is that as I just proclaimed that the top of the K are going to be all the people that are really good at evaluating quality and evaluating work product. And so it's still human beings that are accountable for the.
For the code quality,
I am confident that the answer is it's all about how quickly you can evaluate the content.
So it does matter that it's be readable by you,
the evaluator.
Right.
So that's reason number one. And reason number two, I think that in 2026, even I suspect between now and 2028,
like,
no, the robots aren't good at reading the big ball of mud.
And they aren't good at the bigger projects. It's not there.
Like all of our data is showing that we're starting to kind of.
geek out right now like there's there's we've hit a plateau in sort of like frontier model performance um and um it's still quite good but it's kind of plateauing right now and i'm not seeing like a major breakthrough on the horizon so what that means is you've got something that's really quite good at writing things very quickly and now it all comes down to how quickly you can evaluate it so i'm going to say code quality matters
Scott: Okay, so as I said a minute ago,
just for the record, I'm not fully in team.
It doesn't matter.
But theoretically,
Brad: That's your way
Scott: I entertain the idea.
Brad: of saying uncle?
Scott: No, no, I'm just saying I want to be clear, but I do want to push harder.
I do want to push harder against team. It does matter with this.
Imagine,
theoretically, you've got perfect and complete.
test automation.
Does it matter?
Brad: If the code is quality,
if the test automation
Scott: Yeah.
Brad: catches it.
Scott: Yeah.
Brad: Well,
I mean,
I've also asked the AI to go write tests for me.
and i love that it can do that and it often does but guess what you better code review those tests i read i've had the ai right test that basically passed and tested the exact opposite thing that it was supposed to do it's like it's a little bit like i mean that but by the way you see that with with human engineers too right like oh i couldn't get the test to work so i changed the test when you look at it what you changed the test to actually be
passing the fail case,
not the...
Scott: you're reminding me of this war story moment that happened a few years ago where i was looking at some code just about five years ago i was looking at some code that had been written long before i arrived this wasn't even at the company i met and i was prior company i was looking at this code that was there and i'm just i'm just staring at it and it's supposed to be doing one thing and i'm like
I just think this is doing exactly the opposite of what it's supposed to be doing and I asked another person to come look at it and I was like could you just look at this like I just think this is 100% backwards and they were like yeah that's that's what's happening and I was like how how did this this code has been here for years and it's literally been like doing minus when it should be doing plus I mean like that was how simple it was right
Right?
And I was like, how did nobody notice this?
And how did test pass? How did this whole thing get through?
Brad: And what was the answer?
Scott: Who knows?
But I just remember staring at it.
I just remember not being able
Brad: There
Scott: to believe
Brad: were tests.
Scott: my eyes.
Brad: There were tests and it was passing the test.
The tests were exactly wrong.
Scott: Yeah, probably.
I just remember staring at the code and just like not believing my own eyes, just being like.
That can't be the way it is, right?
Brad: I will
Scott: Anyway.
Brad: say, like, I do think.
I you noticed that I said look you got to have you know good engineering practices that would be good for your humans as
Scott: Mm
Brad: well
Scott: -hmm.
Brad: and let the bots live in that space and I listed linting and type checking and and I will say that you know I uh Paul Brown uh one of the guests on our podcast recently uh he and I were talking about this months ago and he was kind of like
Who cares about Lent? Why do you care about Lent? If the bot is able to write the code without Lent, then what do you care?
And so I think it's fair that there's going to be a valid debate that's going to happen.
I'm currently standing on the ground pretty firmly.
I feel pretty confident.
I guess I'm going to say this is a strong opinion,
loosely held.
And then if you can show me some data that I'm...
Be happy to change my mind on this.
But I think that it all comes down to evaluation of the code that's being written.
And so we need to stack the deck in our favor as the human evaluators to
Scott: Every
Brad: evaluate.
Scott: way we can.
readability is one of those pieces all of it is all they're all parts of it
Brad: By the way, my 2.9 billion tokens doesn't include all of the co-pilot code review requests that I click on.
So now on, like all the all the teams I work on, it's like it's very clear that our practice is, you know, whoever writes the code.
writes the code whether they use an agent to write or do they encrypt and decrypt the keyboard themselves the very first thing that happens is everybody clicks code review by co-pilot and then i can't count that like this you know engineers that i know that would have written the code that way written the code as best they could have
the co-pilot comes back and gives some feedback and they're like oh that is actually a good thing let me consider that let me go change that code and so before we even have a human code review it it's like i won't even look at code that hasn't had co-pilot look at it now and then i will go look at the co-pilot comments and see like well were those valid things did the engineer did engineer make those changes and that gives me a sense of like that that's another piece of evaluating code quality
Right. Like if the original implementation had a lot of sort of fundamental errors that were caught by the AI code review agent,
I'm going to pay more close attention to the final product.
Scott: I want to add one thing to this.
We're talking about the ability to evaluate.
We're talking about the intern,
the eager intern and all these different practices.
I'll throw one more little practice tactic into the sauce,
small PRs.
Brad: Yeah.
Scott: I bet the agents are inclined.
You can let the agent make large PRs, but we know I'm sure there's science behind this, but I just know it from.
working with people if your pr is more than 300 lines it's hard for somebody to reason about what the heck's going on there when i when i see those prs that are 68 files and thousands of lines i'm like this is forget this right
Brad: yeah totally totally well there's there's two things about that first of all let me say uh spiritually i'm 100 aligned with what you just said agree 100 um and i'm also a big fan of the 300
uh lines uh benchmark as a reasonable thing i will say that i have adjusted my mindset around that where it's 300 lines of functional code and
Scott: Mm
Brad: if
Scott: -hmm.
Brad: you've got if that 300 lines of functional code includes you know 1500 lines of of new tests written uh in the pr then i then i can i still consider that to be passing but you're right that i just got done saying how important it is to code review the tests so
Scott: Yeah.
Yeah, team, it matters.
You need to code review the tips.
Brad: Yeah, that's that's a fair point.
That's a fair point. But by the way, by the way, I was going to say with one last point about your
Scott: Please
Brad: shape PRs, another good friend of mine who's like he's not a software developer,
but he's a, you know, you know, an engineer mindset and he's been working on a project, you know, two guys in a garage type thing.
And his co-founder was like ranting to him like you can't give me these 3000 line PRs and he's like, I don't.
I don't know why not and I was like tell me what was in that 3000 line PR and it's like oh yeah I you know kept asking the agent for more and more and more and more and more and I was like okay that was your that was your mistake like you can't ask your co-founder to code review something that has 12 concepts in it that's that's where that was your original sin so
Scott: Brad, it is a crazy new world.
Brad: Yep, yep
Scott: It's tectonic.
It is in my entire career has been one way.
If I think about the major changes in my career,
which I'm not going to collect every one of them up here, but no SQL databases were a huge change.
Cloud was a huge change.
Um,
I,
Brad: The internet.
I
Scott: well,
the internet, yeah,
that was, that kind of started on the internet.
I didn't really do pre-internet software.
This just feels bigger than all of them.
It
Brad: agree.
Scott: just feels bigger.
Brad: I agree.
Scott: It's changing the craft.
Brad: I agree.
I agree.
Scott: It's crazy. It's blowing my mind.
Brad: Well, there you go.
2026,
K-shaped economy for AI coding.
Scott: All right.
Well,
smash,
rate,
review,
subscribe,
do all that stuff, my friends.
Brad: Yep.
Scott: And Brad,
we'll see you here next week.
Brad: See you here next week.
Scott: All right. Bye, everyone.
Brad: Bye.
This episode of How Many CTOs Does It Take Podcast was edited by Saffron Epigow.
Our theme song was AI generated by Brad.
Please follow us on all the social media platforms at at how many CTOs pod you can find us on YouTube LinkedIn Instagram Facebook threads X tick-tock and patreon or at our website at how many CTOs pod.com and remember to rate review and subscribe on the podcast platform of your choice thanks for listening