Managing Skepticism: Balancing Tradition and Technology with AI

0:00 Scott: the skeptics are ending up like the frogs boiling in water they just don't even know that they're getting caught caught up in it it's like all of a sudden it's happening to them and it's like oh i'm using this stuff and it's not scary and it's actually helping me

0:27 Brad: Welcome everybody to another episode of the How Many CTOs Does It Take podcast.

0:31 My name is Brad Hefta-Gaub and my trusty co-host is I'm

0:34 Scott: Scott Porad.

0:36 Brad,

0:36 how are you doing today?

0:37 Brad: doing pretty good. I'm doing pretty good. Life's a little chaotic right now. All kinds of stuff going on,

0:42 but I'm in my happy place when I'm in my office and I'm thinking about code and I'm thinking about startups and I'm thinking about how AI can make everything go faster. And so

0:53 There you go.

0:53 That's where I'm in my happy space right now.

0:55 Scott: I want to talk about your happy space for a second because

0:58 Brad: Yeah.

0:58 Scott: as a CTO,

1:00 as a chief technology officer, often you have many people to manage and people are frustrating at times.

1:08 I've heard certain

1:09 Brad: Hold on a second. I

1:10 Scott: people

1:10 Brad: resemble that remark.

1:13 Scott: And I often use the line like there was a reason I chose to work with computers.

1:20 Brad: actually you know it's funny that you say that uh you know we did this uh almost now a month ago we did that episode with uh uncle bob and

1:27 Scott: Mm-hmm.

1:28 Brad: you just said something that was so funny because it reminded me of a comment that somebody left somebody left this comment on one of those clips that went viral of of uncle bob talking about like knowing the machine and being intimate with the machine down in the assembly language and this somebody responded something to the effect of

1:46 I am a software engineer,

1:49 and clips like this make me realize I've never really been a computer guy.

1:56 Scott: interesting

1:56 Brad: and i was like yes like bob is definitely like the computer guy's computer guy like when he pulled out his little three-bit um computer i was just like oh my god oh

2:07 Scott: so bob was a big get for us should we tease the next big get that we have

2:11 Brad: oh well i mean if you'd like to yeah go for it yeah do

2:15 Scott: i'd like to i'd

2:16 Brad: it

2:16 Scott: like to tell our audience that sometime coming this

2:19 This fall,

2:19 we have Kent Beck coming on the Oh, podcast.

2:22 Brad: man,

2:22 he's your idol.

2:24 Scott: I don't know if he's my idol,

2:25 but often I like what he has to say.

2:29 And we haven't scheduled to record,

2:31 so we don't know when it'll come out.

2:32 But I'm super excited to talk to him because like Uncle Bob,

2:35 he is an old timer that has jumped into the AI deep end.

2:39 Brad: Yeah, totally.

2:40 And he's seen a lot and now he's seeing like this whole new world is exciting.

2:49 Yeah,

2:49 Scott: there's

2:49 Brad: it's pretty cool.

2:50 Scott: there's something he had this tweet from a while ago in fact we should ask him about this when it comes when he comes on the pod in fact did we call it a tweet anymore do you call it

2:59 Brad: I

2:59 Scott: he

2:59 Brad: don't

2:59 Scott: had know. a kid he had an x what

3:01 Brad: I

3:01 Scott: do we think say

3:01 Brad: they still call them tweets.

3:03 Yeah,

3:03 I don't know.

3:05 Scott: he had this tweet which really resonated with me he said

3:09 90% AI is making 90% of what I know useless and the other 10% is 10,000 times more valuable

3:17 Brad: Yeah, yeah, totally.

3:19 Scott: and that totally fits with so many of the things that we've talked about about how engineers are becoming engineering managers about taste about just all sorts of different things but it's the functional knowing where to put the semicolons that are no longer that valuable

3:34 And it's all that other stuff that's so useful.

3:38 Brad: Yeah, it's super interesting.

3:39 One of the teams that I work with a lot,

3:43 what their pattern has become,

3:45 which is so interesting,

3:47 is all of the human engineers are always like,

3:51 let's say they're working on a problem together. Let's say they're writing code, they're doing features, or then they kind of get to the phase where they're doing a release and they've got like.

3:59 they've done a UAT and a problem happens or something like that,

4:03 or they get a customer complaint.

4:05 So they're dealing with something where they've like sent their agents off to go read the telemetry and figure out the bug or something like that.

4:14 The thing that I find really interesting is they talk to each other about their teams of agents.

4:23 And so they're always saying, well,

4:24 my agents are looking at your PR,

4:27 right?

4:28 And or they'll say like somebody wrote some new feature and they'll be like,

4:33 hey,

4:34 everybody send your agents at my PR,

4:36 send your adversary, let's do an adversarial attack on all of the agents coming against my PR,

4:41 you know, and it's they've gotten into this sort of like this cultural norm.

4:49 of all of them very much demanding and expecting each of them to bring themselves as a senior, they're all senior,

5:01 pretty senior engineers,

5:02 bring themselves as senior engineers along with their cohort of adversarial agents that

5:10 Scott: Yeah,

5:10 Brad: they bring

5:10 Scott: from your team,

5:10 Brad: their team.

5:11 Scott: bring your team.

5:12 Brad: And so it's like they're all, each of them has a team and they all bring their team to come.

5:17 collaborate with all of each other.

5:19 Scott: here's a question that just came into my head we're kind of we kind of program these agents right i mean we give them an md file and to tell them who they are and what to do and all this so on and so forth and i would imagine that some people are better at constructing these agents than others right

5:36 Brad: Maybe.

5:38 Say more.

5:39 I'm not going to agree or disagree yet.

5:41 Scott: Well,

5:41 Brad: Say more.

5:41 Scott: let's just be very rudimentary about it.

5:44 I might write an agent which says you are a software design architecture agent of value this software.

5:51 And you might. So one line. What was that? Maybe 80 characters I just spoke.

5:56 You might write an agent that's like.

5:58 10 pages with all sorts of rules and experience and intuition and really clear do this don't do that here's some nuance all sorts of stuff might be in there right so you're now it's a good question about whose agent would be better by the way

6:11 But let's just assume that your agent with 10 pages of instructions and my agent with one line of instructions,

6:16 let's assume yours is better.

6:18 Brad: Okay

6:18 Scott: Yours is not only built on years and years of your own personal experience,

6:22 the craft,

6:23 it's going to be built on years and years of working with the models and figuring out how to refine the instructions to the agents that it's most effective.

6:30 And so the question is like,

6:31 as you said, oh, everyone's expected to bring.

6:34 their own i mean there's a bunch of questions but i was like as you said everyone's expected to bring their own agents their own agent team right my first thought in my head was like oh some people might have better teams than others then my then as i started talking about it's like might employees start to get evaluated on the quality of their team

6:53 Brad: On their team of agents,

6:55 exactly.

6:55 Scott: of their team of agents yeah like hey brad i'm sorry man

6:59 Your team of agents is just not as good as Scott's over here.

7:02 Brad: Yeah.

7:02 Scott: Like his agents are just better than yours.

7:06 Brad: Well, let me first of all, I love the thread that you're pulling on. There's something to it.

7:11 But I'm going to maybe take a slightly different I'm going to come at it a slightly different angle.

7:17 I think it's less about.

7:20 how you set up your agents,

7:22 although there's a piece of that.

7:23 And I think it's more about how you direct your agents.

7:29 Scott: Hmm.

7:30 Interesting.

7:30 Brad: So what I mean by that is, now, let me just start by saying that everything I'm about to say does not apply to a dark factory.

7:41 Scott: Okay.

7:41 Brad: This is like right before the dark factory.

7:45 So if you're in the sort of like, I don't know if anybody's coined it yet, but I know a bunch of people have kind of talked about like the layers of AI pilledness,

7:55 right?

7:56 And so I want to talk about the teams that I'm working with are generally right before dark factory.

8:04 They're like AI pilled up to the point that they're not yet fully.

8:08 fully dark factory right and

8:09 Scott: Okay.

8:10 Brad: so yes they have their agents that they've described yes they um are very proficient with which the various harnesses that they've chosen to use and actually interestingly enough a lot of them are using different harnesses even they're like a heterogeneous harness world so like some are using cursor some are using um cloud code some are using codex um some are using open code and open router and stuff like that

8:33 that but the but the way that the way that they're using them is very much giving the agents a task so they've done something to give the agent your markdown files your skills etc they've done that but then they give a task to the agent and the agent is producing a partial result and then they're giving more feedback to the agent on that partial result and and they're and they're still human in the in the loop right

8:59 Scott: Mm

9:00 Brad: And so my current hypothesis is that at least what I'm seeing with my teams is how they set up their agents has less of an impact on the effect of the agent team than how they steer their agents.

9:18 Scott: -hmm.

9:19 Brad: Right. So even one of my strongest engineers,

9:22 he was talking a lot about how,

9:23 oh, yeah, I've set up this whole I've set up all my skills and I've got this this rag that has all of my experience as an engineer. I've given it every I've given it all the code I've written before AI existed.

9:33 I've I've given it all I give it all my my my comments on other people's pull requests and I've and it sucked all that into it's got like my brain.

9:41 And he was really excited about this like five months ago.

9:44 And then recently he's just like.

9:46 just like yeah i stopped using all that like the models have gotten good enough now that i just tell the models what i want them to do and i tell them how to report back to me and then i read their report back and i give them guidance that way and so he's definitely pivoted away from the i set it up with this perfect state to begin with to now he's he's definitely more in the

10:11 I'm trusting it more even without the setup and then I'm validating its results afterwards.

10:15 Does that make sense what I'm saying?

10:17 Scott: It makes a lot of sense.

10:18 It makes me think of a bunch of different things.

10:21 The first is, and you brought this to the table many times,

10:25 it's how you would treat a junior engineer.

10:29 Brad: Yeah.

10:29 Scott: Do this little bit,

10:30 come back and show it to me and then I'll have you do the next little bit.

10:33 It reminds me of the notion of breaking down.

10:36 big tickets into smaller tickets big tasks into smaller tasks but don't ask the agent to build me a mobile app to

10:43 Brad: Right.

10:43 Scott: build one little don't ask an engineer to build a mobile app ask him to build a bunch of little pieces at a time yeah that actually all really resonates it's it's also interesting because it puts more burden okay so coming back to my point it takes the burden off your team your agents and actually onto all of a sudden the ringleader of the circus again is the really important person hey we've all got the same agents but

11:03 But I know how to, I don't know what the right word is,

11:06 man,

11:07 Brad: Manage them.

11:07 Scott: manage them.

11:09 I know how to manage them.

11:10 Brad: Yeah, how to manage them.

11:12 Scott: It's kind of like, you know.

11:15 people who are listening couldn't see me waving my hand but it reminds me of a jockey

11:19 Brad: Yeah.

11:20 Riding on horse race.

11:21 Scott: we

11:21 Brad: Yeah.

11:22 Scott: can all ride this horse but somehow i know how to get the most out of this horse yes

11:25 Brad: Yeah.

11:26 Another sort of analogy that came to my mind was like, it's not about the race car as much as it is about the driver.

11:34 Scott: it's that you know there was a hot minute

11:37 during the pandemic for those of you who remember the pandemic that i got really into f1

11:43 Brad: Yeah.

11:43 Scott: because that show was on tv i don't know if you remember it's still on tv i think but it was that show did just start this show started like right before the pandemic

11:51 Brad: Yeah.

11:51 Scott: i mean the cars are so similar that it so much comes down to the driver and you've even got a team with two of the exact same car yet one of the drivers always tends to be better

12:02 Brad: Exactly.

12:03 Scott: Yeah.

12:04 Brad: So much comes down to the driver.

12:06 So but to answer to then rewind and answer your question,

12:09 I absolutely think that one of the ways that we should be evaluating engineers in today and at least for the near future is how good are you at driving the F1 car,

12:22 which is current day AI agents and.

12:27 harnesses. So how good are you at that? How good are you at managing those agents?

12:31 Scott: Let me posit that we already know how to do that.

12:34 I mean, it's the same way that we.

12:37 manage engineers now i'm

12:39 Brad: Yeah.

12:39 Scott: not i'm going to say some things that i'm sure people might disagree with but in general like we we look at velocity we look at how many bugs did you ship and how many problems you got into production and we look at your response time to issue well i guess that's more of an operational thing but like we we have we look at how much you know value and impact we look at the whatever we have a bunch of ways of looking at software you produce and whether you're producing it yourself or producing an army of agents we kind of already have

13:04 already have those tools yeah

13:05 Brad: I agree.

13:06 I totally agree.

13:07 I totally agree.

13:08 And I think that like,

13:09 so,

13:10 so I think maybe the next layer of the onion to peel back or just sort of the thing that comes to my mind when I think about this is what are we doing for those engineers that don't feel like they need the agents to continue to perform? In other words,

13:29 I.

13:31 I don't work with that many people. I don't work with anybody that has this perspective,

13:35 but I know there are people out there with the perspective that using the AI agents does not add value for me because I am already good enough or I'm already or they aren't good enough to do what I can do.

13:51 I was talking to an engineer that had that perspective recently and I was.

13:55 pretty surprised and they were i would say they're a mid mid-level engineer not not super i'm not going to call them junior but they're not they're not super senior they've been you know i mean maybe they they might actually be senior based on like tenure but i'd say they're they've been at it for five or six years maybe yeah

14:11 Scott: Okay, so I can share with you some things that we have going on at TaskRabbit, but I want to ask a question first, and it's not a rhetorical question,

14:19 and it doesn't apply to AI,

14:20 it actually applies to everything.

14:22 How do you deal with people who think they're right when they're actually wrong?

14:30 Because I actually find this to be a little bit of a challenge in my life,

14:33 you know, work life or personal life. It's like somebody says, well, I think this.

14:39 It's like, hey, well, you can think that because you want,

14:42 but it's just it's not right.

14:44 Brad: Well,

14:45 I mean, obviously,

14:46 it completely depends on the context and the person and the, you know, but

14:51 Scott: OK,

14:52 Brad: I think

14:52 Scott: so somebody

14:52 Brad: that in the.

14:52 Scott: comes up to you and says an AI will never be as good at coding as me.

14:57 Brad: And does this person work for me?

14:59 Scott: Doesn't matter.

15:00 They're just wrong.

15:02 Brad: I know they're wrong, but my stakes in the conversation are different if it's a random person on the street.

15:07 Scott: Fair enough. Fair enough. I was asking just in general, I was looking for some life coaching from you.

15:12 Brad: I think actually in the conversation I had recently about this, I literally said to the person like,

15:18 hey.

15:21 And you're wrong.

15:23 You're wrong.

15:24 And I hope that you interrogate this position you have and think about it because if you want to continue to be a software engineer,

15:33 then...

15:36 And you're going to need, if this is a career you want,

15:39 you're going to need to adopt this.

15:41 Scott: Yeah, that is an intellectual argument that I, you know, rational argument that I've used with people, which is like, look, it's just another tool in our toolkit.

15:50 And the people who figure out how to use it, and there are people who've been figuring out how to use it. I've seen it with my own eyes.

15:56 They're going to advance further in their career than you.

15:59 And you might be okay with that.

16:01 But they're going to outpace you and you're going to get left behind and you might get left behind to the point where you can't be employed and that could be a problem.

16:08 On a more humorous level, it reminds me of that old Louis C.K.

16:12 bit about atheism and believing in God.

16:16 Do you know what I'm talking about?

16:17 Brad: I don't know this one.

16:18 I don't know this

16:19 Scott: one. Well, he's talking,

16:21 I can't, look,

16:22 I'm not Louis C.K. I can't convey it exactly like he did.

16:25 But he talked about being an atheist and not believing in God,

16:29 but then like,

16:29 you know, when you get the,

16:32 if you die and there is a God,

16:34 maybe you should have just done it. Maybe there's no consequence to believing and then finding out there's none,

16:40 but there might be a consequence to not believing and then dying and finding out there is.

16:46 I can't convey it exactly.

16:48 Maybe we'll put a link in the show notes.

16:50 But like, it's just

16:51 Brad: i'm

16:51 Scott: like,

16:51 Brad: pretty sure

16:52 Scott: he's

16:52 Brad: that i'm pretty sure he was canceled anyway so oh

16:54 Scott: back.

16:55 He's uncanceled.

16:56 Brad: goodness all

16:57 Scott: I

16:57 Brad: right

16:57 Scott: mean,

16:58 Brad: there

16:58 Scott: I don't want to get into the politics of canceled and uncanceled.

17:00 But I just know that he had like a new show out recently.

17:04 Brad: you

17:04 Scott: So but I'm not even a big Louis C.K. fan.

17:07 I just the point I'm trying to make is you don't have to you don't have to think AI is going to be better.

17:13 But you know what? If you're wrong,

17:15 you might be up a creek.

17:16 So

17:17 Brad: Yeah.

17:17 Scott: go spend time on it.

17:19 Brad: Yeah.

17:20 And it's interesting.

17:20 Like, I mean,

17:21 the thing that the thing that I will add to this is that I've written a lot of code in my life and I love writing code and I've loved building stuff and I love building stuff and I've never in,

17:35 you know.

17:37 They're deaf and a lot of the code I wrote I mean, you know, I was I was writing like

17:43 you know c++ professionally building software that's like compiled and put onto you know cd-roms and shipped you know like that's like you know and i've worked on software that was you know did a billion transactions a day like this is hard stuff and i wrote and wrote code for all that kind of stuff right and i certainly have a sense of pride of ownership and a sense of like

18:09 Pride of the skills that allowed me to write that kind of code,

18:12 right?

18:13 Scott: Mm-hmm.

18:14 Brad: But I never felt strongly about where the semicolon was and where the open bracket was.

18:22 Like that was never something that...

18:25 like when people would argue about coding standards i was always just like what the you know like whatever let's

18:31 Scott: Mm

18:32 Brad: just pick one i don't care like it's all i can read it in your way i can read it either way like let's just get this done you know and so i don't quite understand i have heard some people say that they got the joy that they really got out of it was the

18:51 crafting of the code that did the thing and

18:55 Scott: hmm yeah

18:57 Brad: i only ever saw that as a means to the end the end being the building of the thing the

19:04 Scott: yeah or the doing when you say the building of the thing do you mean

19:07 Brad: do like the thing having the thing built having the thing

19:09 Scott: yeah having the thing built

19:10 Brad: having the thing come to life and being able to use it like that was what that was what i got my real that's what i get my joy from

19:17 Scott: Yeah, yeah.

19:19 Yes,

19:19 100% agree.

19:20 I get my joy from a similar aspect,

19:22 which is figuring out how to put it together.

19:25 Brad: Yeah.

19:26 Scott: There was something that was very exciting to me on our team that some engineers are working on.

19:30 They said, we're going to do this. And in my mind, they're kind of putting some

19:34 puzzle pieces together that was going to open up an opportunity and I was like oh wow that I was like I was super excited about I started going telling people about it and they were like well what can we do with that I was like you don't see the possibility with this thing and you know there's a certain kind of thing that I really like this is a this is a little bit of a tangent you know how in Seattle we have these um well now it's they're gone remember um okay now you pay for parking on the street

19:59 See,

19:59 Brad: Mm

20:00 Scott: you

20:00 Brad: -hmm.

20:00 Scott: got an app and you put in a code and you put in your license plate.

20:03 That's how you pay for the parking.

20:04 Brad: Yep.

20:04 Scott: But you remember,

20:05 there used to be this thing which was, you still had to put your, it was a device on the street and you put your credit card in and it printed out a sticker and you put it in the window.

20:14 Brad: Yep.

20:14 Scott: And I loved that thing because it used a credit card reader,

20:18 a cell phone,

20:19 a cellular.

20:20 A little to show you a thing to power it and

20:23 Brad: A

20:23 Scott: a little

20:23 Brad: little

20:23 Scott: basic

20:23 Brad: printer?

20:24 Scott: printer.

20:24 Somebody took some very basic things or

20:27 Brad: Yeah.

20:27 Scott: not very basic, but like, you know, some commoditized things.

20:31 and put them together and then have this whole other thing which was a really cool thing and that kind of stuff always like I love that stuff and like to me that's what so much what software is about like we just put these parts together and like look what it can do

20:45 um and I love that and that's that makes my eyes look that kind of thing I love figuring that out

20:51 Brad: Right.

20:51 And so then in today's world where we have the AI that can write the code faster for us, that just means we have that many more.

20:57 more pieces that we can plug together correctly to build the bigger thing and we can build these pieces faster and build and build more complex things more easily and so that's to me i'm just like why like this is like the golden age this is like the golden age of being a software developer

21:14 Scott: a little bit i mean i would describe it more it's like a more like of a land rush like the the playing field has been leveled and everybody has it

21:24 I don't know. I'm going to say this in an extreme way.

21:26 The playing field has been leveled and everybody has a chance now, right?

21:30 Brad: Yeah.

21:30 Scott: Because the models just,

21:32 it gives you so much power.

21:35 Brad: Yeah, bring your ideas.

21:37 Bring

21:37 Scott: You

21:37 Brad: them.

21:37 Scott: want to hear about, we're not exactly addressing skeptics at TaskRabbit or can I tell you about what we're doing to kind of just try and level everybody up?

21:45 And this is going to sound a little corporate-y, but I actually think that's good, especially for smaller companies to hear.

21:50 Okay, so first thing is we found an AI maturity matrix.

21:56 So, I don't know what company or consulting firm produced it.

22:02 It's got five levels,

22:03 one through five.

22:04 It's got like six different competencies.

22:06 So you can imagine a matrix.

22:08 Each level, it says,

22:09 you know, it's got like automation documentation. I can't remember what the different skills are, the different competencies are,

22:17 and then where you're at in these different levels.

22:19 And then so first thing we did is we did an evaluation.

22:21 and we just asked ourselves where are we at in each of these things and you know some of them we were level one some we were level two some of them were level four right but five is the best five is like dark factory right that's the first thing we did the second thing we did is we just okay as a goal in the next year we want to get our whole organization to level three

22:40 But that's, that's where our goal is.

22:42 And so then that allowed us to just do a gap analysis that's very business speaking, but it just asked us to say, like, well, what, why aren't we at level three now? What do we have to do? Right.

22:51 Brad: Right.

22:51 Scott: And that,

22:53 then that allowed us to sort of spawn like a series of tasks or projects or whatever that we can say, well, we need to do this, we need to do this, and so on and so forth in order to get there.

23:01 And then this, I think, is a really key piece.

23:04 There's a few approaches.

23:06 we could have taken to getting there.

23:09 Well,

23:09 let me pause before going to this part because I think there's an important thing I forgot to mention.

23:14 For this whole effort,

23:15 I've been using a catchphrase and I'm going to even detour off this detour for a second.

23:21 As a CTO,

23:22 I think this is important for the CTOs out there.

23:24 Catchphrases are valuable.

23:25 Brand names are valuable.

23:27 Brad: Yeah, I'm pointing it, yeah.

23:28 Scott: They capture,

23:30 you know, people use buzzwords in an organization. They use three-letter acronyms and acronyms and those kinds of things.

23:35 And having a word or a phrase that people can use as a...

23:42 What's the word I'm looking for? A token or a totem or something.

23:46 They can sort of hang on to,

23:49 Brad: Yeah.

23:49 Scott: yeah,

23:49 to reflect a whole bunch of stuff is valuable, especially as a leader as you're trying to sell an idea into the organization. So the phrase I've been using is, hey, we are trying to shift from organic adoption,

24:02 which is kind of what we were doing. Everybody was trying to adopt on their own and we had some people who were.

24:07 doing awesome stuff and maybe some people are skeptics we're trying to shift from organic adoption to systemic usage whereas it's it's part of our system and everybody there's a floor or a foundation a floor i wanted to where everybody's at now some people can go further ahead if they're interested but this is the bottom line for organization and

24:29 And what we're trying to do is be very intentional about picking pieces that cut across.

24:37 everybody so that the whole organization gets lifted up in little steps as opposed to saying hey we're going to go jump from here all the way out to this faraway place and some people do it and some people don't do it we're just trying to move the floor up bit by bit by bit so the easiest thing which everybody did like a year ago the easiest piece that moves everybody up is we're going to introduce a code review agent so that now everybody's getting that's the easiest piece right and so what are the next thing well we're now we're introducing us

25:04 you know we're introducing a security review agent and an architecture review agent so these little pieces that just apply to everything apply to everybody some of them are going to be pretty more like you know in the build pipeline so they're post commit some of them are earlier like before you start developing it can look at your spec if the security agent and the architecture agent are going to kind of look at your technical design doc and say well we don't think you're doing this the right way so

25:29 We're kind of trying to just lift the whole organization up that

25:32 Brad: Yep

25:32 Scott: way.

25:33 And so in some ways, I mean, this wasn't the intention,

25:38 but like the.

25:39 the skeptics are ending up like the frogs boiling in water they just don't even know that they're getting caught up

25:44 Brad: Right,

25:45 Scott: in it it's like all of a sudden it's happening to them and it's like oh i'm using this stuff and it's not scary and it's actually helping me and so that's kind of the the high level approach that we're taking to get there and just doing it bit by bit and adding layers and layers and layers and we also have said to our team we want to have this baseline

26:05 And we're treating that baseline like version software,

26:08 because you probably heard me make the statement,

26:11 it's a good thing software isn't changing fast right now, otherwise we couldn't keep up. So stuff's changing so fast people can't keep up right now.

26:18 And so we're like, we're going to every six weeks.

26:22 We're only going to make changes to this system, whatever our system is, once every six weeks,

26:26 right?

26:27 And there can be people out there trailblazing, coming back with new ideas,

26:31 but then we'll just sort of adopt,

26:33 we'll just kind of inch it up, we'll ratchet it up every six weeks so that you can sort of keep up because I think part of the problem that people have.

26:41 this isn't a skeptic's problem this is just a practical problem which is like oh man i'm just trying to get my job done and literally every day there's like a new thing that i gotta go i could literally

26:50 Brad: Yeah,

26:50 Scott: spend eight

26:50 Brad: yeah, yeah.

26:51 Scott: hours a day just researching the new things

26:53 Brad: Yeah.

26:53 Scott: so don't tell me you're just telling me all this new stuff i gotta do all the time i can't even i'm just gonna go do what i know how to do which is write code and make people happy right so we're kind of trying to take this version approach like every six weeks

27:06 Now we're introducing this now,

27:07 you know and kind of step it up a bit at a time.

27:09 Brad: I love it.

27:10 Okay,

27:10 there's so many things that you said that I want to like, there's a few threads I want to pull on because it's super interesting what you just said.

27:16 So I want to start with, and I, you know, I'm a believer.

27:19 So I'm going to try to, I'm going to try to play.

27:23 a little bit of the skeptic and just, I'm curious just how you're, how you handle the, the, this sort of perspective from a type of skeptic. So a part of what I hear you saying, and you even teed this up is this is going to sound kind of corporate,

27:38 right?

27:39 Scott: Yeah.

27:39 Brad: And so what,

27:41 how do you, how do you respond to the

27:45 a challenge of, I just feel like you're trying to make me, why are you trying to even make me use this AI stuff?

27:52 It just feels like, it just feels like you're, you're pushing this as some kind of corporate agenda.

27:57 And like, what, what's, why do you care?

28:02 Like, what's like,

28:02 what, this just feels like a corporate agenda.

28:05 Scott: Great question,

28:06 Brad.

28:06 I'm glad you brought this up.

28:08 And I would answer that in a few ways.

28:11 The first is whether we like it or not,

28:14 you and I right now are working in a commercial organization,

28:17 right?

28:19 And commercial organizations, but we're in a for-profit system. Now, even if we were in a not-for-profit system, this might apply,

28:27 but like for-profit systems want to lower costs.

28:30 That's what they want to do, right?

28:31 And they want to.

28:33 They want the iron triangle.

28:34 They want a lower cost and increased quality and get stuff done faster.

28:39 Like they want all that stuff at the same time.

28:42 And, you know, we can discuss whether that's right or wrong, but it's just kind of the reality of the ecosystem we live in, which is that how should I say it? It's not being forced upon us.

28:52 It's almost like it's being,

28:54 it's just new laws of gravity that we have to deal with because.

28:58 We live in a system that's a for-profit system.

29:00 So that's one thing I would say.

29:02 The second thing I would say to you, Brad,

29:04 is I was a skeptic.

29:06 If you go back on this podcast and listen a year and a half ago,

29:10 I wasn't like a naysayer, this will be nothing.

29:12 But people I work with and people who listen to this podcast will hear me say, well,

29:15 it's a new tool in our toolkit. Let's see what it does.

29:18 You know, maybe it'll help. Maybe I've seen a lot of new tools in my career. Maybe it'll be great and maybe it'll be like blockchain.

29:25 Brad: Yeah.

29:25 Scott: And so it

29:26 Brad: Blockchain!

29:26 Scott: took me, you know, six or nine months of.

29:33 looking and trying to look with open eyes and watching what people were doing and watching you and taking now getting out of our little fake conversation but watching you brad and experimenting with myself where i was like finally like wow this is real

29:47 This is really, this allows us to make better software faster.

29:52 So I just think it's going to become the reality of our environment.

29:55 And like I said earlier in this call,

29:57 people are going to figure out that tool just like people who drove buggies figured out how to drive automobiles.

30:03 And it's just going to be like in five, in 10 years, I don't think anybody's, this is going to be a silly conversation in 10 years.

30:08 Brad: Yeah.

30:09 Scott: You know, it's like you still bleed people when they're having illnesses or put leeches on them.

30:12 like what the hell right so

30:14 Brad: Right.

30:14 Scott: those are so those are some some ways i think about it and then the other way i think about it is i mean you don't have to do it right you know but your i guess your peers are going to pace you by and that'll become evident after a while and then there's another little piece i talked to an engineer recently brad uh this is a this is an experienced engineer 20 plus probably years in the business about say 20 ish

30:39 Probably in his mid-40s.

30:41 um so he's kind of like he's a little fearful he was a little fearful right he's kind of just trying to he's a big company he's like i think he's at oracle i think because where he's at he's a little fearful about like getting laid off and he's kind of just trying to hold on and he's thinking i mean if i could just like hang on to this job for like another decade like i'll probably be okay um i don't think that's the best like mindset to come from you

31:04 Brad: Yeah.

31:05 Scott: got to come from a place of like curiosity

31:10 because the people who I see doing the best with it are the best engineers,

31:13 the ones who are the most curious.

31:15 Brad: Yeah.

31:15 Scott: So I don't know.

31:16 I think I lost my way in your answer.

31:19 So my answer lost my way in terms of your question.

31:22 But

31:22 Brad: Well,

31:23 Scott: I think

31:23 Brad: I think

31:23 Scott: those are

31:23 Brad: you answered it.

31:24 Scott: some of the ways I get at it.

31:26 And look,

31:27 we've had engineers on our team who've been like, well, I still don't believe you, right?

31:30 And they've seen a little bit, they've seen the writing on the wall,

31:33 and they've moved on mostly.

31:35 Brad: Yeah.

31:36 Okay.

31:37 So I think that's, I mean, I'll play that back to be that you as a leader are saying,

31:43 look,

31:44 I've seen it.

31:47 I've actually seen it. And look, here's your peer here.

31:50 Here's your principal engineer.

31:52 Look at what they're doing.

31:53 Like, you know this principal engineer,

31:55 you know that they.

31:57 They value code. They value getting stuff done.

32:00 If they're believers,

32:01 then isn't that some evidence for you that this is viable?

32:05 Scott: And they're telling me they're having more fun than they've had in a decade.

32:10 Brad: Exactly.

32:10 Scott: The grind of software engineering seems to have gone away suddenly all of a sudden.

32:15 Brad: Yeah.

32:16 Right.

32:16 Okay.

32:16 So you have these people that you trust.

32:18 really are trustworthy engineers that are the type of engineers that any engineer would look up to saying like this works i'm getting better i'm having more fun i'm enjoying this more we're getting more stuff done look at us eating down the backlog let it look at us look at us solving the you know the um all the technical debt that we you know we we would never take on the technical debt now we could get this so like there's real like you're just saying look there's real tangible evidence that this is meant

32:45 This is making a difference. And

32:46 Scott: Yeah,

32:47 Brad: just look at this evidence and then follow up with the that is the reason we're asking you to do this.

32:55 It's

32:56 Scott: yeah

32:56 Brad: not about a metric.

32:58 It's the metrics that matter are the metrics that matter for any, you know, capital, you know, company and capitalism. Right. Which is, you know, more features,

33:11 higher quality.

33:13 uh lower cost like that that's that's the iron triangle that we've all learned about from day one of being a software engineer and if we can if we move the needle to get all three of those going in a positive direction that's what that's what we all want

33:28 Scott: Yeah,

33:28 there's one other thing we want, or at least I want.

33:31 I have always, as a manager and a leader,

33:34 my mindset is...

33:37 I want, I come at it from my heart, which is like, I want you to be the best software engineer you can be.

33:46 Brad: Oh,

33:46 Scott: I want to create

33:46 Brad: really?

33:46 Scott: an environment where you can do the best work you can do.

33:48 I want you, that's what I want because I know you will enjoy your work more.

33:54 I know the company will get a better outcome.

33:55 I know everybody will have more fun doing it that way.

33:58 And I know that.

34:01 you're probably not going to work here forever you might but you're going to people tend to move around and I just I enjoy building really excellent teams because I want people I say this all the time I want you to be able to go get your next job because this company is on your resume because

34:16 Brad: Yeah.

34:16 Scott: somebody said like oh you were on that software engineering team they were really good and

34:21 Brad: Yeah.

34:21 Scott: so to me like

34:25 when I say like to people like this is what's coming this is what I'm seeing and this is what the talent engineers are doing it's not like I don't come at it from a punitive place like you're going to get fired if you don't do this I come at it from a place of like you do want to work I think you want to work for the rest of your career right maybe you

34:40 Brad: Yeah,

34:41 Scott: want to I

34:41 Brad: yeah

34:41 Scott: know and look I know another software engineer who was in his 40s and gave it up went to go be a preschool teacher said look I don't want to learn a whole new thing I've had a great career doing this I made some money

34:50 I mean, I can go on and do something else, right?

34:53 That's an option.

34:54 But I just think helping people,

34:56 creating the environment for people to do the best work and be the best people that they can be is a win-win for everybody and makes me feel good because it's a win-win.

35:05 I feel good too.

35:06 And I just feel like you get a lot more out of, you get a lot more just results out of people that approach than just, you know, the beatings will continue until morale improves.

35:14 I

35:14 Brad: Yeah,

35:15 Scott: don't think that's

35:15 Brad: absolutely.

35:15 Scott: the way to do it.

35:17 Brad: Yeah.

35:18 Well,

35:19 I think those are the, I mean, you gave the answers that I would give. Those are the best answers I've heard.

35:23 And I'm just curious if you, you know, kind of.

35:28 You had a different approach.

35:29 Scott: you know it's interesting i hear answers you know but i hear answers like we heard from uncle bob like people don't trust it right the code isn't good enough and he and i was like well bob didn't think c was good enough for assembly right

35:42 Brad: Yeah,

35:42 Scott: it's like

35:42 Brad: exactly.

35:42 Scott: and and i think one of the things you also have to say to people is well it's it's the first pitch of the first inning practically with this stuff

35:50 Brad: Yeah.

35:50 Scott: you know it's not going to be right all the time um and

35:54 we're still learning how it works and it's still getting like you mentioned to me before we got on the call fable 5.1 came out right it's

36:00 Brad: Yeah.

36:01 Scott: like the it's all getting better every day and we're all learning how to make it work better

36:06 Yeah, sometimes it's going to suck.

36:08 Brad: I also think isn't there like there's a classic like logical fallacy where like you compare something against perfect and it's like that's not that's not what we should be comparing this to we should be comparing the AI coding agents to humans writing similar code and is that perfect.

36:28 Does that Right. make mistakes?

36:29 No,

36:30 it's not perfect.

36:31 Yes,

36:31 humans make mistakes all the

36:33 Scott: Yeah.

36:33 Brad: time, right?

36:34 Scott: Yeah.

36:35 Brad: And again,

36:36 in my 40 years of writing code and reading and writing millions and millions of lines of code,

36:45 I can tell you.

36:46 The vast majority of the code that I've walked into and said,

36:50 oh, my God,

36:51 who whose neck do I need to ring about this code?

36:54 This is awful.

36:56 It was not written by agents.

36:57 It was written by humans,

36:59 Scott: Right,

36:59 Brad: you

36:59 Scott: right, right, right, that's

37:01 Brad: know, so so I just don't buy the whole like the reality is that for years I would you know, this is a common thing is a new engineer comes in.

37:11 And they complain about the code base.

37:13 It's almost universally the case that all engineers dislike code they didn't write.

37:18 Scott: It's true.

37:19 Brad: Like, right?

37:21 So maybe that's the only thing we're really talking about here.

37:24 Scott: Yeah, part of it. Here's another thing, though. I want to flip the positive thing.

37:28 You've talked about this, actually.

37:30 I'm just stealing your line.

37:32 There are best practices out there for being a really excellent software team.

37:36 We all follow some of them we don't follow.

37:38 Every organization has a variety up and down.

37:42 To really get the agents to work well for you,

37:45 you've

37:45 Brad: Put them in there.

37:46 Scott: got to do them.

37:47 Brad: Yeah, follow the processes.

37:49 Scott: Well, you have to.

37:50 Brad: You have

37:51 Scott: And

37:51 Brad: to.

37:51 Scott: to me, like I get I do get a sense of gratification from that because it just forces me to be good versus us to be good.

38:00 Right.

38:01 We have to be on our A game.

38:02 You can't be on your C game with an agent because it'll do C work to get an agent to do a work.

38:08 You've got to be on your A game.

38:10 Brad: Yeah, exactly.

38:12 Scott: To me, in addition to what I was saying earlier about I love putting pieces together,

38:17 that's a huge motivator like personal.

38:20 satisfaction i get out of just all the work i've done is like striving to be on the a game like that's to me

38:26 always been a big motivator i guess maybe there's an ego part in there which is i just i don't want to be second rate so like i'm not trying to toot my own horn but like that's always been a motivator to me to want to try and like get better better better sure that's graded on some people in the past who've just been like scott why can't good enough be good enough of you ever

38:44 Brad: There was one other thing that you said, and I don't know how much time we have left.

38:47 left here but the other thing you mentioned kind of caused a flight bulb to go off in my brain mind our friend kevin just it depends kevin just posted something i saw on linkedin that he's starting to see even early adopters who were really energetic about ai and adopted it early some of them are actually starting to fall behind because the pace of innovation is happening so fast

39:14 And that that can really that.

39:17 like really can then cause um you know a uh a morale hit i don't know that i have much to say about that other than i think that's really interesting too i almost wonder if if if the thing is that the tool if we think of it as a tool right the tool is only as good as the craftsman using the tool and i wonder i wonder if we've got a second wave of

39:42 of morale hits coming as the tools start to become better and better and it might further bifurcate like the people that can manage the more powerful tools from the people that can't.

39:59 Scott: There's some more powerful tools.

40:01 Brad: Let me ask you a question.

40:03 Do you think the pace of innovation is going to slow down with this stuff?

40:08 Do you think it's going to accelerate?

40:11 Scott: Well,

40:12 I think we're seeing acceleration in a lot of different vectors,

40:16 right?

40:16 So like the models themselves are getting better,

40:19 but then the harnesses are also getting better, right?

40:22 And so, and then the orchestrators on top of the harnesses are getting better.

40:27 So I just think that, I think just everything,

40:29 all the different tools are just moving faster and faster in a way that makes them more and more powerful.

40:35 So, I mean, there is that classic argument of the singular.

40:38 singularity right like they're the pace of innovation gets so fast you know gets so fast at some point that the humans can no longer keep up and and there are some people that claim that we're you know are we're we're on it we're seeing it and we just don't realize it you never know you're on the singularity until you're past it

40:55 Brad: Yes, it could speed up to the speed of light and then madness.

41:00 I just I haven't thought about this because previously.

41:05 My initial reaction is I actually kind of think we'll see it slow down.

41:09 Scott: Mm-hmm.

41:10 Brad: I really don't know if I'm right about this, but my gut kind of tells me that there's going to be some market forces that sort of create an equilibrium,

41:19 right?

41:19 Because the open the eyes and the anthropics of the world can't continue to invest this much money,

41:24 right?

41:26 It just doesn't.

41:28 I mean, they're like, they're like borrowing more than the entire economy kind of situation.

41:32 Scott: Right, right.

41:36 Brad: So it seems to me there's some market forces going on that are going to have them invest less.

41:43 and companies are going to say are we getting i mean i just heard this today i actually i shouldn't say just heard that i've been hearing this a lot lately of like are we getting the roi we're buying a million dollars of tokens are we getting the roi out of it you

41:54 Scott: Yeah,

41:54 Brad: know kind of thing

41:55 Scott: sure.

41:55 Brad: right and like i just feel like these market forces might come into play that might sort of not bring it to a screeching halt that's what i'm saying but get to a pace that's more sustainable economically perhaps

42:07 Scott: By the way, that sounds valid.

42:08 Like I hear you about that, and I think there's pretty strong cases to be made about that piece of it,

42:15 sort of the simple way to summarize it is that the technology might be impressive, but it might not be able to be a business that's a sustainable business,

42:26 right?

42:27 Brad: Or at least as fast paced.

42:29 Scott: Yeah.

42:32 So I hear you on that.

42:33 The thing that I when I first answered your question,

42:37 do I think it's going to continue to do I think it's going to the pace is going to accelerate slower, keep about the same pace.

42:42 I was more answering that in the context of the task that is at hand for us as CTOs working with engineering teams and helping engineering teams leverage these tools better. Right. Like,

42:55 Brad: Yeah.

42:55 Scott: I think that.

42:59 I can just I just look around today and I see the inefficiencies that exist. So, you know, even even if even if all the teams that were right before Dark Factory bumped to Dark Factory,

43:11 right,

43:12 like just getting everybody to Dark Factory,

43:14 that's that's that would be a huge acceleration.

43:18 Right.

43:18 Brad: Mm-hmm.

43:19 Scott: And I don't see I don't see any reason why.

43:24 we can't get to dark factory for most of the things that we're doing right i think that the you know as i've said before like the the the main thing that human in the loop tends to be doing is you don't really know what you want right so you're spending all your time trying like the human in the loop is kind of helping discover the what do you want right so i think that but the but the but the once you know what you want the building of the building of it i think will continue to accelerate

43:52 Right. Right.

43:52 So that's what I mean.

43:54 And I don't Yeah. necessarily think that the models are going to.

43:57 continue to like i think most of the metrics that we have for the models like what like i don't even like at this point i mean i'm not even like okay yeah fable 5.1 came out or soul 5.6 max thinking or what you know it's like they're already good enough that that's not where the acceleration is happening where the acceleration is happening is the are you the tooling around it and the mental models around it and how do we

44:22 evolve our SDLCs to be able to work in this world.

44:26 To me, that's the gap that the acceleration,

44:30 I believe we can accelerate into.

44:33 Brad: yes but i 100 with you and then i also will add to that like fast forward somewhere like say two to five years like we're going to get that part figured out too like right now we use this dark factory phrase right everyone's figuring out their own right and but that's we're going to find the best practice someone's going to come out with you know something called i don't know coffee mug and that's going to be like you know there's react and there's you know there's

45:00 react and there's

44:59 I don't know what is there there's what was before react was started with an a what was that one called

45:05 Scott: Angular?

45:06 Brad: angular you know there's a there's a my point is is the industry is going to settle around some standards for a bunch of this stuff I mean we have standards but coalesce around some things that we all kind of use and that'll even slow that stuff down right because like you know I've got

45:21 there's just in our order there's like different people building their own little dark factories on their computer because they know it can be done but it's like this comes back to what we were i was talking about earlier with how we're trying to do it it's like no let's all do it together and use the same one and just like we've done with software then we can all contribute it and all get better

45:38 Scott: Right.

45:39 Brad: but

45:39 Scott: And I'm just saying like, just even use your analogy of like slowly raising the foundation. Like I think that's a fantastic strategy for you to be employing.

45:45 in your in your organization to kind of get change to do change management across i love it that's great advice i'm going to you know reuse that advice if i you know have the chance um the

45:57 So my you asked me the simple question.

45:59 I mean, we got here because I said,

46:02 hey,

46:02 what about the people that were early adopters that actually weren't skeptics and now some of them might be starting to feel like they're falling behind.

46:08 Brad: Yeah,

46:08 Scott: And I think you asked a legitimate question of like,

46:11 well,

46:11 Brad,

46:11 do you really think it's going to continue to accelerate?

46:13 And my answer is I don't know that it's going to continue to accelerate forever.

46:18 Like it's like I don't

46:20 Brad: know.

46:21 Scott: about that.

46:21 But what I'm saying, I feel like it will continue to accelerate is.

46:24 I can tell there is a gap that

46:27 Brad: Yeah.

46:27 Scott: can be accelerated into.

46:30 Brad: Yes,

46:30 yes.

46:31 You're right.

46:32 Yeah.

46:32 I think it'll continue.

46:33 Yes, I think it will continue.

46:35 Here's my hypothesis.

46:36 This might not age well.

46:40 It will definitely accelerate for another two years.

46:44 But somewhere between two and five years,

46:47 I think it will start to.

46:50 accelerate the rate of acceleration will decrease it'll still be going forward but it'll decrease because i think a bunch of these things that we're figuring out now dark factories and stuff will kind of get settled right and we'll kind of have some stability

47:06 In the industry,

47:06 Yeah, for

47:07 Scott: I'm not going to argue with you the next two years, we're going to continue the acceleration and and maybe that acceleration is going to be I mean, this is a whole different podcast episode, but today was a really bad day if you were an agentic engineer,

47:21 Claude, Cursor and OpenAI Codex all were having major outages today.

47:27 Right.

47:27 Brad: Oh, gosh.

47:28 Scott: So, you know, there's and,

47:30 you know, there's more and more people complaining about.

47:32 about, you know, GitHub stability and all the major service stability.

47:35 And so even when I say accelerate,

47:38 I don't necessarily mean features as much as I mean we have room.

47:44 We have room to

47:45 Brad: Yeah.

47:45 Scott: improve the whole system.

47:49 Brad: Yeah, it will improve. You know, I reflect upon how different things are now than they were a decade ago.

47:57 Scott: yeah

47:58 Brad: well

47:58 Scott: for sure

47:58 Brad: three decades

47:58 Scott: yeah

47:59 Brad: ago right it's

48:00 Scott: yeah

48:00 Brad: wild stuff we were concerned you look in this podcast you mentioned CD-ROMs and

48:06 Scott: right

48:07 Brad: when you when you said that I was like what's a CD-ROM I

48:12 Scott: floppy know disk yeah

48:12 Brad: what a CD I know what a CD-ROM is but like that was just that's just like ancient

48:18 Scott: 30 years ago

48:21 Brad: Are

48:21 Scott: 30 years ago was the last time I think I worked on a software product that shipped on a CD-ROM.

48:27 Brad: you able to share with us what that was?

48:31 Scott: Real player.

48:32 Brad: Oh, really?

48:33 That shipped on a CD?

48:34 Scott: Yeah, there was a version of that that shipped on CD.

48:36 I don't know why it was mostly people downloaded it on the,

48:39 but we're the real, I think maybe we also had like the real server that we would ship that way. But yeah, people could download it.

48:45 But.

48:46 Yeah.

48:47 Brad: Interesting.

48:48 Scott: Yeah, there

48:48 Brad: Well,

48:49 that's a good spot to stop.

48:51 Scott: you go. There you go.

48:52 Brad: I don't know where we got today,

48:53 but it was fun.

48:55 Scott: Yeah,

48:55 it was fun. Like I yeah,

48:57 it was fun. Like I think this is this.

48:59 Like I said, we started this is my happy space thinking about how to build stuff with agents and how to make teams go faster.

49:06 That's my happy space. So you helped me. You helped me, you know, dwell in my happy space for a little bit.

49:11 Brad: I'm glad to be of service,

49:12 Brad.

49:13 All right. All right. What are we going to tell people?

49:15 But we're

49:15 Scott: We're

49:15 Brad: going to tell

49:15 Scott: going to tell

49:15 Brad: people

49:15 Scott: them,

49:16 Brad: what?

49:16 Scott: rate and review,

49:17 smash that subscribe button.

49:20 Brad: Tell your mom.

49:20 Scott: Tell your mom. All right.

49:22 Brad: All right.

49:22 Scott: Good talking to you, Scott.

49:24 Bye.

49:24 This episode of the How Many CTOs Does It Take podcast was produced and edited by Saffron Heptacal. Our theme song was AI Generated by Brad.

49:34 Please follow us on all the social media platforms at How Many CTOs Pod.

49:39 You can find us on YouTube,

49:41 LinkedIn,

49:42 Instagram,

49:42 Facebook,

49:43 Threads, X,

49:44 TikTok,

49:44 and Patreon.

49:45 or at our website at howmanyctospod.com.

49:49 And remember to rate, review,

49:51 and subscribe on the podcast platform of your choice.

49:54 Thanks for listening.