The State of AI in 2025: What the DORA Report Reveals
In this episode of "How Many CTOs Does It Take?" podcast, hosts Scott Porad and Brad Hefta-Gaub dive into the 2025 DORA Metrics Report on the state of AI-assisted software development. They discuss key findings such as the near-universal adoption of AI, its impact on productivity, and its role in software delivery instability. Brad and Scott also explore the attributes of high-performance teams and the importance of organizational maturity in leveraging AI effectively. They conclude with reflections on AI's probabilistic nature and its implications for real-world applications.
Resources:
- How Many CTOs Pod: https://howmanyctospod.com
- Scott Porad: https://www.linkedin.com/in/scottporad/
- Brad Hefta-Gaub: https://www.linkedin.com/in/bradheftagaub/
Brad: These themes probably existed independent of AI assistance.
These are also attributes of what high-performance teams have available to them.
Scott: All right, Brad,
welcome back to another episode of How Many CTOs Does It Take?
And I'm very excited to actually interview you today because earlier this week you sent me a link to an article that you said you're really fired up about and
Brad: Yeah
Scott: contemporaneously.
I received something similar on the same topic from another person, and I don't know what it is.
I mean, I know the headline, which has something to do with Dora Metrics and AI,
but I actually don't even think we've introduced the show. This is the How Many CTOs Does It Take podcast.
My name is Scott Porad. This is my trusty co-host.
Brad: I'm Brad Hefta-Gaub, and in this episode, we're going to talk about the newest Dora report on the state of AI.
Hi,
software development for 2025.
Scott: before we start we got to take a quick look at hacker news how many of the top 10 articles are about ai one only one the
ai hype is over all right dora metrics and ai let's hear about it tell me about this article that you found brad or this thing that's going on it's more than one article because i got two of them
Something's happening out there.
Brad: Well, so first let's start out. We all know what the Dora program is. It's essentially,
you know.
You know, it's a door is all about research program,
it's kind of supported by Google and it's all about metrics for kind of measuring the overall productivity of your engineering team. Right.
And so we talk about the door metrics are essentially like typically it's the, you know, how quickly you do releases, how frequently you're doing releases, how much downtime you have from releases, how quickly you repair any downtime. Those are kind of the classic.
sector metrics that are out there and yeah so the Dora the Dora group working group basically went out and did a very very extensive research paper so it's essentially like a you know a think tank type you know analysis paper that you might get from a you know a consulting group or any kind of a working group and they looked at
Kind of where are we at in adoption of AI?
Where are we at in how that adoption is? Is it positively or negatively impacting team performance?
What types of teams are being more or less successful with adopting AI?
And it's an extensive report.
So it's 142 page long report.
Scott: And I see that into.
chat GPT or Gemini and just get a summary.
Brad: Probably.
But they did a good job like all good reports.
They lead with the executive summary and they kind of enumerate their key findings.
And I think that's certainly one place that we could start is kind of like what are the key findings that they came with.
Scott: Let's do it. Let's start wherever you want. What are the key findings that they came with?
Brad: All right. So right out of the gate, one of the key findings is that AI adoption has become nearly universal.
So basically, in the people they surveyed, 90% of respondents are saying they use AI as a consistent part of their workflow.
And more than 80% believe that it has increased their productivity.
Scott: I'm skeptical already.
Brad: Well,
there's a really interesting chart that I found on like page 32 of the report.
And then I realized it was actually called out on page,
you know, on that key findings part of the report, which is it's this awesome like the landscape of AI's impact.
And I'm going to just read these sort of the vectors that they kind of got some results.
results on,
and you'll see something interesting in it. So vectors that AI impact,
individual effectiveness.
Okay.
Top impact all the way far to the right.
Yay.
Everybody feels more productive.
Next most impactful,
next most highest impact.
Software delivery instability.
And it is increased with AI adoption.
And the chart says,
note,
an increase here is not a desirable outcome.
Scott: Right.
Brad: uh organizational performance uh positive high impact uh valuable uh work positive high impact code quality positive high impact product performance positive high impact these are all kind of getting less and less impactful um team performance software delivery throughput and then burnout actually above zero so actually increasing burnout and then friction decreasing friction so basically of these various
areas metrics on the impact the vast majority of them are generally considered positive if they are if ai is improving them and three of them would be considered negative if they were increasing and like the the second top increase is software delivery instability is being reported as a real actual outcome of ai
Scott: Let's have an impact.
Brad: it's having an impact uh not necessarily the one that we would want so um so that's kind of a really interesting so i i feel like that's certainly worth like the aha 90 of people are using it 80 of people saying they're more productive with it and um software instability delivery instability is being reported at a
uh estimated effect standardized at 0.1 increase i don't know what the the unit of measure of that is would have to like go dig into that but clearly not a good thing well
Scott: Yeah, I'm just,
look,
I'm going to trust the report that that's what it says, that it doesn't jive with my experience of the engineers I talk to on a regular basis.
I see more. I just hear more skepticism. That's all.
Brad: Actually, they've got some great charts on that. And they actually did something really interesting. And again,
this is like 140 page report. And so there's there's probably something in there for anyone to find,
right?
Like what's the saying lies, damn lies and statistics,
right?
So we can make these statistics say whatever we want them to say. And at 142 pages of statistics,
I'm sure any team could find something in there to support their particular.
political agenda.
One thing I do think is interesting is they did a really interesting deep dive on types of organizations and they categorize these types of organizations into seven different clusters of organizations and those different clusters of organizations were experiencing very different results.
And so they have a whole chapter called understanding your software delivery.
delivery performance.
A look at seven team profiles.
Scott: Mm
Brad: And
Scott: -hmm.
Brad: they basically clustered these groups into,
like I said, seven groups. So cluster one,
foundational changes.
These teams are stuck in survival mode,
facing significant challenges with fundamental gaps in their processes,
environments,
and outcomes.
And about 10% of their survey respondents were in that category.
Scott: Mm-hmm.
Brad: Cluster two,
the legacy bottleneck. Teams in this cluster are a constant state of reaction where unstable systems dictate their work and undermine their morale.
And this is about 11% of their respondents.
Constrained by process,
these teams are running on the treadmill despite working on stable systems. Their efforts are consumed in inefficient processes leading to high burnout and low impact.
17% of respondents were in that category.
High impact low cadence. So these teams are producing high impact work reflected in strong product performance and high individual effectiveness.
However,
these are coupled with low cadence deliveries more model characterized by low software delivery throughput and high instability. So in other words.
and seven percent are in this so these are sort of like they're making high impact but they're but they're kind of trudging along and from a dora perspective right the dora metrics are all about like fast iteration and fast you know delivery so this team would kind of score on a on a sort of like they might feel like they're making impact but they're kind of slow to move
Scott: Yeah, well,
they're aiming well and hitting their target.
They just don't shoot very often.
Brad: Yeah, exactly.
And let's see, I'll rattle off the next three here.
Stable and methodical teams working on steady artisans, delivering high quality code at a deliberate and sustainable pace.
15% of the respondents are there.
Pragmatic performers team consistently delivering work with impressive speed and stability,
even if their environment hasn't reached a state of peak engagement,
20% of teams there. And then the all hail greatest cluster here,
the harmonious.
Monious high achievers, this is what excellence looks like and something like 20% of the door responders are in that category and each of those different segments clearly are and this is one of the things of this door report that.
It's not a tools question.
AI, integrating AI into your development cycle is not a tools question.
It's a systems level question.
And you've got to set your systems up correctly. And systems, of course, we mean like all the systems, like, you know, your cultural systems, your process systems,
your technology systems,
right?
So that's kind of one of the other key things that came out of this report.
Scott: That is fascinating.
You remember John Helm, who was on this podcast.
Brad: Yeah
Scott: He has a saying that I picked up from him 20 years ago,
which to me is very relevant to that little piece right there,
which is he always would say software is just a function of the people and processes who created it.
And so what I hear.
I would adapt it. I'll call it John Helm's AI corollary.
AI effectiveness is only a function.
of the people and processes in which it is embedded.
Brad: Mm hmm.
Scott: Yeah.
Brad: Mm hmm. And and one of the this is this is this was its own key finding here.
So successful adoption of AI requires more than just tools.
Our new DORA AI capabilities model identifies seven foundational practices,
including clear AI policy,
a healthy data ecosystem and a user centric focus.
that are proven to amplify the positive impact of AI on organizational performance.
And so they have a whole chapter on sort of like the things they've found that the team's reporting high performance all had in common.
Scott: Okay, that's handy.
Brad: Yeah, those.
Scott: Now, are those things they found,
are they...
tangible or are they fuzzy like
Brad: Good
Scott: are they things
Brad: question.
Scott: are they things like are they things like
The team has a great culture,
which is very fuzzy,
or they very tangible like,
oh, the team always,
you know, has code reviews done within 20 minutes or making that
Brad: Yeah,
Scott: example up.
Brad: yeah, that's great.
Well,
again,
it's like a whole chapter in this 140-page report,
so I haven't digested all of their recommendations,
but let me rattle off the seven key themes that they identified.
Clear and communicated AI stance.
So a very clear stance in our organization.
What is our stance on AI?
Scott: Easy, okay.
Brad: Working in small batches. I want to drill into that one. That one sounds like a theme that we've seen in all of our discussions and use of AI.
Healthy data ecosystems.
Not sure what that means, but I would imagine that their
Scott: Exactly what
Brad: data makes
Scott: that means.
Brad: all.
Data, you know what that means?
Scott: Well,
I mean, I've worked in unhealthy data ecosystems.
Brad: Okay.
Scott: So, you know, if your data pipeline, what's your data ecosystem?
At least in my world, I work in an online transaction business,
right?
Is your transactional data clean?
Is it getting into, is the pipeline's functioning? Is there a semantic layer that's abstracting your raw data into data that people can do stuff with?
with and then ultimately it's asking the question i think which is or not asking the question but it's saying look if if those parts aren't in place it's really hard to get your ai to do magic stuff because it's operating against the garbage that comes up
Brad: Yeah, so they say that that's exactly what they say about health data systems and they say in our analysis,
healthiness of a data ecosystem is measured as a single factor comprised of three individual indicators measuring respondents'
perception of the overall quality of internal data sources,
the accessibility of internal data sources,
and the degree to which the internal data sources are siloed.
load or divided from one another so
Scott: Can we sidebar here for
Brad: yeah
Scott: a second?
Can I give you all an example of an unclean data system?
This happened to me within the last month.
So an important thing to know is the company I work for has a fiscal year that ends September 1st, and then there's an audit thereafter.
And for a hot minute, there was a panic because the auditors were saying.
some very material number the revenue is off by some very material number from what we reported to what they were seeing like like you know in this business like you know like like a one two three kind of percent like a big number yeah
Brad: That's a big number, yeah.
Scott: well
It turns out there was this operational workflow,
which was some people, so,
you know, we run a two-sided marketplace and sometimes there'd be a problem where I can't remember exactly the details.
Let's just say that the worker wasn't getting paid on time.
Maybe they weren't getting paid on time because the customer's credit card failed or something.
I don't know the details.
So Ops, in order to pay that worker,
would create a second order.
And so here's what I'm going to do. I'm going to create a second order for you. I'm going to mark this as approved and you'll get paid.
And then eventually we'll collect the money from the first order and it'll all work out.
But what it made it look like in our system is that there were two orders.
And what happened is when the credit card actually ran or whatever,
the second order,
the first order looked like it got paid because the operator manually did it. And the second order got paid because the money actually came through. And so.
So the auditors are like, well, wait a second, you look, it says you have too much money here. You don't have as much money as you say you have.
And we ultimately tracked it down to that.
So it was a great example.
Oh,
to me, it's an example of two things.
The data wasn't clean.
And
Brad: Yep.
Scott: also something I call, you know, I sometimes refer to as cry sourcefulness,
which is you have operators just trying to be resourceful and try to solve a problem.
Brad: Yeah, totally.
Scott: And actually,
this wasn't really an example of cry sourcefulness. We'll save that for another.
for another podcast but they were just being resourceful and trying to solve a problem for customers and conceptually in their head they thought well the money's all going to work out it'll be fine but from an accounting perspective it didn't so that's an example of sort of unclean data
Brad: Well, and to sort of go back, that makes a ton of sense. And to go back to that question of like,
what are the what are these attributes that make you more likely to have productivity in your AI empowerment?
And after healthy data ecosystems,
another item that's listed is AI accessible internal data,
right?
Scott: Yeah,
Brad: Like the AI needs to be able to access those healthy ecosystems of data, right?
Scott: that's
Brad: Like that's, that's,
Scott: right.
Brad: that's a key part. And, and, you know, I feel like I feel like this report.
You know, I bet you if we went back in time, like to the 2021 DORA metrics reports,
these themes probably existed independent of AI assistance.
Like these are these are also attributes of what high performance teams have and have available to
Scott: Well,
Brad: them.
Scott: you get teams that can effectively automate things.
Brad: Yeah,
exactly.
Scott: Can I, before you go any further, can I sidebar my sidebar for one more second?
Brad: Yeah,
Scott: That particular problem I told you about,
the sort of audit thing,
it ended up getting resolved and it was fine.
And it actually exposes another thing that drives me crazy.
People are like,
oh, there's a bug in the system, all the revenue's off and all the engineers are out there looking for the bug and nobody can find a bug,
right?
Because there was no bug, right? The system was working.
Just there was like a bug kind of like it was like a people process kind of bug.
But it's always this thing that drives me bananas. And I guess I just take it.
take it personally like people start blaming us oh the system's buggy it's not working and then it sort of makes it way up and it's getting reported in the meetings oh there's a bug in the accounting system and that's why we had this thing that was off but we fix it like no no it wasn't a bug it wasn't
Brad: Yeah,
Scott: a bug it
Brad: it's
Scott: worked just fine
Brad: so funny you say this because I just experienced something that's very similar to that where there's a subsystem that we have that has generally been.
and plagued with bugs and um the theory of the case was like oh there's a bug over here and i'm sure it has to be this flavor of
Scott: Mm
Brad: bug and because here's my here's here's my empirical observations i've made and
and they were like oh we must have broke something because it worked in september and it doesn't work now you know and i was like hold on a second let's go let's like capture what we know what we know we know versus what we think we know and um after kind of like looking at it there was indeed a bug
But the bug actually always existed, and it was just that we didn't see the bug in September because this particular bug related to.
longer and longer time periods expose the Yeah, bug where
Scott: yeah.
Brad: shorter time periods don't and and so it was a real interesting example it's it's not exactly the same thing that you're describing but it's a similar thing where it's so easy for us especially when we've we've kind of like
bumped our head against a particular you know doorframe walking through it every time we're like convinced when we have a headache we must have hit that doorframe again and
Scott: that's right well
Brad: it it turns out we could have a headache for other reasons oh
Scott: i just think i i take that word bug so personally because it fundamentally means like a person made an error right
And anyway,
okay,
so number
Brad: So, yeah,
Scott: four, number four accessible,
accessible internal data.
Brad: number five, no surprise here, strong version control practices.
Right.
Scott: That
Brad: Again.
Scott: one, that one we've got nailed.
Brad: Right.
Right.
Working small batches mentioned that one already.
Users.
Scott: Oh, actually,
hang on a second.
You know where I take that back.
that one we've got nailed that
Brad: But yeah
Scott: one we've got nailed when it comes to code but for a lot of these models I don't think as an industry I think I think some of the machine learning models and stuff have gotten better but I don't know where that the state of the art is I know that in in the sort of
Brad: Data modeling world,
we have these machine learning models and AI models now.
I don't know there's the maturity of versioning the models.
Scott: Well, you know what's interesting?
Brad: It might have developed in the last few years.
Scott: Maybe.
I actually think this is a really interesting question of strong version control practices. And again,
this is a report that's looking at an industry-wide report.
And they kind of identified these seven clusters of organizational kind of characteristics.
And what they're saying here is that, you know, this is a really nice line here.
A key aspect of mature version control is its function as a psychological safety net.
This safety net allows development teams to experiment and innovate with confidence,
knowing they can easily revert to a stable state if something goes wrong.
One of our most tangible examples of this is the reliance on rollback and revert features,
the ability to undo changes swiftly and without.
without the fuss, uh,
is not just a convenience. It's a critical enabler of speed and resilience.
And I got to be honest with you, you said we've nailed that.
And I think that you and I come from a long history of a school of hard knocks learning,
learning that you, you want high quality version control system and you want high quality version control best practices.
Right.
And, you know, but it's a, it's a learning experience for some developers. I was just.
It's just I was literally just talking to my friend who's starting a two guys in the garage project and he doesn't come from a classic software engineering background.
And I was, you know, I was talking to him about, you know, his current challenges and he was saying like,
oh, yeah,
my my buddy says my my my pull requests are way too large.
I'm like, well, how large are your pull requests?
And he was like,
oh,
you know.
a dozen files and like 2000 lines of code changed and i'm like yeah like that's that you're not making it easier
Brad: No.
Scott: on your your your co-founder here you know that's that don't do that um and you know i think that that i would not be surprised if uh there aren't
organizations out there that for whatever reason either because they don't have you know principles don't have the experience to kind of demand this or for whatever reason they started running fast and haven't like kind of standardized on these things that i could imagine strong version control best practices are might might be something that teams struggle with and
AI would,
that would definitely, if you don't have good quality version control systems, and then you're running hard with AI writing lots of code for you, then I am not surprised that that increase in product instability line would go up for those teams.
Brad: Yeah, yeah.
I just, I have a hypothesis that software development practice,
there's like software development practices within a data organization.
on average are not as strong software development practices in the software development organizations are very strong and then within the data organizations are not but that's okay that's a good one that's a real good one i'm glad you brought you dug into that one more
Scott: Yeah, and I blew past this one, but say working in small batches, I want to double click on that one a little bit. That's another good example where this is all about like the degree to which the teams break down their changes into manageable units that can be quickly tested and evaluated,
right?
So how many times have you and I like talked about this concept in engineering efficiency or how many times have we been dropped into an organization and
And, you know, one of the key things that we're looking at is,
well, how do you break down your work? How do you how do you how to make sure your work is bite sized?
You know, that's the that's the way you get quality estimates.
That's the way you get quality reproducibility. That's the way you get, you know, faster turnaround time.
It's such a it's such a kind of core element to what I consider to be high performance engineering teams.
Brad: Mm-hmm.
Scott: Right.
The high performance engineering teams are better at breaking tasks down.
Brad: The name of the game.
Scott: And again,
I feel like this is a good example where if you kind of run into,
you know, headlong into I'm I'm empowering agents,
I'm I'm doing all this AI stuff,
but you don't have like a kind of a system level thinking around we break things down into small pieces.
then the AI agents are happy to follow your lead and write 2,000 line PRs,
Brad: Yeah.
Scott: right?
Brad: And I really want to do an episode with you just to talk about how to train and incentivize teams and individuals to break things into smaller pieces.
It's like a thing that either you get it or you don't.
And the people who don't, how do we teach them to do it?
Different
Scott: Yeah,
Brad: episode,
Scott: that's a question.
Brad: different episode.
Scott: We'll have to think about that.
So the next attribute they have on teams, teams with these attributes are more likely to be successful adopting AI.
A user-centric focus is also measured,
has been included in past iterations of our annual survey.
A user-centric focus refers to the degree to which the teams think about the experiences of the end users of their primary application service.
So a seven-point Likert scale indicators,
apparently.
All right, so that
Brad: because
Scott: sounds to me like,
sure,
it makes sense.
It makes sense to me that if you have a user-centric model,
you're probably going to get more results out of it.
Brad: to some degree AI
feels like one of those things is a solution in search of a problem and if your team is more focused on the problems you're going to find more effective solutions
Scott: That's fair.
That's fair.
Last item that they list is quality internal platforms.
So and again,
I think this is the quality of these platforms.
you know, set of capabilities that is shared across multiple applications or services,
you know, these are kind of like your classic platform conversation,
you know, that you and I'd like to have,
like,
are your internal platforms high quality?
Brad: Yeah, well look we'll just do all that stuff and they'll just be great
Scott: Yeah.
And I think that that I mean, so here's so OK,
so let's zoom the camera out for a second. OK,
so what does this mean?
Right.
So if those are all the so we have this data set where we have 90 percent of people,
90 percent of organizations are adopting AI.
Right.
A whole bunch of people are like,
yes,
this makes me more effective. Right.
And one of the metrics is a,
you know.
0.1 units increase in instability of software releases oh my god right like that's horrible that's like that's like a really bad outcome yeah
Brad: You know, you outlined those seven different archetypes of organizations earlier.
I think it was seven.
Does it do a crosstab of those? And you said the scores of the things that like were better or worse and there's the organizations that do a crosstab. It's like I have a hypothesis based on everything you've said so far.
Here's my hypothesis.
The organizations that are higher performing are getting more out of AI.
Because all the things
Scott: Yeah.
Brad: you just said, which were attributes of an organization that's getting a lot out of AI,
sounded to me like things of high performing organizations.
right and so I suspect that if you're already an organization that I'm just trying to remember the things you said has clean data and has good internal systems and all these things AI is probably really helping you out and if you're an organization that doesn't have that stuff already then you're down in the bottom I don't know what you called it the hapless and helpless or whatever the first group was I don't remember what they were but
You said, what was the first group you described of those groups?
Because they got a great description for the name of it. I can't remember the name.
Scott: Cluster one is stuck in survival mode,
facing significant challenges with fundamental gaps in their processes,
environments and outcomes like that sounds like a bad place to be.
There's two other groups that also sound like they're in pretty bad spaces. The legacy bottleneck are in a constant state of reaction where unstable systems.
dictate their work and undermine
Brad: their Mm-hmm.
Scott: morale and then there's the constrained by process these teams are running on a treadmill despite working on stable systems their efforts are consumed by inefficient processes leading to high burnout and low impact those are the three those are the three like you know um you know heroes to zero scale those are the three in the closer to zero scale you
Brad: In my hypothesis,
I have to look at this report myself, or maybe they've made the data available, is that those three categories,
which I think were about 35% of the people,
Scott: Yeah, 28 to 38%.
Brad: 38.
So 38, that's a lot, that's more than a third.
Right?
That's two out of five, almost people.
Those organizations are not having a lot of success.
Scott: They're not.
Brad: They're
Scott: And yeah,
Brad: not.
But
Scott: well,
Brad: they're not having a lot of success with anything.
It doesn't have anything to do with AI.
Scott: but here's the thing that may and again,
I, you know, I haven't digested all 142 pages of this report,
but I would not be surprised if.
They're going nowhere faster,
right?
Like in other words, they're already struggling and AI just makes them go faster.
They're just struggling faster.
Brad: Yeah. So going slower, faster,
the
Scott: Yeah,
they're going in the wrong direction faster.
Brad: direction faster.
Do you think that this report and the things you've outlined could be used almost as a
The scorecard for organizational maturity assessment.
Scott: Yeah, I think that, you know, I don't speak for the Dora group,
but I think that Dora generally considers themselves as trying to quantify in a very measurable way.
what they deem as the highest priority attributes of mature software organizations.
So I think that's consistent with their vision statement anyway.
Brad: Pretty cool.
Well, you've been calling this a report.
Scott: Yeah.
Brad: It is not a report.
It is a book.
Scott: It's a book.
142 pages.
It's
Brad: It's
Scott: a book.
Brad: been 42 pages.
Some people argue that the old man in the sea is the greatest American novel ever, and it was only like 120 pages.
Scott: There's some pretty cool charts in here.
Time spent using AI on a recent workday.
So we've got this nice clean distribution where the axis is hours.
and the number of respondents that report those hours.
So what do you think the median number of hours that the several thousand people that respond to this report said they use AI in a recent workday?
Brad: Well, before I tell you my answer,
and I have a number in mind,
I want to tell you that I think on all these AI surveys right now.
The answers are inflated.
And I think directionally, it all tells us the same thing. This is my very first skepticism at the beginning.
I think right now, and this isn't exactly right,
but like it's politically incorrect to say I'm not using AI or AI isn't helping me.
Right.
Scott: Yeah.
Brad: So.
I think the answer is 5.75 hours per day.
Scott: Oh, interesting.
Well,
the median was two hours,
Brad: Oh,
Scott: but it's got a pretty long tail going all the way to 12 hours.
Brad: does that include the agents you have working for you?
Scott: Right?
No, this is time spent using AI.
I mean, this person, someone says 12 hours, they're saying they're working 12 hour days and they're using AI.
Here's another one,
reflexive use of AI.
How often do users turn to AI when encountering a problem or task?
So always,
most of the time, about half the time, sometimes,
never.
Where do you think the pig and the python on that one is?
Brad: Sometimes.
Scott: Yeah,
39% of respondents said sometimes that's the pig and the python there.
Brad: pig in the python what does that expression mean oh
Scott: The lump in the curve,
right?
Brad: gotcha uh yeah that well that jives with the two hours to me right right if that if that number was 5.75 hours i would they would have said most of the time
Scott: It's interesting.
This is another,
and I'm not quite sure how I would have answered this question.
General reliance on AI at work, distribution of reliance levels among AI users.
So 8% of respondents say a great deal.
20% say a lot.
37% say a moderate amount.
A little 30%.
reliance on AI by task number one largest reliance on AI by task is writing new code
Brad: Hmm.
Scott: so that's a little surprising to me um because we would have you know code reviews is down at 56 percent and writing new code is at 71 percent so that's kind
Brad: You
Scott: of interesting
Brad: know, we were just talking about breaking work down.
I was talking to one of our engineering managers last week.
trying to name Tom and he told me what he was finding AI really effective for was take we kind of talked about this again we'll call back to Michael Natkin and sort of telling the AI to make you a plan he was saying it was actually being really effective for taking his product requirements document and his technical design document and breaking it down into tasks actually turning those tasks into JIRA tickets it was helping him and his team decompose their work
Scott: Yeah, that's interesting.
So like at least on their survey,
things that sound very similar to that are creating specifications at 56% and analyzing requirements at 49%.
Brad: Yeah.
Scott: So definitely some some,
you know, people are using it for those types of things.
Brad: Yeah, you told me about a client that you have, I don't know if it's an active client or past client,
where you used AI to take their requirements and turn them into more of a spec, a written spec for the engineers
Scott: Yeah,
Brad: and
Scott: feature briefs.
Brad: everybody loved it.
Scott: Yeah.
Yeah.
And in fact, I kind of did a little mini teaching tutorial session with that product owner and say, here's how I'm doing.
Here's
Brad: Yeah.
Scott: how I did it.
And their reaction was,
oh, great.
I'll start doing that too.
And they've been happy with the results.
Brad: Okay.
Scott: So yeah.
Brad: So can I finish with an AI story today?
It's a minor one,
but it's a cautionary tale.
It's an instructive lesson for all of us.
Our CEO recently sent an all company email,
not an unusual thing to happen.
She observed,
and we used Google Workspace at our company, and at the top of the email, there was the Google Workspace Gmail AI summary.
And she was annoyed because she said,
oh, this AI summary isn't accurately reflecting what I said in this email. And in a particular thing that was referring to,
there was a number in there.
It was referring to 33 people.
And it listed it as 33%. And that's what we say 33% of the people and she was like, no, no, it's 33 people.
Okay, well, another person on our team said, well,
my AI summary says something totally different.
Not only does it not make a mistake,
it says something totally different,
and there's an instructive lesson in there.
I said this to our senior leadership team where I'm the only technical person.
AI is not deterministic.
It is probabilistic. You're
Scott: Yep.
Brad: going to get a different answer every time.
And it taught us a little bit about how that, you know, for instance,
Gmail,
they're not.
basing it off the source email they're basing it off the received email but everybody gets a different answer and it's a it's a real lesson with this AI you know it was a good it was a good opportunity to teach my senior leadership colleagues co-workers
counterparts a little bit about how AI works this week with that AI summary
Scott: Yeah.
That is that's a great story. And for sure, those of us that have been plugging stuff into these LLMs for a while, we've noticed this exact same question two times in a row, two totally different answers.
So there you go. That's the state of Dora for AI.
Brad: We'll stick it in the show notes.
Scott: Yeah,
we'll definitely link this in the show notes.
Brad: I love it. Now I'm going to go read that book.
Scott: Yeah.
Brad: All right. It was nice to talk to you, Brad.
Scott: All right. Good talking to you, Scott.
Brad: Till next week.
Scott: This episode of How Many CTOs Does It Take podcast was edited by Saffron Epigal.
Our theme song was AI generated by Brad.
Brad: Please follow us on all the social media platforms at at how many CTOs pod You can find us on YouTube LinkedIn Instagram Facebook threads X TikTok and patreon or at our website at how many CTOs pod calm and remember to rate review and subscribe on the podcast platform of your choice thanks for listening