Transcript: From Bugs to Bloat: Product Debt, KPIs, and the Cost of Features
0:00 Scott How can we know what quantitative way can we know if product management is doing a good job?
0:22 Scott Welcome to this week's episode of How Many CTOs Does It Take?
0:27 Scott A.K.A.
0:29 Scott CTO Therapy.
0:32 Scott My name is Scott Porad, and I am here with my trusty CTO therapist.
0:38 Brad Brad Hefta-Gaub.
0:40 Scott Brad,
0:40 Scott how are you today?
0:42 Scott Brad,
0:42 Scott here's the question.
0:43 Brad What's the question?
0:44 Scott Today, are you in the chair or are you on the couch?
0:47 Brad I think I'm on the couch today,
0:49 Brad but I don't know.
0:53 Scott Okay,
0:53 Scott well,
0:53 Scott I'll sit in the chair.
0:57 Scott I'll just do my best.
0:59 Scott I don't play a therapist on TV, nor do I play one on real life,
1:02 Scott but I will do my best.
1:04 Scott Trent,
1:04 Scott Brad,
1:05 Scott how are you feeling?
1:06 Scott What's going on?
1:08 Brad Well, we just had today was really interesting day.
1:12 Brad We had a incident retrospective. We had essentially a
1:17 Brad Whatever you want to call it, SevZero incident where some customer domains were having problems with logging in,
1:26 Brad like one of the most classic,
1:28 Brad like if they can't log in,
1:30 Brad they can't use the system.
1:32 Scott Mm-hmm.
1:35 Brad The team, this incident happened a couple days ago,
1:37 Brad but today we had our meeting to kind of go over the results of the incident investigation and the fixes and everything.
1:44 Brad And the team basically discovered that there was a bug in code and they had to fix it. And that bug was preventing a particular subset of users to be able to reset their passwords.
1:55 Brad And this subset of users were essentially these.
1:59 Brad legacy accounts from a long long time ago where um we didn't have like a kind of robust authentication system but we had kind of kept this code around so classic example of some tech debt but actually kind of product debt right like it was a bug but the bug was in this code you
2:24 Brad That had the product team still wanted to keep alive,
2:28 Brad but the engineering team had kind of assumed was no longer being used.
2:34 Scott Hmm.
2:35 Brad Yeah.
2:35 Brad So how am I feeling?
2:36 Brad Oh, I'm feeling really proud of the team for they did a really great job.
2:40 Brad Like they're a pretty novice team. They don't.
2:44 Brad you know we don't have a lot of customers we don't have a lot of runtime with this so they're a pretty novice team in doing this sort of thing this is only really like the second kind of real incident they've had to deal with and they did a great job of following the playbook that i gave them of doing a really high quality investigation you know getting a really high quality timeline
3:03 Brad We've got a bunch of recently installed,
3:07 Brad you know, recently, like in the last three or four months, we rolled out hotel observability and they were able to use that to actually just figure out what the heck happened.
3:19 Brad So that was cool.
3:20 Brad That was all good stuff.
3:22 Brad But the team was really beating themselves up because they felt like, why weren't we?
3:30 Brad advocating stronger to eliminate this feature because not only was there this bug in the code but the feature itself has weaknesses to it vulnerabilities to it and we kind of kept the old vulnerable feature around and that was kind of the the fruit of the poison tree that caused the incident
3:55 Brad So it's a tough one,
3:58 Brad right?
3:58 Brad This is a great example where the product owners kind of have to be in that egoless retrospective and they need to be able to consider their part in the incident.
4:14 Scott Yeah.
4:16 Scott So why didn't the team advocate harder for that, stronger for that more or whatever?
4:23 Brad I mean, it's a good question, you know, in the end,
4:26 Brad myself as the CTO and the and kind of the lead developer on the feature, we're both kind of saying like, yeah, I blame myself for not advocating stronger to get rid of this, you know, feature.
4:39 Brad you know there was a perception within the product team that they're that you know for these old legacy users they want to add any friction to make them switch over to some new system and there was maybe a little bit of a misunderstanding of how much friction there really was because it turns out there's probably not that much friction they get like an invite email and then click through it and boom next thing you know they're logged in you Mm know so but
4:58 Scott -hmm.
4:58 Brad you know there's the classic there's the classic you know real concern like hey emails come in and get
5:04 Brad dropped into spam folders and you know people are like i can't log in anymore how do i log in um i mean to answer your question directly like why didn't the team advocate better i think that's there's some soul searching going on you know within the engineering team about like why did we why did we let this
5:24 Brad Fall through the cracks.
5:26 Scott I have a more interesting why patient breath.
5:29 Brad Yeah.
5:30 Scott Because why?
5:31 Scott I mean, because someone didn't big whip. I mean,
5:33 Scott it happens.
5:33 Scott But I actually think here's a more interesting why.
5:36 Scott Software across the industry gets bloated.
5:41 Scott It happens.
5:42 Scott Software gets bloated.
5:44 Scott And software is expensive to build and expensive to maintain.
5:49 Scott It has never been anywhere that's except I've never worked anywhere on any project or any company that's been good or disciplined. I'm not sure the right word to use about, let's call it grooming the bloat, about going back and saying,
6:06 Scott do we really need this?
6:07 Scott Yeah. I can think of actually one time not that long ago.
6:15 Scott This is when I was primarily working on Dolly. So this is the 2021 through 2024 time frame when we were kind of building, we were kind of, we weren't totally rewriting something,
6:32 Scott but effectively we were kind of rewriting something.
6:34 Scott And there were all these features, you know, there's a long list of features that.
6:40 Scott we need to do include and we were able to persuade product management like well look let's let's do the very very what seems like an uncomfortably bare bones version of this new rev of this thing and put it out there and then we'll see if we really need these things and
6:56 Brad Yeah, that's a great idea.
6:57 Scott i have gotten you know it was took a lot of negotiation a lot of it was a you know we all got along so it wasn't like fighting but it was definitely
7:05 Scott not ideal and but I've since gotten positive feedback from that product management team of like yeah it turns out we didn't really need a lot of this stuff and some of it
7:19 Scott was needed and got added and so sometimes it was very painful to add it I'm sorry I should say it wasn't painful to add it it was painful to wait while it got added customers
7:29 Brad Yeah.
7:30 Scott were having a bad experience but so I think the real question I want to ask is as an industry as teams why aren't we very good at that product cruft bloat grooming and how could we be better at it
7:41 Brad Yeah,
7:42 Brad that's a good question.
7:43 Brad I sort of was like maybe there's like
7:44 Brad there's like a marie kondo like does it spark spark joy if
7:48 Scott Yeah.
7:48 Brad not get rid of it you know is it did you use it yesterday or does it spark joy if not get rid of it yeah
7:55 Scott Yeah.
7:55 Scott I worked with an engineering director once who was really good at that.
8:00 Scott He was just great at throwing stuff out.
8:02 Scott I wonder if it's because we don't account for the cost very well.
8:07 Scott So this was like a...
8:10 Scott Going back to that way old John Helm conversation we had about six months ago,
8:15 Scott this was like a call option that came due.
8:18 Scott It wasn't causing you guys any harm until the minute it was.
8:22 Brad Yeah.
8:22 Scott But so there was an expected cost, you know, you could you could compute an expected value or an expected cost of just having that around.
8:30 Brad Yeah, and I think that, you know, give the product team credit where they deserve it, that they did genuine, like when we, you know, in retrospective now we're talking about it and it's like.
8:46 Brad there was a little bit of like wait we thought we decommissioned all these accounts and there was a well i made a choice we made a choice this customer was special we wanted to like not add any friction to that customer right
8:57 Scott Mm
8:57 Brad i
8:57 Scott -hmm.
8:58 Brad mean i think that's a legitimate stance to take for your product team right like you you want to be able to say like hey this is a this is a legacy customer they're a vip customer whatever the reasoning is to say like that that
9:10 Brad it's our perception that that feature is important to
9:14 Scott It's
9:14 Brad them right
9:15 Scott valuable.
9:15 Brad is valuable to them and um that's the job of the product team right like we like you know like let's we're all in this together and each of us have different jobs and and one of the jobs is to to really be that strong advocate for what value the customer sees in a feature
9:38 Scott Mm-hmm.
9:38 Brad I think that in this particular case, you know, it's complicated and I'm not going to get into all the little gory details, but I think it's it's also the case that in this in this particular case, the engineering team in retrospect.
9:52 Brad sort of silenced themselves they cut themselves off early and didn't say yeah but this is a problem like this isn't just a thing of like an old feature that we'll have to kind of remember to work around or but like hey it's got it's got vulnerabilities to it we should definitely like get we should aggressively get people off
10:10 Scott It's
10:10 Brad of this
10:10 Scott a
10:11 Brad thing
10:11 Scott different class. It's in a different class of its own because of those
10:14 Brad yeah
10:14 Scott vulnerabilities.
10:14 Brad so it's not the same as like a you know a cool little
10:18 Brad widget that the particular customer really loves and you're kind of you know kind of working around it to keep it active you know like whatever it doesn't fall into that category it falls into a category of something that's a little bit more serious and probably should have been taken seriously I think that there you know again we're a pretty young organization I think we're learning to build out our processes and if we had a little bit more process it might have been there might have been a little bit more structured to the way we would have that conversation
10:44 Brad station,
10:44 Scott Mm-hmm.
10:45 Brad right?
10:45 Brad Like, I think that, I think that
10:48 Brad You know, again,
10:48 Brad it's not like we're not dealing with, you know, like if this had been at 98.6, right,
10:53 Brad then there'd obviously be like health related, you know, HIPAA related things,
10:57 Brad SOC 2 related things,
10:58 Brad all these very strong compliance types of patterns would have would have kicked up and there would have been a place to have
11:06 Scott Mm
11:07 Brad that conversation.
11:07 Scott -hmm.
11:07 Scott Yeah.
11:07 Brad Right.
11:08 Brad We're smaller.
11:09 Brad We're younger.
11:10 Brad This is a, you know, kind of a much more.
11:13 Brad lightweight related SaaS offering that is you know really about like it doesn't have those types of implications so that typically have those kinds of like security type implications around it and we're moving fast and we're trying to innovate we're trying to build stuff and I think that you know I think that one of the we've talked about this in the past that one of the really cool things about being at a startup is that
11:40 Brad Everyone has to be on their toes.
11:42 Brad Everyone has to, if they see something,
11:44 Brad they have to say something,
11:45 Brad right?
11:45 Brad There's nobody coming through the kitchen to clean up the dirty dishes,
11:49 Brad right?
11:50 Brad And so if anything,
11:52 Brad I think the team probably in reflecting on this kind of feels like that's the piece they let down.
11:58 Brad That's the part they let down, right?
11:59 Scott Yeah
12:00 Brad Where they accepted or didn't think to say,
12:04 Brad hey, this is really important. Let's follow up on did we convert all these customers.
12:07 Brad customers over yep
12:08 Scott I feel like I heard your team learned two good things today.
12:12 Scott One, you need to have product management in your retrospectives.
12:16 Brad which they were so that was good
12:18 Scott Oh,
12:18 Brad and
12:18 Scott they were. Okay,
12:18 Brad i yeah
12:19 Scott I thought.
12:19 Brad they were there they were there and i would say i feel like that's pretty important for most retrospectives
12:26 Scott Yeah.
12:28 Brad right unless
12:28 Scott For
12:28 Brad it's
12:28 Scott sure.
12:28 Brad really just a raw it was only engineering involved right but if it's got any kind of a whiff of a product feature gone awry or a business process gone awry you know i think you once told me a story about a retrospective where it was like a demo was happening in a place that shouldn't have been happening and the demo caused some kind of system to go sideways or something like that you know um
12:55 Scott I don't remember that, but I'm sure that's happened.
12:59 Brad So anyway,
13:00 Brad I think there's a lot of value in having some representative of your product team be part of retrospectives that get read on product features.
13:08 Scott And the other thing I heard is there are certain classes of things that a team should push harder on.
13:19 Scott And
13:20 Brad Mm hmm.
13:20 Scott that varies by business, but certainly security,
13:24 Scott I don't know if this is a security vulnerability,
13:26 Scott but certainly vulnerabilities of a certain type are one of them.
13:29 Brad Yeah.
13:30 Brad Yeah.
13:31 Scott You
13:32 Brad And
13:32 Scott said
13:32 Brad I agree that that that is kind of the way that those are,
13:34 Brad I think, the key that that second one in particular,
13:37 Brad I think, is the key.
13:39 Brad uh takeaway you know because it was cascading failures of course it was never one thing right it's cascading failures right like um you know and i and i ironically it's almost like the bug actually kind of helped mitigate the vulnerability so it's almost like while fixing the bugs when they discovered the vulnerability they're like oh no okay wait a second you know and so the cascading failures were all around like
14:04 Brad Wow, the code review let the bug get through the QA process let the bug get through the, you know, and,
14:11 Brad and, you know, there was all kinds of cool good things that came out of the retrospective. Why was the code? Why did the code review let the bug get through?
14:19 Brad It was a massive.
14:20 Brad uh pr that was related to it in fact it was about adding observability like the pr that introduced the bug was about adding observability and
14:31 Scott Yeah, the irony.
14:32 Brad so it was a classic like oh we're it's a big pr but really it's only just sprinkling some if then statements around everything kind of thing right let's you know so so you know how many times have we learned that lesson right
14:47 Brad Right.
14:47 Brad Massive PRs are impossible to code review.
14:50 Brad Right.
14:50 Scott Yeah.
14:52 Scott Yeah.
14:52 Brad The.
14:53 Scott Hey,
14:53 Scott can I ask a question about something you said earlier?
14:55 Brad Yeah. Yeah.
14:56 Scott You said a phrase that I had never heard before.
14:59 Scott So now the patient,
15:01 Scott the client is going to teach the therapist here for a second.
15:03 Brad OK,
15:04 Brad yes.
15:04 Scott So you talked, I think it was, I think you said.
15:09 Scott Well, I'm thinking of, I'm going to have the word wrong because the word in my head is OLED,
15:13 Scott like TVs now,
15:15 Scott but
15:15 Brad Oh,
15:15 Scott she was called like
15:15 Brad yeah.
15:15 Scott OLED or OTAP something or observability.
15:18 Scott What's
15:18 Brad that?
15:18 Brad Open, open telemetry.
15:20 Scott Oh, what is that?
15:21 Scott I never heard that before.
15:22 Brad Open telemetry is a kind of industry pseudo standard,
15:26 Brad a little open source, a little bit kind of standard, and it's around like telemetry in general.
15:33 Brad A lot of the big telemetry providers like Datadog and,
15:37 Brad you know.
15:38 Brad amazon and stuff like that they have like o-tel compatibility um but open
15:43 Scott Oh.
15:43 Brad o-tel stands for open telemetry uh it's an open source project um that that has you know
15:51 Brad It essentially has coding plug-ins for all the different languages and all the different stacks,
15:55 Brad and it standardizes on the way that the essentially the formatting of the telemetry data so that people can build like data analyzers or data visualizers, and those visualizers can just say, I support any open telemetry formatted telemetry data stream.
16:13 Scott Address you learn something new every day, even
16:15 Brad Yeah,
16:16 Scott when you're not there trying.
16:16 Brad you go.
16:17 Scott Awesome.
16:18 Scott So you said something else in there that I wanted to ask about that actually reminded me of a conversation I had today with somebody.
16:26 Scott You said product management.
16:29 Scott This is a little bit of a pivot, by the way. There's a pivot of topic, if that's okay.
16:33 Scott You said that product management, you know,
16:36 Scott their role is advocating for what's valuable to the customer.
16:39 Scott Okay, so I have a report that is produced on my team every month, which is a set of engineering KPIs.
16:50 Scott We have about five or six top level KPIs, but then overall,
16:55 Scott which is like the sort of executive summary.
16:58 Scott And but then overall, I bet there's about 40 different KPIs and they include, you know, things like cycle time.
17:05 Scott They include number of open bugs,
17:08 Scott average time to fix a bug,
17:09 Scott uptime, just lots of different KPIs.
17:12 Scott And they're all under the umbrella of this.
17:16 Scott How do we know if engineering is doing a good job?
17:20 Brad Okay.
17:20 Scott And I started doing this a while ago because I got frustrated that stakeholders would have opinions about whether the engineering team was doing a good job and they were entirely subjective.
17:30 Scott And so I said,
17:30 Brad Yep.
17:31 Scott well,
17:31 Scott could we at least like try to quantify what are the things that you're actually expecting us to do and could we try to quantify that?
17:40 Scott And so we've got that and,
17:43 Scott you know, the stakeholders I work with really appreciate it because they appreciate the transparency.
17:48 Scott I think they don't really,
17:49 Scott but that's beside the point. I think it gives me a tool to tell the story of our team and how we're performing. And it also gives me, you know, sometimes it's you got to be vulnerable.
17:59 Scott I'll give you a concrete example. Last month, our mean time to response,
18:02 Scott which is one of our top six mean time for incident response,
18:05 Scott which is one of our top six.
18:07 Scott metrics was bad because we had an incident that we did not respond to for two days and so we have like
18:13 Brad Oh,
18:13 Scott a five
18:13 Brad yeah.
18:13 Scott minute SLA and it was like 213 minutes on average for that month right so um because because we didn't right
18:21 Brad Yeah, you have one two day response with a five day SLA or five minute SLA and you're pretty much screwed for the until you reset the counter.
18:29 Scott but it's a it's a tool for us as a team both to tell our story but for us to also know like
18:34 Scott Like,
18:34 Scott hey,
18:34 Scott this is what is important to us, right,
18:37 Scott as a team in terms of, you know, both operating our system and delivering new software.
18:44 Scott So, okay,
18:45 Scott so just today somebody asked me, someone on the engineering team asked,
18:52 Scott well,
18:52 Scott how do we know if product management is doing a good job?
18:56 Scott And I said to myself,
18:58 Scott That's a good question.
19:00 Scott How do we know if product management is doing a good job?
19:04 Scott And that's the question I want to ask you. How do we know if product management is doing a good job? Is there a set of KPIs that we could put to that? And to be a little bit fair to you and to product managers out there,
19:18 Scott I've been thinking about this set of KPIs that I use for engineering for my entire career pretty much.
19:24 Scott So it's taken a while to refine it.
19:28 Scott Maybe it'll take like another 30 years to figure out this stuff with product management, but maybe there's some product managers or some product chief product officers or whatever out there that know it. I don't know. What do you think? How do we know?
19:38 Scott What quantitative way can we know if product management is doing a good job?
19:42 Brad Yeah, that's a great question.
19:44 Brad First of all, I would love to get some chief product officers to come onto the pod and talk to us about that. I think that's something that, you know, early on in the in us talking about doing this pod, we talked about the idea of like bringing in these stakeholders from different disciplines and talking to them about their their experience.
19:59 Scott Working with CTOs and I think this would be a great topic to to dive into with them like what's the what is the the models that they have seen work successfully between kind of like managing the that relationship and a piece of that relationship between engineering and product is the what what is the definition of a successful product
20:18 Brad Well,
20:18 Scott team.
20:18 Brad if you're one of our 39 listeners out there.
20:22 Brad If you are a chief product officer or no one, we would love to hear from you.
20:26 Brad But let's talk about it.
20:27 Brad So this is, okay,
20:28 Brad I have a few ideas,
20:29 Brad not comprehensive,
20:31 Brad but certainly
20:31 Scott Yeah.
20:32 Brad one of them is I think of product managers as investors.
20:36 Brad They're investing engineering time,
20:38 Brad which is money because we pay wages to engineers.
20:41 Brad And what is the ROI on those investments?
20:45 Scott Mm-hmm.
20:46 Brad It's actually the first one is, well,
20:48 Brad how accurate are you at predicting the ROI on an investment?
20:52 Brad And then there's an even more basic one.
20:54 Brad Of the features on your backlog, what percentage of the investments on your backlog have you calculated an estimated ROI for?
21:04 Brad That's like the very basic,
21:05 Brad right?
21:05 Brad Do we even attempt to try and see if these investments are worthwhile?
21:11 Scott Yeah, totally.
21:12 Scott And I think that, you know, I think that sort of opens this can of maybe it's worms, I don't know, but it opens a can around,
21:22 Scott you know, how do you measure,
21:24 Scott how do you measure the, the result of a feature, right?
21:30 Scott Like an obvious.
21:32 Scott You know, like you and I talked about this on a, you know, we discussed this on an episode once before for a few minutes where it's like, okay,
21:39 Scott like just because sale, it's just because sale number go up, you know, revenue number go up, that doesn't mean that it was the features that got asked for that made the revenue number go up, right?
21:50 Scott It could have just been like better marketing that month or better sales that month or all of a sudden, you know.
21:57 Scott You know, a customer started using your
22:00 Brad Mm
22:00 Scott product more,
22:00 Brad -hmm.
22:00 Scott you know, like they could be using the old features, not the new features. Right.
22:04 Brad You can run
22:05 Scott So,
22:05 Brad a split test.
22:05 Scott yeah, I think it's really.
22:07 Brad A-B test.
22:08 Scott Yeah, certainly.
22:09 Scott If you have enough scale to do A-B testing,
22:12 Scott that's certainly the type of thing that you can do.
22:15 Scott I also think that there are other kind of classic metrics like net promoter score and things like that where you kind of try to judge like does your customer base love your product or not?
22:26 Scott Does that change over time when you add more features?
22:30 Scott Yeah, it's a good question.
22:32 Scott I do think that.
22:32 Scott I do think that you're right that the main value add that we would expect from a product team is identifying the surface area of, you know, customer needs that the customers value enough that they will exchange money for that value,
22:54 Scott right?
22:55 Scott Like that's the most important role of a product
22:57 Brad I
22:57 Scott team,
22:57 Brad agree.
22:57 Scott I think.
22:59 Scott um and so any metric that's going to help us measure that is certainly going to be a good metric yeah
23:06 Brad I think about like after we put a metric around,
23:08 Brad they wrote, you know, the product manager wrote good requirements.
23:13 Scott i mean there are like certainly a whole bunch of uh logistical things that um
23:20 Scott matter too,
23:21 Scott like, you know, I'll pivot a little bit, but let's stick to the topic of the role of product management.
23:28 Scott I'm a big believer that you can't do a bug triage or you can't do a kind of technical debt assessment without having the voice of product there.
23:45 Scott Right.
23:45 Scott Like like I need to like the product owner needs to tell me if a bug really reads on that customer value or not.
23:54 Scott Right. And if if there's a bunch of bugs or a bunch of deficits in some manner that is having some kind of a drag or some kind of a negative effect on the value perceived by the customer.
24:11 Scott Then I I need a great product partner who's going to be able to help me weigh those two things like, hey,
24:17 Scott do I.
24:18 Scott Do I want to add this new whiz bang feature or do I actually want you to go improve the behavior of existing features by fixing bugs, by making them faster or whatever that might be.
24:28 Scott And that to me, that's an inherently a product question,
24:31 Scott not an engineering question.
24:32 Brad I agree 100%. To me, bug triage is a product management function because to us, to engineering,
24:37 Brad it's just an item in the sprint backlog.
24:41 Brad And you could pick fixing
24:42 Scott Yeah, totally.
24:43 Brad an old item or...
24:44 Brad We're making a new item. It's just a thing we're going to chew up and spit out the other end.
24:48 Scott Yeah.
24:49 Scott So I think that that falls into that category.
24:52 Scott So like the whole like how quickly does it take us to adjudicate? So like how quickly do we fix bugs?
24:57 Scott Like maybe that should go on the engineering
24:59 Brad Yeah,
24:59 Scott KPI.
25:00 Scott But how quickly do we adjudicate whether or not we're going to fix the bug?
25:04 Brad could be,
25:05 Brad yeah.
25:05 Scott That feels to me like a KPI that should almost live in the product owner KPI list,
25:11 Scott not the engineering
25:12 Brad Yeah,
25:12 Scott KPI list.
25:12 Brad this is an interesting question.
25:17 Scott And I think there's a handful of those types of things. So you mentioned like, you know, what's the, you know, some qualitative measure of the of the of the description of the specification of the new features,
25:29 Scott you know, whatever that might be some kind of a way to.
25:32 Scott you know, measure that.
25:35 Scott To me, that adjudication of bugs,
25:39 Scott you know, some sort of ROI analysis of features,
25:42 Scott these are all types of like logistical things that I think relate to like,
25:46 Scott are we capturing the value to the customer?
25:51 Scott of the Yeah. software we're building.
25:53 Brad Well,
25:53 Brad that's one dimension. And then the other dimension is, is the product manager effectively conveying to the engineer what to build?
26:02 Brad But a product manager is a tough job.
26:08 Brad to acknowledge all the product managers out there. We love you.
26:11 Brad I have worked as a product manager before.
26:13 Brad I've also worked as a manager of product manager support,
26:15 Brad so I get it.
26:16 Brad We have a hard,
26:17 Brad we product managers, I'm putting my product manager hat on.
26:20 Brad We have many jobs. One of them is understanding the marketplace, the landscape,
26:25 Brad et cetera, looking, finding opportunities and problems.
26:28 Brad That's like a kind of, I think of that as the outward facing job.
26:32 Brad And then we've got a prioritization job, like how we judge what order we put these in.
26:36 Brad And then we have a communication job of like,
26:38 Brad how do we tee this up for the engineering team to go execute on it?
26:45 Brad You know, some companies do that depending on what you're building and all sorts of cultural issues with a very detailed spec.
26:51 Brad And with some people, it's just a conversation. That's what Agile is about, what Scrum is about.
26:54 Brad Let's just have a conversation and go build some stuff and we'll come back and tell you if you tell us if it works for you.
27:00 Scott Yeah, totally.
27:02 Scott Yeah.
27:03 Scott And then there's that, that's all kind of the input stuff. And then the output stuff too.
27:05 Scott You've got like, depending on where you divide the lines between product and kind of sales and marketing too, you've got the, how do you communicate the value,
27:14 Scott you know, to the, to the team that's communicating the value to
27:18 Brad Yeah,
27:18 Scott potential customers.
27:18 Brad all the go-to-market stuff,
27:19 Brad yeah.
27:20 Scott Yeah.
27:21 Scott I was just going to say like,
27:22 Scott uh,
27:23 Scott I'm feeling like I was hurt,
27:26 Scott Scott.
27:27 Brad Well,
27:28 Brad I was feeling like we're almost 30 minutes into this conversation and we have not talked about AI yet,
27:32 Brad which
27:33 Scott Are you sure you want to talk about AI in this
27:34 Brad to
27:34 Scott episode?
27:35 Brad me is therapeutic all by itself.
27:40 Scott I love it.
27:41 Scott Well.
27:42 Brad Well,
27:42 Scott I need
27:43 Brad Brad,
27:43 Scott to
27:43 Brad it's been a good talk today.
27:44 Brad I don't have anything to say about AI today,
27:46 Brad but maybe
27:46 Scott We need to talk about AI at all.
27:48 Scott Well, let's make sure we cover it next because there's always something new happening in AI.
27:53 Brad All right,
27:54 Brad well, until next week,
27:55 Brad Brad,
27:56 Scott All right. Till next week.
27:57 Brad smash that subscribe button,
27:58 Brad rate and review.
28:00 Scott Rate and review.
28:01 Brad Bye, everyone.
28:02 Brad This episode of How Many CTOs Does It Take podcast was edited by Saffron Epigab.
28:07 Brad Our theme song was AI-generated by Brad.
28:11 Brad Please follow us on all the social media platforms at at how many CTOs pod you can find us on YouTube LinkedIn Instagram Facebook threads X TikTok and patreon or at our website at how many CTOs pod comm and remember to rate review and subscribe on the podcast platform of your choice thanks for listening