Transcript: Org Structures and Measuring Platform Team Success
0:00 Brad I feel like one of the most important parts, if not the most important part of the job of a CTO is to be that bridge from is the business working to how is this thing called technology that I am the chief officer of contributing to that business success?
0:34 Scott Yeah,
0:34 Scott so welcome to How Many CTOs Does It Take podcast.
0:37 Brad Yeah.
0:38 Scott My name is Scott Porad. My co-host here is Brad,
0:41 Brad Brad Hefta-Gaub.
0:42 Scott and we're here to talk about what?
0:44 Brad The Well,
0:45 Brad you sent
0:45 Scott engineering me. management.
0:45 Brad Yeah,
0:46 Brad you said yeah.
0:47 Brad And you sent me a really interesting sort of list of topics that I think every almost every engineering leader runs into at some point in their career.
0:57 Scott Yeah.
0:58 Brad And I and I think that there's like an endless amount to talk about on these on these topics.
1:04 Brad Do you want to do you want to kind of tee it up
1:07 Scott like? Well, yeah.
1:07 Scott So look,
1:08 Scott we could do that.
1:09 Scott This is all I want to talk about. It's the only problem.
1:12 Brad It's
1:12 Scott It's
1:12 Brad always
1:12 Scott the only
1:13 Brad the
1:13 Scott problem.
1:13 Brad only problem of all businesses,
1:14 Brad right?
1:15 Scott It's funny.
1:15 Scott I was just texting with a friend this morning,
1:17 Scott a guy who's a director of engineering somewhere.
1:20 Scott He was asking me how it's going.
1:22 Scott And what did I say to him?
1:23 Scott I said, well, you know, we've got too much work and not enough people and lots of tech debt. Have you ever heard of a situation like that before?
1:33 Brad Yeah, like that never happens.
1:35 Scott That never,
1:36 Brad Never
1:36 Scott ever.
1:36 Brad ever happens.
1:37 Scott I was going to say the topic on my mind is I've got this engineering team that I'm trying to uplift to help me better.
1:45 Scott I think the team.
1:48 Scott It doesn't feel like it's firing all pistons and the stakeholders don't feel like it's firing on all pistons and not all of that is the fault of the team. Certainly some of it is.
1:59 Scott I would say there are some historical and structural and cultural things in place which make it very hard for the team to thrive.
2:09 Scott But within that contest,
2:10 Scott I'm doing my best.
2:12 Brad Totally.
2:12 Scott So let me let me tee up a topic for you.
2:15 Brad Yeah,
2:16 Scott totally. I mean, I got many topics to discuss.
2:17 Scott Well, to start with this one, I'm going to first I'm going to ask you just general opinions,
2:21 Scott then I'll give you some more detail so you can refine those so they're relevant for you. So the topic is, how do you organize?
2:31 Scott engineering teams to be most effective and I'm going to give you just one willing to start there and then I'll start adding more details so what are your sort of philosophies or principles or ideas or thoughts on how to do that
2:44 Brad Yeah, that's a great rich vein of learnings and failures and successes.
2:58 Brad Man,
2:58 Brad I hate that my answer,
3:00 Brad my off-the-top answer is going to be it depends,
3:04 Brad right?
3:05 Scott Do you know Kevin Goldsmith?
3:07 Brad I don't.
3:08 Scott Well, Kevin's a CTO.
3:10 Scott He was at Spotify a lot. He's a distro kid right now.
3:13 Scott He just published a book. We'll have to get him on here.
3:16 Brad Yeah.
3:16 Scott He just published a book called It Depends.
3:19 Brad Yeah.
3:21 Brad Totally, right?
3:22 Scott Well, what does it depend on? That's the question.
3:24 Brad Well,
3:24 Scott What does it depend on?
3:25 Brad I okay.
3:26 Brad So here's my quick answer to that.
3:28 Brad It depends on what the high order bit challenges that the business and team are facing in the moment.
3:37 Scott Okay.
3:37 Brad Right.
3:38 Brad So a good example of this might be,
3:41 Brad let's say that the.
3:47 Brad You know, let's say,
3:48 Brad for example.
3:52 Brad the stage of the of the software life cycle you know like how old is the code what state are you in a rebuild are you in a building stage are you like starting out like the current venture that i'm working with like we're so brand new right you have a um significant once you're in the legacy uh code base stage then you have to ask yourself well what are the problems that i'm facing right now
4:17 Brad Right now,
4:17 Scott Mm
4:18 Brad right?
4:18 Scott -hmm.
4:18 Brad Like you might have a problem where like it makes sense to have like one of the classic divisions in managing your engineering team.
4:33 Brad And we saw this at the 98.6 where you and I were both together most recently where.
4:40 Brad You could say like,
4:41 Brad oh,
4:41 Brad we have two distinct business problems that we want to kind of accomplish simultaneously.
4:46 Brad So let's have two fully fledged, fully autonomous,
4:49 Brad fully able to execute,
4:53 Brad fully, fully equipped teams to go take on these two different business problems. Like let's split it that way. Right.
4:59 Scott Mm-hmm.
5:00 Brad But you also might be like,
5:02 Brad well,
5:02 Brad actually, these two business problems have.
5:06 Brad the same fundamental foundation that they need to work on top of.
5:10 Brad So let's implement this with a platform team and then two application teams on top of it, right?
5:17 Brad So those are two very different configurations of potentially the exact same problem statement where you intentionally choose two teams or three teams.
5:33 Brad Right.
5:33 Scott Okay, so let me ask you a few questions.
5:35 Scott First,
5:36 Scott I want to throw another it depends dimension in there,
5:39 Scott which is the profitability of the company.
5:45 Brad Oh, yeah,
5:46 Brad of course.
5:49 Scott And this relates directly to the platform team.
5:54 Scott proposed and how it connects to that is in two ways the first is as an engineering manager what am i holding the platform team accountable for how do i measure whether they're being a successful platform team or an unsuccessful platform team is is one dimension aspect there and the other aspect there is in my experience the less profitable the company is or if they're burning money and unprofitable
6:22 Scott The harder it is to explain to the business why you
6:25 Brad Why
6:25 Scott need to
6:26 Brad you need
6:26 Scott invest
6:26 Brad a button.
6:26 Scott in these things that don't seem to really directly derive revenue or
6:30 Brad Totally, totally, totally.
6:31 Scott profits.
6:33 Scott So my question for you is let's just let's set aside the business part of it. We'll get there. If you have a platform team,
6:39 Scott how do you measure?
6:42 Scott I mean it doesn't have to be numerical it could be quantitative but I like numerical better uh you know how do you measure if they're being successful what is it what is the measure of their success in
6:52 Brad I think that like straight out of the gate, there's a couple like there's a couple metrics that are just sort of like you put this on all all teams and all software,
7:02 Brad which is like sort of the does it do what it's supposed to do metrics.
7:08 Brad So those are like.
7:09 Brad Meantime between failure,
7:11 Brad number of, you know, sort of like SLAs, any
7:16 Scott Mm hmm.
7:16 Brad kind of SLA around uptime, any kind of SLA around performance metrics like, you
7:22 Scott Yep.
7:22 Brad know, speed
7:23 Scott Yep.
7:23 Brad of the services or whatever. So if you've got good, you know, quantitative metrics for measuring like, does it work?
7:32 Brad Right.
7:32 Scott Yep, so you have that in place.
7:34 Brad So you got that in place. So one measurement of the platform team is if it's not working, then that's a direct measurement of like,
7:44 Brad hey, platform team needs to make it work. Right.
7:47 Scott Yeah.
7:47 Brad Or if it's not working well enough,
7:50 Brad right,
7:51 Brad like it's you know, you've got some, you know, like you're at.
7:56 Brad you're at three nines and you want to be at four nines you know you're at four nines and you want to be at five nines for whatever reason well those are you know those are varying degrees of is it working um and that's a and so then you're measuring like are you hitting the goal um and then of course
8:16 Brad Well,
8:17 Brad that makes sense,
8:18 Brad right?
8:18 Brad Like we agree on that basic idea
8:19 Scott It
8:19 Brad that
8:20 Scott makes sense.
8:20 Brad there's
8:20 Scott And we put
8:21 Brad metrics where it works.
8:21 Scott this platform team in place so that each sort of application team wouldn't have to do this stuff on its own. We thought there was some leverage there that
8:31 Brad Exactly.
8:31 Scott they
8:31 Brad We
8:31 Scott would get.
8:32 Brad thought there was leverage,
8:33 Brad right?
8:33 Brad We knew that there was this common substrate that the business wanted to go do multiple things.
8:41 Brad They like to have the freedom and the.
8:43 Brad the agility to move quickly on the multiple things and the people building the multiple things, these applications or these higher level things, they're all like,
8:51 Brad gosh, we all need that substrate.
8:53 Brad We all need a reliable substrate to work on.
8:56 Brad We're all for building it ourselves.
8:59 Scott You know the perfect example,
9:01 Scott right?
9:01 Brad Yeah.
9:02 Scott Authentication and user accounts,
9:03 Scott right?
9:05 Scott You know, we got this.
9:06 Scott big SaaS product and a bunch of different things in it. We don't need two user databases.
9:12 Brad Totally.
9:13 Brad Exactly.
9:14 Brad That's a great example.
9:15 Brad And so you get leverage.
9:17 Brad So, okay,
9:18 Brad so we just assumed some facts not yet in evidence,
9:24 Brad right?
9:24 Brad We assumed that the business has multiple applications that it wants to accomplish,
9:29 Brad multiple application lines, multiple lines of business, multiple...
9:34 Brad uh sort of paths if you only have one path if you're only building one thing right so again i'm going to go back to your original meta question like how do you organize a team if you only have one application i would be reluctant to make that platform application split right um
9:57 Brad Because all you've done, if you don't have two applications or you don't have two use cases of the platform layer,
10:03 Brad all you've done by making that split is you've just increased a barrier to communication,
10:11 Brad right?
10:12 Brad You've increased a barrier to autonomy.
10:16 Brad You've introduced roadblocks that you don't really get an advantage of.
10:22 Brad Right.
10:22 Scott Yeah.
10:23 Brad But it sounds like in your case,
10:25 Brad you do have multiple kind of things you're trying to accomplish that are different enough that having them.
10:39 Brad segregate from themselves about the things they're trying to accomplish but borrow the common the common features like authentication you know
10:50 Scott Well, here's where my head's going,
10:52 Scott right?
10:52 Scott As soon as you have something like 10 engineers.
10:55 Scott such
10:55 Brad Right.
10:56 Scott that you'd divide up into two work streams so that group a is building they're all built let's say all these 10 years are building consumer facing you know user facing and it could be consumers but user facing features and capabilities right and you've just divided in two so that you know this team is doing one thing and this team is doing the other thing sort of as soon as you get to two then all of a sudden the need for platform comes in
11:20 Scott Right.
11:21 Brad Right. And the only question is, do you you can still have the so you can have the.
11:30 Brad Code architecture.
11:35 Brad Differentiation and but not necessarily implement that through a team ownership.
11:44 Scott Right, right.
11:45 Scott Well,
11:45 Scott I think what I've seen more commonly is the opposite,
11:49 Scott which is you've got code that is a classic ball of mud,
11:56 Scott and
11:57 Brad Mm hmm.
11:57 Scott then now you've piled many teams on this.
12:02 Brad Yeah.
12:03 Scott um not just two but maybe it could be two usually with two you the team can work it out small enough that everyone can get in the same room and have a conversation right you got a team of 10 engineers let's just say to keep it simple you're going to you know a third of them are going to be senior engineers a third of them are going to be mid-level a third of them are going to be junior you can get everyone in a room and basically work out the details there what happens i've seen is
12:27 Scott Now we're going to pile 15 teams on this thing and
12:30 Brad Yeah, totally.
12:30 Scott I don't know team seven
12:34 Scott you want some feature and authentication and goes and just hacks it in there because they're not really focused on authentication they're focused on doing whatever they want to do to push whatever business goals that's in product manager brought to them ahead
12:45 Brad Right.
12:45 Scott and okay so team seven does that but then you know every team does that once a year for seven years and you've got what's 15 times seven it's like you know you get 100 hacks on the authentication system
12:58 Brad Right,
12:58 Scott and
12:58 Brad right, right.
12:58 Scott now it's just a
12:59 Scott a brittle mess and somebody comes along and says hey let's make this integrate with octa or something and i'm making this up and it's just like oh my god this is this is a disaster and so then someone says well we need a platform team to own authentication and then the platform team is supposed to somehow go like extract all this out of the monolith and then so that's like their first big project extract it out into some nice service and then
13:26 Scott sort of their customer becomes the product
13:29 Brad all the other teams yeah
13:30 Scott all the other teams um
13:32 Brad sure
13:32 Scott and like rinse lather repeat about for however many domains you have in your application which let's just say it's going to be somewhere between seven and fifteen
13:44 Brad right but that's it but what's okay so like if we rewind though like i think that
13:52 Brad I totally understand the way you're going and it's the way that we almost always go,
13:56 Brad right?
13:57 Brad Like we immediately start having the conversation about the teams and who's the ownership.
14:02 Brad And I guess the thing that I would say is maybe that's what we should be challenging and that the first principle is not the who's not the how we divide who's responsible.
14:18 Brad But maybe the first principle needs to be,
14:21 Brad are we aligned on sort of like the code paradigms? Like, are we aligned on what we consider correctness to be?
14:39 Brad Mm-hmm. Because what I think your story is really telling.
14:42 Brad You're sort of saying like every time the team went in, all the seven different teams and went in and they sort of like, you know, layered on their opinion on the authentication service.
14:56 Brad Right.
14:56 Brad It sounds like in that history,
14:59 Brad which,
15:00 Brad of course.
15:01 Brad You've come too late, right?
15:03 Brad Like we always show up as the CTO hired to fix things and it's like,
15:07 Brad yeah, there's already 10 years of history,
15:09 Brad right?
15:09 Scott Right, right.
15:11 Brad But to me, it's like the reason that a lot of organizations go with, well,
15:16 Brad I need a platform team,
15:18 Brad the authentication team,
15:19 Brad and they own the code,
15:21 Brad they own the authentication code,
15:22 Brad and they become essentially gatekeepers of the functionality and the code itself.
15:28 Brad And, you know, what you're partly what you're saying is we need someone to give a damn.
15:36 Brad We need someone to give a damn about this code and to defend it and to be its steward, to be its code steward.
15:46 Brad And a very, very easy way to do that or a way to sort of like enforce that is we're going to give you a team,
15:54 Brad we're going to make you get control and everyone has to come through you.
15:59 Scott Yeah,
15:59 Brad But
16:00 Scott so that's we.
16:00 Brad not the only way to solve that code problem.
16:04 Scott That is true.
16:04 Scott Now let me, this is interesting because I've been kind of going a different direction recently, but I want to explore, and I'll tell you what that is, but I want to explore this for a second.
16:13 Scott It comes back to that profitability question.
16:16 Scott If I am a profitable business,
16:18 Scott okay,
16:20 Scott let's assume the features of my authentication system have been defined.
16:23 Scott It's all set.
16:24 Scott The system has been built.
16:26 Scott It is operating correctly.
16:29 Scott A less profitable business will say,
16:31 Scott I need to redirect these resources to go and work on something to make me more profitable.
16:35 Scott A more profitable business will say,
16:38 Scott okay,
16:38 Scott well,
16:39 Scott I can afford to have a team here just maintaining this thing.
16:41 Scott Right. Because there is a maintenance cost to it and I can afford
16:44 Brad Right.
16:44 Scott that.
16:45 Scott But a less profitable business will say, I can't actually afford that right now.
16:49 Scott I need to put as many resources as I can. I need to devote to getting the profitability.
16:57 Scott I think that adds another dimension of complexity there,
17:00 Scott which is the give a damn.
17:04 Scott It's hard for an unprofitable business to continue to give a damn.
17:09 Scott because they want to redirect the community to have people that give
17:13 Brad Right.
17:14 Scott a damn.
17:14 Scott So you said there's other ways to accomplish that besides having a team.
17:17 Scott What would that be?
17:19 Brad I have to rewind to sort of give you the answer to that,
17:23 Brad but let me kind of give the answer and then I'll rewind. This is something you and I have talked about,
17:29 Brad the rewind we've talked about is.
17:32 Brad You know, the classic question,
17:34 Brad whenever you're whenever you're whenever you're looking at a organization,
17:39 Brad you're trying to up level, you have to answer the question like, is it is it a people process or tech problem?
17:47 Brad Right. And it's almost always a combination of the three. Right.
17:51 Scott Mm-hmm.
17:51 Brad And so my partial answer to the.
17:56 Brad There's ways to give a damn is you
18:02 Brad can institute things like clearly articulating accountability or changes to code. Right. So,
18:16 Brad for example, you can say all code.
18:21 Brad all code must be you can implement processes that sort of force give a damn on code that is community owned right like it's the comedy it's it's the tragedy of the commons right is the problem that we're trying to deal with and and so and
18:42 Scott Yeah, that's true.
18:43 Brad so
18:44 Brad The solutions to the tragedy of the commons is, yeah, you can put an armed guard up and make everybody go through them, right?
18:55 Scott Yeah,
18:55 Brad Or
18:56 Scott yeah,
18:56 Brad you
18:57 Scott that's
18:57 Brad can
18:57 Scott the
18:57 Brad create
18:57 Scott team.
18:57 Brad a team that owns authentication or you can create a culture where everyone values the commons, right?
19:09 Scott Yeah,
19:09 Brad And.
19:10 Scott that's very interesting,
19:10 Scott right?
19:11 Scott Because in that unprofitable business,
19:14 Scott the less profitable the business is,
19:16 Scott the less the commons are valued.
19:20 Brad Well, and it's funny that you say that. That's a really great line.
19:23 Brad It's funny. I was on my run this morning and I was thinking about getting together and talking with you about this. And I was like, I was like, you know, what's interesting? I think I think software engineers,
19:33 Brad computer scientists, people that we hire is basically like in our teams. Right.
19:40 Brad I think that they have a natural kind of innate knack for understanding if things work or are working.
19:50 Scott Mm
19:50 Brad And
19:51 Scott -hmm, mm
19:51 Brad they don't always necessarily have the answers to how to make it work,
19:55 Brad but they definitely know when something's not working,
19:59 Brad right?
19:59 Scott -hmm.
20:00 Brad And I think that applies to not just their code and the way they think about building code,
20:06 Brad but I think that applies to they look around the organization and if they might not be able to tell you, hey, I know why this business isn't profitable,
20:17 Brad but.
20:18 Scott Mm-hmm.
20:20 Brad They probably get this sense when it's not profitable,
20:24 Brad like there's
20:24 Brad Yeah.
20:24 Brad something wrong here.
20:25 Brad Yeah.
20:26 Brad Right.
20:27 Brad And it's not just the business, but it could also be the processes.
20:30 Brad Right.
20:31 Brad Like if you've got processes in place and they're just like,
20:33 Brad yeah, I'm following the process, but it just seems wrong.
20:35 Brad You know, they wouldn't necessarily be able to tell you how to change the process,
20:39 Brad but they're definitely going to have this instinct of like something's not right about this process.
20:45 Brad and the same would be true for culture right and so yeah i think that like getting really clear with engine like don't bullshit an engineer right like getting really clear with them like hey guys like the the auth system is fucked up right we agree right
21:07 Scott Yeah,
21:07 Brad and
21:07 Scott we agree.
21:08 Brad and someone's gonna go like
21:10 Brad Yeah, you're right. It's fucked up.
21:12 Brad Well,
21:12 Brad And maybe.
21:12 Brad you mentioned an interesting
21:14 Brad Yeah, go ahead.
21:15 Brad thing about culture,
21:18 Brad which is what I've seen in a lot of places is where sort of the business side of things is focused on its – how should I say this right? Because the business and the engineering, they all have the same goals.
21:33 Brad What I'm trying to say is – I don't know how to describe this exactly. Let me do my best.
21:38 Brad when the near-term business requirements, and those near-term can still be within a couple of years, when the relatively near-term business requirements and demands and focus overwhelm the caretaking of the commons, right?
21:53 Brad And so the business is kind of cutting off its nose despite its face,
21:57 Brad right?
21:58 Brad But it's really hard to explain to a business person the value of those comments because we can't put it in – the language they speak is dollars and cents,
22:07 Brad right?
22:07 Brad Yeah,
22:07 Brad And so when I say, look, when an engineering leader myself already says,
22:12 Brad look, trust me,
22:14 Brad you do not want to let the authentication system decay.
22:16 Brad right.
22:18 Brad And they're like,
22:19 Brad yeah, but what's it – tell me how that affects my P&L.
22:22 Brad Right, exactly.
22:22 Brad Because that's the because I'm playing a game and that's the scorecard I use.
22:26 Brad Yep.
22:26 Brad And so I'm like, I don't really so it's interesting.
22:30 Brad So we talked about culture, but it's interesting. A part of that culture, I guess, is what I'm trying to say is the balance or the effect of the balance between sort of.
22:41 Brad In the tragedy of the commons, you've got the commons and then I guess you've got the consumers of the commons, right?
22:45 Brad I don't know what you call those people,
22:46 Brad but it's the balance between those things.
22:48 Brad And I guess as engineering leaders, it's our job to really build into, you know, it's hard, but to build into the culture that the business,
22:56 Brad the consumers of the commons understand the value that they want to care take because they understand that caring for the commons is caring for their own business.
23:04 Brad They're almost cheap, right?
23:05 Brad Well,
23:06 Brad and so 100% agree with that point,
23:08 Brad right?
23:09 Brad Like the – if we're basically like talking about like the simple example of the commons is like it's the public square and there's litter all over the place,
23:22 Brad right? Well,
23:23 Brad the – if –
23:26 Brad You know,
23:27 Brad this is a you know, there's an element of like if enough people say I'm going to clean up the litter and I'm not going to ever litter.
23:36 Brad then eventually
23:36 Brad Mm-hmm.
23:37 Brad like it becomes like if it's completely shrewd with garbage then you're going to have to do some kind of intervention but if you can prevent the garbage and the litter from piling up then you're better off now the one thing i would i i'll challenge you a little bit here because i think that um you one interpretation of this is to say hey
24:02 Brad business owners you're pushing the team in such a way that they're unable to maintain the commons right and yes that happens and yes that's
24:24 Brad You got to work that side of the equation to be like, hey, business owners, like we need to think about your ROI.
24:30 Brad We need to think about this tipping point where if you never maintain your law,
24:35 Brad your authentication system,
24:37 Brad you know, here's your ROI on your authentication system.
24:40 Brad If it falls over and gets broken,
24:43 Brad then none of your customers can use your software and use your product at all. And then your whole business goes to zero.
24:50 Brad And in that particular example, right,
24:52 Brad like you can you can retell the stories of past lives,
24:57 Brad right,
24:57 Brad because that was one of the very real issues that we dealt with in our past life where we use this external authentication provider and it was the thing bringing down our SLA, right? Like it
25:07 Brad Oh,
25:07 Brad was
25:08 Brad really?
25:08 Brad the yeah,
25:09 Brad it was the thing that prevented people from patients and doctors from logging into the app,
25:14 Brad right? It
25:14 Brad Interesting,
25:15 Brad interesting.
25:15 Brad was like.
25:16 Brad you know we're like getting these you know we're getting these like you know fry zero you know problems like hey the authentication is down and we had to make the case like we got to we got to get off of this system it's it's killing us yeah
25:28 Brad I've been kind of advocating for a slightly different direction with my team.
25:33 Brad So I told you when we first started talking general thoughts, now we'll just dial in a little
25:37 Brad bit. yeah dial in
25:38 Brad Okay, I've got 120 engineers.
25:41 Brad plus product managers designers etc etc all hacking on one monolith that is 15 years old as you can imagine the tech debt is high cycle time velocity wherever you want to say the ability to get things done rapidly and nimbly is slow
26:00 Brad Yeah.
26:02 Brad it's a it's a multinational app meaning it serves u.s canada and europe so there's complications just not by itself so and you know everyone's unhappy with everyone you know business people are unhappy with how long things take you know there's legitimate reasons things takes long like hey we got to do it for all these countries like it's just you know there's just a bunch of work there but
26:25 Brad And there's, you know, a lot of it's driven by tech, a lot of it's driven by organs that are, okay, so this org is designed, as you would kind of think,
26:33 Brad there's an infrastructure,
26:33 Brad kind of infrastructure as a service,
26:35 Brad platform as a service,
26:37 Brad applications on top, software as a service on top of it.
26:40 Brad That's the kind of idea.
26:43 Brad And the frustration I've had is like,
26:47 Brad when I say this sort of the platform person.
26:51 Brad How do I know, how do you know if you are doing a good job, right?
26:54 Brad Right.
26:55 Brad If you were to present me a single slide with no more than seven KPIs on it,
27:02 Brad which tell me the story of here's how you know if my team is doing a good job,
27:06 Brad what would be on it,
27:08 Brad right?
27:09 Brad And, you know, it's like because that platform layer has many different domains in it,
27:15 Brad it's like, well, it's all over the place, right?
27:18 Brad So I'm kind of contemplating a shift, and I think it's an appropriate shift, or you said it depends,
27:26 Brad for this phase.
27:27 Brad Hold on a second. Before you talk about your shift,
27:29 Brad I want to ask you a question.
27:30 Brad When you asked that platform leader for their KPIs of their platform's performance,
27:37 Brad in your mind's eye, what did you have their answer be?
27:43 Brad I don't know.
27:44 Brad I mean, there's the correctness piece,
27:46 Brad The
27:47 Brad right?
27:47 Brad Are correctest beats that's on the list.
27:48 Brad these systems up
27:49 Brad And then I have another one. I have a second one.
27:51 Brad and running properly?
27:52 Brad Are they secure and compliant?
27:53 Brad Things like that. That's very easily.
27:55 Brad But what I didn't know is.
27:57 Brad How to measure what is the value that these systems are providing to the business? When you say a platform guy,
28:05 Brad you've got 38 engineers working on stuff to
28:08 Brad What are they doing?
28:09 Brad move this business ahead towards its goals. How would you?
28:15 Brad How would you justify or maybe not justify isn't the right word, but how would you illustrate or explain this is how you my team,
28:22 Brad this is how I know if my team is doing a good job moving the business ahead.
28:26 Brad And I guess maybe to me, just, you know, that sort of correctness uptime layer is the system working is insufficient.
28:33 Brad Totally it is.
28:34 Brad Okay,
28:34 Brad There's another big half of this I think you have to also include in that list.
28:39 Brad let's hear it.
28:40 Brad It is effectively,
28:41 Brad are my customers happy with my product?
28:46 Brad Right.
28:46 Brad And my customers are the application developers building on top of my platform as a service.
28:51 Brad And how do I measure their happiness?
28:54 Brad Their happiness is going to be something like the velocity that my customers report back.
29:05 Brad about their ability to move quickly because
29:08 Brad Yes.
29:09 Brad of my platform and so then the specific so for example i might have like you know i mean i can come up with several different like you know um uh quantitative measures for that so it might be something like you know uh
29:25 Brad You know, velocity slash story points of application teams that touch platform APIs.
29:32 Brad Right.
29:33 Brad Mm
29:33 Brad So if
29:33 Brad -hmm.
29:33 Brad you can measure like.
29:36 Brad application teams build you know new features at a rate of you know x units per week
29:44 Brad So
29:44 Brad or you're something
29:44 Brad essentially
29:44 Brad like
29:44 Brad getting
29:45 Brad that exactly
29:46 Brad into your Dora metrics,
29:46 Brad the cycle time of your application teams.
29:49 Brad you want to you want your cycle time of your application teams to be the metric of is your platform delivering value
29:56 Brad Okay, that's a whole,
29:57 Brad this is like a whole other 74 podcast we could talk about in terms of velocity.
30:01 Brad But let's assume that's correct for a second.
30:05 Brad I did not talk to you about this in advance because you were actually walking into my trap.
30:09 Brad Go ahead now tell me your plan
30:13 Brad Well,
30:14 Brad I sort of said to myself,
30:15 Brad fundamentally, we're having these platform teams because we think they are making.
30:21 Brad are what I refer to as product teams or application teams.
30:25 Brad We think they're making the application teams faster.
30:28 Brad That's
30:28 Brad Yeah, why the
30:29 Brad we do it.
30:29 Brad app teams get leverage from the fact that we have a platform.
30:32 Brad That's why we do it.
30:34 Brad So I've kind of taken these, but not all of them.
30:39 Brad Some of the platform teams,
30:41 Brad we do it because we think it saves them money.
30:46 Brad like actual dollars and cents i'll
30:48 Brad Like you could a platform team that owns authentication could all.
30:55 Brad give you a concrete example we have a we have a payments and commerce team right and they're working on things like uh you know a simple project that we're going to write it's not simple but like a straightforward way to explain this different types of retries when things fail
31:10 Brad like we have transactions that fail and we sometimes don't actually the transaction happens through the application and we have money transact but we actually don't ever collect it from someone's credit card so we lose money right and so
31:26 Brad You know, for instance, a variety of reasons,
31:28 Brad right?
31:29 Brad It could be bugs, it could be design of the system, whatever.
31:31 Brad And so they're working at making our making us collect a greater percentage of our actual money so that we're more profitable.
31:39 Brad That's
31:39 Brad Yeah.
31:39 Brad what they're focused on. Okay, so I've taken all these platform teams and infrastructure teams and put them into two categories.
31:48 Brad And there's some imperfection here because nothing is ever just one
31:52 Brad thing. Yeah,
31:52 Brad But
31:53 Brad of course.
31:53 Brad the two categories are either your primary focus is generally
32:00 Brad trying to generate velocity or reduce cycle time or your primary focus is trying to reduce cost actually I shouldn't I should say differently than reduce cost because this is a sticking point with me your primary focus is to operate within the cost budget that has been allocated for this thing right because yeah I could reduce as I said with them I can reduce the cost to zero by turning it off so
32:24 Brad Right, exactly.
32:25 Brad um
32:26 Brad If you need to deliver.
32:27 Brad To deliver your service in the envelope of the budget we gave you.
32:31 Brad Right, right.
32:32 Brad So, and the cost,
32:34 Brad I also say this, the cost is balanced by that correctness, meaning there are two, those, those
32:41 Brad And
32:41 Brad costs
32:41 Brad arguably by
32:41 Brad actually.
32:42 Brad that velocity too.
32:45 Brad Like eventually you get if you if you if you get sophisticated enough about this,
32:49 Brad the cost of your platform,
32:51 Brad if if if your platform costs you a little bit more, but it generates way higher
32:57 Brad fair
32:57 Brad velocity
32:57 Brad yes
32:57 Brad from
32:57 Brad good
32:58 Brad your app teams,
32:58 Brad point that
32:58 Brad you may choose that
33:00 Brad yes
33:00 Brad trade off. You might choose that.
33:01 Brad is right so i guess yes so both teams the the
33:05 Brad the velocity team and the cost team are both have a guardrail which is correctness and then on top of the correctness one is focused you need to operate within the budget we have which is you know x dollars per transaction or cents per transaction what it is and the other is you need to
33:26 Brad reduce our cycle times by whatever target we have and so the reason i'm kind of trying to move that direction is i mentioned we have about 120 engineers here and each of those teams that that group of people is let's just say 50 people altogether and so i can divide those up i have a leader for each of those groups and i can say to that leader very clearly hey brad your goal we have got a we have um you
33:52 Brad We have budgeted $1.18 per transaction in cost this year between hosting and credit card fees and all that kind of stuff.
34:01 Brad Your job is to keep the system correct and operate the system within that budget.
34:06 Brad And if you're doing that job,
34:07 Brad then if you can come to me and say,
34:09 Brad look,
34:09 Brad Scott,
34:09 Brad I'm doing this.
34:10 Brad Right.
34:11 Brad you're doing a good job high
34:13 Brad Yeah, exactly.
34:13 Brad five right and so um and
34:17 Brad And hell,
34:17 Brad within that
34:17 Brad if you can even beat that number, then higher five.
34:21 Brad But your goal is to hit that number.
34:24 Brad goal is to hit that number and within let's say 25 people that we let's say we divvied up equally you've got some you're empowered to figure out how to get there right
34:33 Brad Yeah, totally.
34:34 Brad and same with the velocity person i can say look
34:37 Brad look, your goal is correctness and then figuring out how to reduce the cycle time of these application teams and you get to choose how to do that.
34:47 Brad Yeah, totally.
34:48 Brad And if you can come and so what I like about this approach is it is sort of outcome centric.
34:55 Brad It is not software centric. Now the challenge with this approach is Conway's Law.
35:01 Brad Right.
35:02 Brad as always but we've already got a i already have a conway's law problem which is it's a monolith anyhow so we need to decompose the monolith and maybe that velocity team will say hey we think the best way to solve to improve cycle time is to yank some stuff out of the spondyla or maybe they think it'll make even make the development environment better or maybe they think it'll be make the staging or the cicd pipelines better i don't know what
35:24 Brad Right.
35:25 Brad Okay.
35:25 Brad Maybe it's maybe it's refactoring the front end framework one more time or,
35:30 Brad I
35:30 Brad you know, I
35:31 Brad love everything
35:32 Brad What
35:32 Brad you're saying.
35:32 Brad do you think of this idea?
35:33 Brad I love everything you're saying. I love everything you're saying.
35:36 Brad One of the questions that pops to my mind is, how have you solved the ownership give a damn problem?
35:43 Brad I assume that these two teams are both operating within the same portion of the monolith.
35:50 Brad um it's yeah well no not necessarily i mean like i haven't totally you know okay how to solve the give a damn problem it's
35:59 Brad Oh, do they
36:00 Brad definitely well
36:00 Brad share code?
36:03 Brad everybody shares code because it's all one big thing um but uh you know look let's think about this
36:10 Brad In the velocity team,
36:12 Brad right,
36:12 Brad I'm taking, so one of the things I'm trying to do with this is not kind of have the teams within there be as defined or at least giving these teams the ability to redefine them as they see fit.
36:25 Brad But the ones that I'm taking and bucketing together in the velocity space is what we call the web and mobile platform team.
36:32 Brad So those are basically the teams that are.
36:34 Brad making it easier for our web and mobile developers to to do
36:39 Brad Build cool shit.
36:40 Brad their work
36:40 Brad Yeah.
36:40 Brad yep and i should note for the for the audience right i'm working on a consumer internet product right consumer internet product
36:49 Brad Yeah.
36:49 Brad service right that's what i'm doing here so it's basically web and mobile clients with a back end you know it's pretty pretty standard databases messaging pretty ordinary so in that velocity you're gonna have the web and mobile
37:01 Brad We have a test automation team.
37:04 Brad We've got, there's four in there.
37:11 Brad Oh, there's test automation.
37:14 Brad We have a quote unquote developer experience team,
37:17 Brad which kind of ends up becoming the junk drawer team.
37:21 Brad In
37:21 Brad Yeah,
37:22 Brad theory, they're supposed to actually be working on engineering philosophy,
37:24 Brad but if they
37:25 Brad you don't have
37:25 Brad don't a really.
37:25 Brad junk, if you don't have a junk drawer team,
37:27 Brad you, you, you.
37:28 Brad You aren't really in business,
37:29 Brad I guess.
37:30 Brad it's true um so but they're supposed to be the engineering velocity team but they're kind of end up being junk or two and then we have this thing that we call core services right which is like we're authentication bits okay on the cost side we've got devops security
37:50 Brad payments like i mentioned to you and uh what's the fourth one oh uh support uh you know support experience and uh you know that kind of in terms of helping our support agents be more efficient or resolving issues
38:04 Brad Yeah.
38:04 Brad dispatching on things getting our sports those are the cost side of it so we've got this little bucket right now which you know this little team that we call core services which is the bucket that owns you
38:15 Brad Like what are the projects on their roadmap for like the next year or six months or whatever upgrading authentication upgrading the message bus,
38:22 Brad right? So like they kind of own those pieces
38:25 Brad The commerce the payments team sort of owns the commerce piece
38:29 Brad The support team sort of owns
38:32 Brad the commerce the the support tooling but they're all kind of within this big monolith so and there's other parts that um that
38:40 Brad How
38:40 Brad aren't
38:40 Brad big sure of a how okay,
38:42 Brad so have you rolled this out you're
38:44 Brad well no i'm just having this conversation with people now to talk i'm trying to i'm trying to say like hey i think we could be more effective
38:50 Brad socializing
38:51 Brad if
38:51 Brad the idea and
38:52 Brad i think we
38:53 Brad what's
38:53 Brad could be
38:53 Brad the reaction
38:53 Brad more effective
38:53 Brad what's the reaction
38:55 Brad Well,
38:56 Brad I get a lot of funny looks because people want to have it, they want to, they hear about Conway's Law and they say, well, we want to have the organization reflect the software we want to have, which is an infrastructure layer,
39:05 Brad a platform layer, an application layer.
39:08 Brad And my response is I get it.
39:10 Brad But what I'm actually thinking about is kind
39:15 Brad of.
39:16 Scott management slash effectiveness i'm really thinking about that thing which is as a leader of a team as a leader of a group or even a team how do i tell people if i'm doing a good job
39:28 Scott because i think this gets this is going to get to another thing which is like a topic for a whole other conversation i think our industry has a disease and the disease is we're a bunch of people for whom straight a's isn't good enough
39:41 Brad Mm-hmm.
39:41 Scott And so like we have all these engineering teams and nobody ever knows if they're doing a good job and the pressure on them is immense to always do more because businesses by their nature always want to have more.
39:54 Scott And I think that causes a lot of unhealth and dysfunction and frankly lack of effectiveness.
39:59 Brad I think I've probably told you this before.
40:01 Brad Back a couple of companies ago,
40:04 Brad we had an operations,
40:06 Brad you know, we were running an online business.
40:08 Brad We had a weekly or maybe monthly operations review where all these different teams have metrics and there was I remember this.
40:16 Brad one situation that really struck me there was somebody they had an SLA I can't remember exactly what it was let's just say they had an SLA if something had to happen under like 10 seconds okay and you know we're doing this review and the person leads that group says well last month we were at 12 seconds we did some work we got it back down under our SLA at 10 it's down to seven seconds and we got some more things we're going to work on this month to get it to five
40:44 Brad And I just remember turning to that person saying,
40:46 Brad I don't get it.
40:47 Brad Like you're within the SLA.
40:50 Brad Why would you keep on working on that when you've got other problems to solve?
40:54 Brad Or implicit in that statement is if there's nothing to do, then we should go reapply those engineers to work on something
41:01 Scott Oh,
41:01 Brad else in
41:01 Scott 100%.
41:02 Brad our business because our business has agreed that 10 seconds here is a satisfactory SLA for this thing.
41:10 Brad And what struck me about that so much was that feeling of being good enough. I wanted to say to this person,
41:15 Brad like, hey, you're good enough. You've achieved
41:17 Brad Right.
41:17 Brad the goal.
41:17 Brad Yeah.
41:18 Brad Like, be happy.
41:19 Brad Yeah.
41:20 Brad And something,
41:20 Brad there's this illness in our industry which we are never satisfied.
41:25 Brad Ever.
41:27 Brad And so part of what I want engineering leaders to be able to say,
41:30 Brad this is how I can tell you if my team is doing a good job.
41:35 Brad is actually believed that, well,
41:38 Brad that's important in itself,
41:39 Brad it will create positivity and success begets success and then morale will be positive and there'd be a bunch of positive things that come out of it over the constant like we're not doing enough, we're not doing it fast enough, nothing's ever good enough,
41:51 Brad etc,
41:52 Brad etc.
41:52 Scott Right.
41:53 Scott Okay.
41:53 Scott So everything that you just said resonates with me. And I think that, again,
42:00 Scott like I would.
42:01 Scott I would kind of go back to first principles.
42:03 Scott And I think that the challenge of saying to a group or an engineer,
42:10 Scott an individual engineer,
42:11 Scott an engineering team,
42:12 Scott a group of engineering teams,
42:15 Scott the challenge of saying to them of what does success look like for you? How do you know if you are delivering value,
42:24 Scott right?
42:25 Brad Management 101.
42:26 Scott It's management 101,
42:27 Scott right?
42:28 Scott And I think that one of the one of the this is this is to me the most important part.
42:37 Scott I feel like one of the I feel like one of the most important parts, if not the most important part of the job of a CTO. How many CTOs does it take is being that bridge between we are a business.
42:53 Scott And there is no question about how the business is measured to be a success.
42:57 Scott Is it profitable?
42:58 Scott That's the only answer that matters as a business.
43:01 Scott To be that bridge from is the business working to how is this thing called technology that I am the chief officer of contributing to that business success?
43:16 Scott positively or negatively?
43:18 Scott And what am I doing to make it positive,
43:21 Scott right?
43:21 Brad Mm hmm.
43:22 Scott And that bridge, that mapping of the business to the why we even have a technology group at all,
43:28 Scott that's the core of our job as CTOs.
43:32 Brad I say this all the time. If, you know, if they could do it faster with smokes, faster,
43:38 Brad better,
43:38 Brad cheaper with smoke signals.
43:40 Brad smoke signals were once modern technology they
43:42 Scott Exactly.
43:43 Brad would do it that way today interconnected apps are modern technology that's how they're doing it in the future it's gonna be something else they're gonna do with this is just a means to an I will minor point I will take issue that the goal is not always profitability but we'll set that aside second
44:00 Brad The
44:00 Scott The business business, whatever
44:01 Brad yeah.
44:01 Scott the business objective is, let's let's be clear about that because it could be like it could be could be hyper growth.
44:06 Scott It could be whatever.
44:07 Scott Right.
44:08 Scott But ultimately in capitalism,
44:10 Scott it's about it's ultimately in capitalism.
44:14 Scott It's
44:14 Brad At some
44:14 Scott about.
44:14 Brad point it becomes about profit.
44:16 Brad So yeah, that's right.
44:18 Brad I think what you're saying is, well, what I hear you saying is our job is to connect what we do with.
44:25 Brad We have people who hired us to do it and yet to
44:30 Scott And me and
44:30 Brad this approach is
44:31 Scott to me,
44:31 Brad going to that's right. is going to do that better
44:33 Scott And well,
44:34 Scott here's the point, right?
44:35 Scott I said bridge.
44:36 Scott Right.
44:37 Scott Because.
44:38 Scott you've got to explain to the business owners like why do we fucking matter at all like why do you even have us why aren't you just using smoke signals you should be using us because dot dot dot right
44:48 Brad Yeah.
44:48 Scott you're explaining to the business owners that but you also have to be the bridge we also as ctos have to be the bridge to these engineers to connect them to the business
44:59 Scott Right.
44:59 Brad Yeah.
44:59 Scott Because as I said earlier,
45:01 Scott like innately, they know when something's not working. Right.
45:05 Scott They know if we're treating them like mushrooms. Right. They're not going to they're not going to stand by to be just kept in the dark and fed shit.
45:12 Scott Right. They they know that they have this innate sense that are they are they adding value? Is the business working?
45:22 Scott Is my contribution doing something?
45:24 Scott something right so
45:24 Brad Yeah.
45:25 Scott helping them connect that helping both sides of
45:30 Brad Yeah.
45:30 Scott this conversation connect that's the key
45:35 Brad I think that's right.
45:37 Brad I mean, I feel like I'm getting validation for this notion of where I'm trying to push my team.
45:42 Scott Oh,
45:42 Brad he
45:42 Scott 100%.
45:42 Brad was that correct gotcha
45:43 Scott Yeah,
45:43 Brad so
45:43 Scott I think so.
45:44 Brad is
45:44 Scott And it's
45:44 Brad dhg approved i'll tell everyone that
45:46 Scott a VHE approved.
45:48 Scott But I'm also sort of saying like, I think part of your next step is to socialize this, connect with your engineers by helping them really interrogate the question of how am I adding value to the business?
46:07 Brad that's part of an island you know the thing put the engineer i'm trying to put the engineers in a position which engineers are good at which is solving problems if
46:17 Scott Totally.
46:18 Brad you give i'm if you give an engineer engineers are smart you know not unusually smart but they tend to be a smarter lot and if you say look here's a chart and make this go up and to the left or down to the right or whatever you want to say
46:31 Brad They will find ways to do it. They're smart.
46:33 Scott Yeah.
46:33 Brad Like, oh,
46:34 Scott Give an engineer a problem,
46:36 Scott problem solved.
46:37 Brad yeah,
46:37 Scott Give an engineer the wrong problem,
46:39 Scott wrong problem solved.
46:41 Brad that's right.
46:42 Brad We're so busy climbing up the ladder,
46:44 Brad we neglected to realize it was leaning against the wrong house.
46:51 Brad That's exactly right. Part of me feels like by zooming out a little bit and saying to an engineering leader and a group of engineers,
46:57 Brad this is the metric I want you to move.
46:59 Brad Not these are the software systems I want you to run, but this is the metric I want you to move will free them to do what they're good at.
47:08 Brad and probably do some things that aren't very sexy which will make a real difference so I'll give you a concrete example there's all this talk in our organization about like let's build these teams to build these new platform domain services blah blah blah but there's these other people saying you know our development environment is really slow
47:29 Brad and it's hard to get the monolith is so big that's kind of hard to run locally and it makes it really hard to test things and also you know our business it's hard to get the data in the right condition for doing QA so we don't do a lot of QA locally so
47:45 Scott Yeah.
47:47 Scott Huge problem.
47:47 Brad There's other people out there saying,
47:49 Brad if we want to help engineers go faster,
47:52 Brad fix these problems.
47:53 Brad Don't make yet another domain service that's halfway complete and doesn't have a team to maintain it and so
48:00 Scott And I think
48:00 Brad forth.
48:00 Scott that's a great example of, right,
48:04 Scott what I said earlier,
48:05 Scott which you had already kind of come up to the conclusion yourself. I was saying like a measurement of a platform team is correctness.
48:13 Scott cost of you know cost of running it and then are my customers happy customers being these developers that are using the platform right and
48:22 Brad And the measurement of happy is cycle time.
48:24 Scott and the measurement of happy is probably cycle time right but that's a great example where would you know okay so let's let's play this out should i
48:37 Scott work on making build times faster and local debugging practicable or
48:44 Brad Mm-hmm.
48:45 Scott could I add a new
48:50 Scott authentication layer that is you know one function call and everything else gets handled right whatever I'm just making up two examples those are both great sounding projects
49:04 Scott One of them improves the, you know, one or both of them, but one of them improves the cycle time of more teams more than the other.
49:16 Scott And whichever one it is, is the one they should be empowered to go do.
49:21 Brad That's right.
49:21 Brad And that's their choice.
49:23 Scott Yeah.
49:24 Scott But they need to have it through that framing,
49:27 Scott right?
49:28 Brad Mm-hmm.
49:28 Scott Framing is not,
49:30 Scott well,
49:31 Scott I added more APIs,
49:32 Scott right?
49:33 Scott Like the classic like, you know, how to measure engineering efficiency,
49:39 Scott right?
49:40 Scott The joke of clocks,
49:41 Scott like let's use thousand lines of code as our measurement, right?
49:44 Scott It's the world's worst measurement of engineering efficiency.
49:47 Scott But you could pick a hundred quantitative metrics.
49:53 Scott or a platform team that would all effectively be clocks right like number of APIs you know
50:01 Brad Right.
50:01 Scott who fucking cares an API is not an API unless it's being used to improve the you know cycle time of your app teams I
50:10 Brad Yeah, that's right.
50:12 Brad Well,
50:12 Brad Brad,
50:13 Brad this has been helpful today.
50:14 Brad I feel like I've solved it.
50:16 Scott all right
50:18 Scott How
50:18 Brad I'm
50:18 Scott many
50:18 Brad just going
50:18 Scott videos
50:19 Brad to take this
50:19 Scott does
50:19 Brad recording and
50:20 Scott it take?
50:20 Brad just
50:21 Scott In this case, two.
50:22 Brad I'm just going to take this recording. I'm going to sit my whole team down like when we were in elementary school and the teacher rolled out the VHS cart.
50:31 Scott Yeah.
50:31 Brad And actually pre-VHS cart for me, it was rolled out the film projector.
50:36 Scott Yeah.
50:36 Brad And I'm going to say, kids,
50:38 Brad watch this video.
50:40 Scott Love it.
50:42 Scott Well,
50:44 Scott I love to hear like let's get together a week from today and we'll hear about what kind of resistance you ran into in trying to pitch this idea.
50:54 Brad Yeah, well, I've been I've been socializing it for the last week.
50:57 Brad And, you know, the resistance isn't strong.
51:01 Brad It's just not quite how people imagine it. There's there's been a little bit of a culture.
51:07 Brad a lot of people have been involved in improving velocity and they're kind of they're concerned about well you know I still want to be involved in that and I'm trying to still look it's not that you I'm not saying we're only putting our velocity efforts into this one team I'm just trying to say like this is how this team can tell us they're doing a good job and this is how this other team can tell us you know the cost this is how they can tell us that they're doing a good job and
51:33 Scott Yeah
51:35 Brad It's not perfectly clean because I mentioned for instance that payments team Well, they're also working on some new capabilities for payments like, you know the ability to do say buy now pay later that kind of thing,
51:44 Brad right?
51:45 Brad So it's not perfectly clean
51:48 Scott But Maybe
51:48 Brad at least directionally it allows me to say to a group people like
51:52 Brad This is the target and this is the target the business cares about
51:55 Scott Yeah, it's not wrong to have multiple metrics that you're shooting for.
52:00 Scott And so maybe having them both have both metrics is reasonable,
52:03 Scott but tell them that you currently want the dial of the one team set aggressively towards improved velocity of your users and you want the dial of the other team set more aggressively towards,
52:14 Scott you know, hit your cost metric, you know, run your service correctly on.
52:18 Scott in the cost envelope that we've given you and the fact that you know in in q2 2024 that's where i've set the the dials um uh
52:28 Brad Mm-hmm.
52:29 Brad That's clever.
52:29 Scott you know we might come back we might come back in q3 or q4 and adjust your dials slightly differently uh
52:35 Brad Yeah,
52:36 Brad that's right.
52:37 Scott hey this is another really great episode of how many ctos does it take podcast um
52:44 Scott My co-host,
52:45 Scott Scott Porad. I'm Brad Hefta-Gaub.
52:47 Scott Thanks for taking the time to listen.
52:49 Scott We hope you got something out of it. I know I got something out of this conversation.
52:54 Scott And what do we like to say,
52:55 Scott Scott?
52:56 Scott Remember to smash that subscribe button,
52:58 Scott rate and review.
53:00 Scott And thanks again for spending time with us today.
53:04 Brad This episode of How Many CTOs Does It Take podcast was edited by Saffron Epigal.
53:09 Brad Our theme song was AI generated by Brad.
53:13 Brad Please follow us on all the social media platforms at at HowManyCTOsPod.
53:18 Brad You can find us on YouTube,
53:20 Brad LinkedIn,
53:20 Brad Instagram,
53:21 Brad Facebook,
53:21 Brad Threads, X,
53:22 Brad TikTok,
53:23 Brad and Patreon, or at our website at HowManyCTOsPod.com.
53:28 Brad And remember to rate,
53:30 Brad review, and subscribe on the podcast platform of your choice.
53:34 Brad Thanks for listening.