How Many CTOs?
S2E45 · November 17, 2026 · 53:42

AI, Compliance, and Consumer Connection: The Future of Fintech with Olly Downs

Listen to the audio 0:00 Episode Hook
0:00 53:42
Transcript

In this episode

In this episode of "How Many CTOs Does It Take?" podcast, hosts Scott Porad and Brad Hefta-Gaub interview Olly Downs , Chief Technology, Product, and AI Officer at Curinos, about building decision intelligence for financial institutions. Downs describes Curinos' comparative analytics for deposit and lending pricing, and how its platform models strategy and operationalizes it through existing bank systems to drive efficient acquisition, relationship deepening, primacy, retention, and profitability.

The discussion covers banking's rapid adoption of generative AI, the need for deterministic constraints and auditable "bounding box" rules, and detailed observability and evaluation practices using tools like Fiddler AI and Monte Carlo Data. Downs explains why Curinos is cloud-based on Databricks, outlines a platform-oriented org structure, and highlights bottlenecks in compliant AI-assisted coding and the continued importance of clear product definition and change management.

The conversation ends with the differences between continuous vs quarterly releases and the need for engineers to celebrate their software achievements.

← Back to episode

Transcript: AI, Compliance, and Consumer Connection: The Future of Fintech with Olly Downs

0:00 Olly We haven't found the right way yet to ensure that our code is fully compliant if we don't do our own code reviews.

0:24 Brad Welcome, everybody, to another episode of the How Many CTOs Does It Take podcast. My name is Brad Hefta-Gaub, and this is my trusty co-host.

0:32 Scott Scott Coran. Hello, Brad.

0:34 Brad How are you doing today,

0:35 Scott How

0:35 Brad Scott?

0:36 Scott are you? Good. And I'd love to introduce our guest today. We've got Ollie Downs with us, who's

0:41 Brad Awesome.

0:41 Olly Nice to meet you guys.

0:41 Scott here in the Seattle area. And somehow our paths have never crossed.

0:47 Olly Yeah, that's remarkable.

0:51 Scott It is remarkable.

0:52 Brad Ollie, tell us a little bit about yourself.

0:54 Olly I'm Ollie Downs. I'm Chief Technology Product and AI Officer at KiraNose, where we're focused on delivering decision intelligence for financial institutions. But I've had a wild and wonderful career in early stage businesses. I've done nine startups. It's all in the Seattle area and several of our major local enterprises, both Zillow Group and Zulily during the course of my career. Qumos acquired the banking focused assets of my last startup Amplearo that I founded back in 2016.

1:30 Brad Very

1:30 Scott Yeah,

1:31 Brad cool. Yeah, I

1:31 Scott I

1:31 Brad am with Scott. I'm like, how is it we've never ran into each other? But well, you know, I guess the answer is the Seattle tech community is actually pretty big. There's a pretty big community here.

1:45 Olly It is, it is. It's been fun. I feel like I've had a few years kind of out of it because Kuranos is a private equity owned business and we're, you know, in the middle of a major growth cycle. But we have, we've plugged back in actually by sponsoring the fintech incubator at Comotion Labs with the University of Washington. We really, we found that. Despite our sort of scale, we have more data than we know how to innovate with ourselves, and we provide just great access to the banking industry because we have almost a thousand North American financial institutions as our customers.

2:24 Brad Yeah, it's so interesting to me. I have not spent any time in fintech, and I hadn't spent any time in health tech. until a couple years back when scott and i worked together at 98.6 and i i've always as a as a as a person who's primarily like most of my background has been in in consumer focused applications you know i was early on at real networks so streaming media and and things like that but but the notion of these there's a couple of these industries like health tech and fintech where it's like there's there's there's a sense at least from the outside that the sort of

3:01 Olly like

3:01 Brad Like the regulatory hurdles or the compliance hurdles or the sort of just the barriers to entry are significant. And so it is interesting to think about how incubators can help in that space for entrepreneurs that are interested in getting into that space.

3:23 Olly Yeah, like getting access for product discovery to the industry. is one of the particular big hurdles if you're you know a set of founders trying to figure out you have an idea and you're trying to vet it and refine it it's very hard in financial services or health care in particular to just to get access to the right people to give you feedback which we found to be quite a significant enabler for our member companies it's been a lot of fun too

3:50 Scott I have a question for you about Karinos.

3:53 Olly Yeah.

3:54 Scott You said it's to help with financial decision-making. Now that could mean a lot of things.

4:00 Olly Yes. So the way to think about how we show up as Kiranos is we really, we got our start, our predecessor companies really created a very unique position in comparative analytics around deposit and lending pricing. And

4:21 Scott Hmm

4:21 Olly we uniquely have data agreements, data contribution agreements with banks that allows us to see the portfolio pricing that they have across their lines of business and all of their accounts by type. And we leverage that to help them design their product pricing, not just for profitability, but obviously for competitiveness. And then we take that, and this is sort of the enrichment of the decision intelligence, into how do they activate that product pricing? Historically, you might think of that as their rate sheet, right? How do they actually take that to market? Both to drive efficient acquisition, by which we mean not just customers who will open a new account with you, but actually sustain and engage a profitable relationship over an extended period of time.

5:12 Olly And how do you activate those products to deepen the relationship with the back book of customers that you have so that you grow that base over time and you increase the number of customers that you might consider having their primary banking relationship with you. with you oh

5:26 Scott That sounds like it all goes so smoothly.

5:29 Olly yeah well

5:29 Brad I

5:29 Olly you

5:30 Brad was going to say like that easier said than done, Ollie.

5:32 Olly we're old enough to remember what happened at Wells Fargo previously right and that's a situation where efficient acquisition or the objective of acquisition got out of control which was there was a lot of incentive to the Wells Fargo sales force to sign people up for new checking accounts but

5:51 Scott Mm-hmm.

5:51 Olly people who weren't even people were getting signed up for new checking accounts and of course opening new accounts itself doesn't beget any business value for the bank it largely incurs cost And so the question of how you navigate that intelligently, you know, is really difficult for most banks today has really been viewed as a human process. But of course, that Wells Fargo example sort of tells you, even if you've got a human process to help you with compliance and governance, you can still have the wrong objective function.

6:23 Scott Mm-hmm.

6:23 Olly And so as you think about how do you do this with AI, particularly as we begin to see, you know, consumer agents being able to shop for you, including analyze price and opportunity, right? How do you navigate that as a bank? How do you say, well, this is the type of customers that I want to offer this type of pricing to, even if that might be inefficient for me in the short term, because I believe they will graduate to be great customers for me in the long term. And so we're bringing the AI enablement. If you like, on the other side, on the bank side of the equation, in terms of how do they price efficiently, how do they respond to market dynamics, right, Fed interest rate changes, competition in areas,

7:10 Olly M&A happening. and continue to remain competitive and attractive to the right type of customers for how their business is positioned.

7:18 Brad Is this AI intelligence – so, Ali, I'd love to kind of double-click on this and understand what the next layer down is. Is this really about strategic decision support, or is this about actually operating the front door to whether or not the person can sign up for the new product? Or is it everything in between?

7:39 Olly It's really interesting because I would say that the genesis of our business was the strategic decision making and providing, if you like, the business intelligence with which you could go away and make some great decisions and then operationalize them. And we've moved with our Q&A one platform into the activation of those decisions at the individual household, individual customer. at even individual account level, not by directly taking the action, but by building a system that can orchestrate the enterprise systems that typically take those actions, whether those are digital banking platforms, whether those are the banking cores that set and manage the pricing and rates sheet in the product, whether that is enabling acquisition channels,

8:27 Olly whether that's the direct mailers that you get. get or whether it's the digital channels for acquisition. So we think of ourselves as providing now a system to model the strategy but also to operationalize that strategy through existing bank infrastructure.

8:44 Scott I have a very in the weeds technical question.

8:47 Olly I love it.

8:50 Scott Is this an on-prem solution or a cloud-hosted solution?

8:53 Olly Oh, oh, golly. No, we're very much a cloud-based solution. In fact, we're built on Databricks.

9:00 Scott Okay.

9:00 Olly I think, you know, Databricks in a unique way in the industry today brings, you know, petascale data compute very close to AI and machine learning operations done in a way that you can govern well. And so we're very proud that we're a built-on Databricks partner. And as you can imagine, many banks are going through significant data modernization initiatives today and really to unlock their data assets. And in many, we sort of find because of this strategic objective element to the decision intelligence we provide, we really, we tend to be connected with the business buyer in the bank that might be the head of retail, the head of deposits, the head of treasury, the CMO. But of course. russ in them to buy a technology platform.

9:46 Olly The technology buyers who are buying data bricks, they're used to, you know, doing a five-year master data management project, you know, back in the days of yore with sort of vague notion of what business value is. These two things allow us to connect that together. So if you're deploying data bricks, you can buy a Q1 business outcome for your business partners and deploy that very efficiently.

10:09 Scott Yeah, it's interesting you said these banks are kind of undergoing modernizations, because when I think of a bank, I think of cobalt and mainframes and old stuff. And I don't know if banks have an unfair reputation or if it's true.

10:27 Olly I would say that what's happening right now is such a remarkable wave of innovation that

10:33 Scott Mm-hmm.

10:33 Olly it's taken hold in the banks. I'll be honest with you. So I've been at Qunos about three and a half years. I was a little skeptical when I joined. We did a great deal for my startup Amplero actually with BECU back along. They were an early adopter of our capability. But I was very skeptical about focusing on the banking. industry. What we found over the last few years is two years ago at one of our customer forums we were asking the question about generative AI. Who was using generative AI day-to-day within the bank in their work? No hands went up. This last year it was nine out of ten. The 50 largest banks said they have about 160 AI use cases documented and in the process of prototyping or POCing over the last year. So this sort of tidal wave has gone much, much faster than, quote, digital transformation in banking,

11:29 Olly which I would say many banks would still say they're in probably 20 years plus on from when we would have thought it would have happened as technologists, particularly in the Seattle area. here are the Bay Area.

11:40 Scott Interesting.

11:41 Olly So there's a wave that's happening much faster than anything else in banking.

11:47 Brad Interesting. That is interesting.

11:48 Scott I was just curious to know where you saw in the banks this was happening, because one of the things we saw criticism of at least the generative AI when it first came out, which is like, you know, you'd ask it to add two plus two and it would say it's five, right? And what's the right word to use? Insufficient for a bank? Something like that. And so it's probably five. So I'm just wondering where are banks finding a use for it where things can be probable versus deterministic?

12:21 Olly There were the sort of normal things you would expect going on in the back office, whereas probabilistic decision making anyway happening, lending underwriting and those sorts of things as customer service and sort of probabilistic service response in our part of the universe and decision making and decision. decision orchestration part of the universe the way you can trade off regulatory compliance and the need for determinism with in smart algorithmic decision making is essentially by architecting systems where there are constraints a bounding box that gets set and particularly in banking right we have to think about disparate impact and so You shouldn't be promoting something to someone for which they're not eligible or they're not credit worthy,

13:09 Olly but you should be ensuring that everyone who is eligible and credit worthy has the opportunity to be engaged with that product. So if you think about those constraints as setting this sort of bounding box of rules. that you can create a very nice human in the loop process around and the auditability observability of the system that those rules are being observed then within that bounding box who you engage over what channel about that product is the thing that you can play with to maximize engagement from the customer and also profitability for the bank so we've sort of built systems where we say okay Here's how you control this and set the boundary box and then within this here's how you have the provable decision intelligence.

13:56 Olly But even with that algorithmic reasoning what we've been able to do is structure okay well every interaction opportunity that is considered any particular experience for which the customer is eligible to receive or be engaged with by the banker if we if we play up through the CRM is written down. The context of the individual at the time that the interaction decision was made and that set of possible decisions are written down. And then the decision that was chosen and why it was chosen relative to the business outcome the system is pursuing from a strategic perspective, all those are completely auditable. So while there may be a stochastic decision made within the set of eligibilities, how that decision was made is written down and recorded in detail and you can refer to it individually.

14:46 Brad Yeah, this reminds me very much of the type of work that... I think is happening now in health tech where and and this is like you know Scott and I worked together at 98.6 and this was that was kind of before the the the sort of certainly the agentic LLMs were were practicable but nowadays it's like well if we build our if we build like we can build deterministic systems that sit in those those tools and

15:15 Scott Right

15:15 Brad we can leave the The sort of like. Instead, we don't have to worry about the LLM saying 2 plus 2 equals 5. If the LLM needs to do addition, it can call the deterministic tool that is guaranteed to correctly do

15:28 Olly Exactly,

15:28 Brad the addition.

15:29 Olly exactly.

15:29 Brad But what we're leaning on the LLMs for is the, oh, at this point, I need to do addition, right? Like that is what it's actually good at is sort of like knowing when and what tools are needed or predicting like which tool is the most appropriate tool to use it. Use at this point, but then let's leave the actual like determinism to deterministic tools

15:51 Olly And that's very much how we've architected our agentic workflow is essentially the skills essentially required to make to perform the mathematical reasoning are the same skills that we take through the bank's conventional model risk management process so they can know how that algorithm is going to make a decision. And then how you interact with that algorithm, both how you drive it, right, as a user in the workflow, and how you receive and consolidate information from it can be, you know, probabilistically driven, LLM driven, but we've taken that same approach, which essentially we have skills that conform to. algorithmic determinism in ways that can be easily understood by scientists.

16:37 Brad This was kind of the direction I was going to pull us in anyway. You were talking about how you've gone from the strategic to the tactical and that and it sounded like and so this will be also a little bit in the weeds. But I'm kind of curious about what. I mean, when you talk about like integrating with existing banking systems like this just feels like is this like a. I don't know. How many of these custom connectors are you building or is there some kind of standardization happening? You know, like in the tech space, I mean, in the health tech space, we have like fire based, you know, like there's standardization around the protocols which EHRs talk now, you know, and so every EHR is going to support the standard protocol. But is that is that happening in the banking space or is it are you constantly building little custom connectors and things like that?

17:27 Brad Things like that.

17:27 Olly Sometimes we put in custom connectors, but we're beginning to formalize around ODCS, you know, YAML-based data contracts, both on the inbound side and the outbound side. I would say for us on the outbound side, the systems that we're orchestrating typically are... More modern, or at least recent technology era components, the systems surfacing some of our inbound data can be legacy core banking systems, COBOL coded and things like that. But we tend to be able to sit one step up, usually in the system where analytics is being done, which still may not be a modern big data system, but it's a system that has, you know, supports APIs. or supports interactive querying where possible.

18:14 Olly But we've been working hard on ODC-S data contracts that are bidirectional with our customers. And then with Databricks, as that sort of very recent wave of modernization is happening, we've been working on developing genie skills, Databricks genie skills that actually can drive a lot of the data mapping from that core system and it's sort of ugly. of ugly metadata into a schema that will directly match the drivers of context for our Kyronos 1 platform.

18:44 Scott Can I take a detour here for a second, Brad?

18:46 Brad Yeah, please.

18:47 Scott Ollie, but

18:48 Brad It's your podcast too, Scott.

18:52 Scott you're the host today. You're the host today. Databricks. I just know a little bit about Databricks. Could you be a marketing person for Databricks for a second? Tell us in our audience, it sounds like you guys are really getting a ton of use out of this platform, Databricks platform. And I'm curious to know, I know the highest level of what it is. Yes, but that's really it. So I'd be curious to learn about it.

19:14 Olly You know, Databricks started as Enterprise Spark, you know, if you

19:19 Scott think Okay.

19:19 Olly sort of think of the adding an enterprise layer onto open source project.

19:25 Scott Mm

19:25 Olly sort

19:25 Scott -hmm.

19:26 Olly of state, but I would say has evolved very rapidly now beyond that. You get lots of the great memory management and scalable data compute that you want from Spark, but it's actually very hard to deal with with open source Spark. And then what they've built is a machine learning operations stack originally, essentially open source MLOps flow. brought that very very close to the data and combined with their unity catalog which is their view of sort of object management in the system and it allows you to really nicely govern objects which of course is critical for industry like we operate in banking and those objects can both models whether those are LLM models reinforcement learning models

20:17 Olly Predictive models, basic machine learning methods, and whether those objects are data assets, identities in some cases, right, sensitive. Third-party demographic information allows us to really control access there too while we're doing very very scalable compute and the platform over the past couple of years has also moved to fully embrace serverless so not even really provisioning a defined infrastructure anymore which has dramatically boosted the cost-effectiveness but also the performance of compute on Databricks. For us we've modernized our own platform from legacy Hadoop, which I

20:56 Brad Mm-hmm.

20:57 Olly would say I loved Hadoop. It's the first system I've architected that has actually scaled for me through nine orders of magnitude. I always think about, you know, you need to re-architect the whole thing every two or three orders of magnitude. Hadoop with Amplera, we went from millions of decisions a month to tens of trillions.

21:17 Scott Wow.

21:17 Olly over the course of 18 months on the same architecture. But we found moving to Databricks, we actually reduced our compute times by about 90% and our cost by about 70%. So again, it's sort of pre-provisioned Hadoop clusters. So for us, it's been a great efficiency gain and it's been a great performance gain. And then at the same time, it has woven into it the ability to manage and audit. data quality. So Hmm we're able to have good checks and balances in place throughout our data pipelining which is so critical for us in this vertical along the way and be able to be very very proactive when it comes to finding problems with customer data way before decisions are made downstream of that data element sort of coming into play.

22:10 Olly Okay, so now I have a follow-up question. This is cool. I have a follow-up question, which is I've spent a lot of time in my career thinking about how organizations are designed, you know, org structures people, but it's all been around about producing for most of my career consumer internet software apps. What's the structure of your org built or how is that organized? around this is a totally different kind of operation that you're doing compared to my experience i'm curious to know how how that is organized because to me i think oh that's the data team over there doing stuff but clearly you got a huge thing going on and how do you organize that We we've taken very much a kind of a platform approach to our business and to the underpinnings of our products, essentially. So we obviously all of our products are data driven and enabled.

22:59 Olly And Mm-hmm. so we think of our core data platform, our own lake house as the basis of how data flows in the organization and how we compute with data both for sort of our own normalization, but then also for each. for each of our products and you know we're we're a legacy business in many ways and we have a number of highly sub-verticalized products and when I started actually as Kira and O's CTO I viewed this as kind of like a double Conway's law problem in the sense that the banks were siloed Right. They have all these subdivisions with their own systems and data and decision making processes. And to serve them well, we had become siloed. Our products were siloed and how data flowed in those products itself was siloed and our technology,

23:49 Olly our engineering team was siloed to those products. Silos And so, all the way yeah, down. so we totally we totally disrupted that we've disrupted ourselves. And for the banks, we've really talked about it as helping them with their coordination friction when it comes to this decision-making process. But in terms of organising ourselves, we've really thought about it as we have three key platforms. We have our core data platform in Lakehouse. We have our AI platform, which is both our. proprietary reinforcement learning technology our core machine learning and how we manage and operationalize our generative AI capabilities and then thirdly we have a application and API interfaces

24:34 Olly platform team and between the three of them they essentially provide the tooling that enables our product suite end-to-end yes And so is the right way to think about that as like layers, you've got the lake house layer at the bottom, you've got the sort of API and products at the top and you've got the AI sort of decision making in the middle. So three. is that like a sandwich in between if you like yeah yeah Yeah. Interesting. One of the things that I'm very curious to hear about, we already talked a little bit about the notion that when you need determinism, you use your tooling that's deterministic and then you can layer the sort of probabilistic machine in kind of around in a middle and orchestrating that. I'm really curious to hear about what challenges and what solutions you've applied to those challenges for correctness.

25:24 Olly I'm assuming that correctness, again, I'm sort of like mapping this onto health tech in my mind. Mm-hmm, mm-hmm. So one of the challenges, so I'm just kind of curious to hear about like, how are you handling like correctness? How are you handling like evaluations? How are you handling, especially since you've got like this large data set problem? And then you maybe have like business rules that have been captured from the humans that you're trying to automate. And then you've got the probabilistic machine somehow driving a lot of these things. So talk to us about validation and correctness. Yes, we have a few layers of things going on for sure. From an AI and ML observability perspective,

26:10 Olly we actually work with with Fiddler AI as our AI observability platform. They allow us to keep track of across all of our algorithmic models drift both input drift in our features input features to those models drift in terms of semantic input conversation or input drift for our LLM interactions and then output drift. Then we continually track eval output from our customer success and technical success teams who are continually interacting with those experiences themselves both for internal purposes and for the customer so we're able to track pretty granular eval cases that give us an ongoing sort of our own drift on these are the things customers are interested in these are the things they're asking our AI about or querying our AI about and this is how we're interacting with those things so we capture a great set of examples there along the way too which allows us to sort of test correctness and also ability to serve because you can imagine the way we sort of biases our guard railing is set up very much to

27:19 Olly narrowly constrain the applications of our agent interactions and if you deviate from them and particularly because you know we we operate this sort of comparative analytics element you know there are types of competitive or anti-competitive questions that the curious customer bank may well want to ask but they certainly cannot ask of the data asset that we have so we very narrowly constrain our guard railing in terms of the topics of conversation that can happen happen and stay very closely on top of that. Then every time we refine our decisioning algorithms which happens daily, we have an evaluation that takes place almost like a champion challenger. Would we expect given this update that the type of decision making we would now make today to materially change and if so what are the drivers of that?

28:10 Olly And that allows us actually to introspect every day. every day? Is there some trending set of driving changes that are happening such that we might expect our decisions to be dramatically different in a way that would polarize them to a particular product set or polarize them to a particular audience that we might not desire or might not be fair? So we're able to test that very frequently. Interesting. We also have to prove that we do that part of our relationship with the banks is we can prove and they have dashboards so they can actually see that those tests are happening and the same type of outputs of those evaluations. Yeah. Yeah. Is there is that stuff that do you so you use Fiddler AI is.

28:55 Olly Is this stuff that you've had to, to, to be able to create that proof, that provenance, that, that, that demonstrable evidence that you do this testing, is this stuff that you've had to basically build yourself or, Yeah or do you see anybody like how, what is, what's happening in this space around, you know, products for solving these types of problems? This is a big challenge, I would say. We've sort of continually, I think we've all walked up the line of this in our careers, you know, particularly since the, you know, explosion of open source capabilities 15 years ago, 10 years ago, which is, you know, something's coming out to solve the problem that you're already solving internally and you're always making this build versus buy or subsume sort of trade off. I would say in this.

29:45 Olly In this area, there are some elements of some of our algorithms that are quite unique and we've had to build our own observability capabilities for them and then others where they sort of live in this common space, either in the predictive modelling sort of common space of your or in the LLM semantic topic management sort of space for which tools are evolving in both of those areas very quickly. We've sort of viewed Fiddler as being best of breed in the former and then really moving quickly into the latter. But it's a continually changing space and data bricks themselves are innovating a lot in observability, both from a data perspective, but also from a model perspective. And the fact that they have ML flow in the mix and Unity catalog, it allows you to get confidence around what the assets are.

30:34 Olly And then ML flow already mad. helps you manage model experiments and as a result you can you know start introspecting those experiments as a mechanism of observability too so everyone's moving in a in the direction and so we're continuously evaluating what is the right thing to do I would say that the place that the space that observability move very rapidly in the other vendor we work in with for Distributed observability is Monte Carlo data and they have really good AI enabled proactive thresholding and monitoring and those sorts of things. Everyone's moving into this coding observability space, which is, you know, far upstream of operating the product. It's in this world of building the product and making it adhering to your SDLC while getting the benefits of AI scale.

31:26 Olly That for us has been very secondary and there's been a lot more pressure on sort of the provable governance of our products themselves as they meet their customer. And I think probably for us next year, it's going to be very much about next wave agent driven coding and how do we govern that ourselves as we evolve the products, but not right now. That's interesting. You went there. Yeah, I just got that. Are you, are you, so, so wait, so how is your team building this software? Are you, are you, are you, do you have a dark factory or it's all still human in the loop? Are you still reading all the code you write? How, how is this working? Where do you live on this spectrum? We are human in the loop. We are, I would consider us to be sort of first gen AI coding adoption.

32:15 Olly Which is we've got the coding acceleration, but not the agile process acceleration both to feed the beast and to make sure that really what's coming out of the beast, you know, the code review can be really done efficiently in a way that honors our. compliant sdlc for this industry Yeah. and so we're right at that point where actually had it was it a webinar today actually about ai dlc and and sort of the ways to to fully embrace that and so that's why i was saying actually you know we're going to tee this up for next year but we've hit that bottleneck which is we we haven't found the right way yet to to ensure that our code is fully compliant if we don't do our own code reviews right we can take suggestions and recommendations on those things but we're still there

33:03 Olly Yeah, it's so interesting that you say that. I'm fascinated by that, especially with highly governed industries like fintech and health tech. I mean, I think this is one of the most foremost challenges of CTOs right now. now, which is especially in this space where you have to know that you've written your code in a compliant way, the pressure to move faster is significant. Yeah, there's a lot of pressure to move faster. Yeah, I sort of view it like a traffic jam when it's just beginning to clear. The traffic like speeds up and then it slows back down again and it speeds up and it slows back down again. I feel like we're in that sort of phase, which is, you know, either it's a code review bottleneck or it's actually a requirements and really good PRD and inbound documentation sort of bottleneck

33:57 Olly Mm-hmm. and we sort of oscillate between. in between those at the moment. Yeah, it's like all of the guys on our teams, sorry for the gender specificness, but all the people on our teams who had two arms and could swing two hammers now are octopi and they have eight arms and they can swing eight hammers. But we don't have more architects, not software architects, I'm using this building analogy, and we don't have more building inspectors. Right. Yes. And to a lot of our stakeholders, it's the octopi with six more limbs that's very exciting and investing in the building inspectors and the architects is not as exciting. Yeah, not as exciting. And yeah,

34:42 Olly I would say some of the best accelerations in our... The loss of our velocity this year and our sprint predictability have really come just from a real intentionality around end-to-end agile process and in particular what gets into the top of our sprint grooming funnel and how ready is it and well-defined it's like good old-fashioned like definition of what you're really trying to deliver still matters and I think it just this just compounds the need for that to be so clear because you can produce the code for anything. Mm hmm. Right. It's yeah, it will do exactly what you tell it to do. And so you've got to be really clear about what you're telling it to do. Yeah. I am curious. It is interesting to me. Is the ability to be really good at that top of the funnel,

35:32 Olly I know what I want, I've described it really well, how much industry variance exists, right? Do you feel like in consumer software, it's just like... whatever random idea the product manager has right and some product managers may have more or less effort they've put into thinking it through right like i think scott you and i've you know talked before about like how the challenges of getting people to plan you know like right but then in in spaces like this fintech or health tech it's like You know, you've got like externalities that are sort of like going to demand that you actually know what the F you said you were going to build.

36:19 Olly It's definitely very different, you know, when I think back to my time at Zillow and at Zooli, you know, we had very high throughput, just like Amazon does, very high throughput experimentation capabilities. And, you know, when you've got millions of consumers you're interacting with, you get feedback on whether something was a bad idea very, very quickly. But you don't get that in the same way in these governed industries and in businesses even that aren't regulated in that way. But just where the decision making right in B2B, the decision making pace is slower. So even in the bank to consumer space, like the pace at which I might make a decision to open a new type of savings account or move my money to a new. primary banking provider for me, right?

37:06 Olly I might make that decision over the course of six to 12 months. And the reason for me making that decision probably emerged over the past several years. And so being able to plug into how those decisions get made, the context of them, get them, make sure you surface the right signals in a reliable way. You can't test five iterations of that very quickly. Mm And -hmm. when you deploy it to even test it, it has to be really quite reliable because of the type of decisions that you're trying to influence. So yeah, very different timescales and pace at which you can learn. Yeah. Yeah, I also I think we've I've also been in industries where it's like, your customers just doesn't want to consume the innovation that quickly, right? Like this, this, like, they don't, they're not interested in like the new UI that you supposedly built that's going to make their job that much more efficient, because in their mind, it's like, I got to retrain everybody,

37:59 Olly you move the buttons from the left side of the screen to the right side of the screen. I don't care if you think that's more ergonomically correct. All my documentation, all my training is at it. He's out of date. Yeah, very much so. And, you know, part of our Q&A1 solution compound, and this was important to us, it actually bridges between the digital customer experience and the branch customer experience. And if you sort of think outside of the top 10, 20 banks into the... smaller national super regional and regional banks right how they show up in that with their branches is very important to them and we we actually we we had a great design partner for our for our crm plug-in and they like you know we really don't want you to help us craft our lead dialogues for our bankers we just just want you to surface the signals and so we launched the we launched the pilot they had to educate there

38:52 Olly you know a pilot group of bankers and literally end of the first week of them testing these acting on these signals they're like well actually we want the we want the dialogue recommendations and we want the justification insights because we're actually really curious and these signals we're acting on have really been very effective but we actually we want to train ourselves into this but there is this whole our change management process it's it's training real people because you have some human in the loop even you've got this very scaled decisioning over digital channels right you're still fitting into a workflow to meet the compliance constraints and so that change management almost is a bigger piece in these types of enterprise I think of that in the medical billing sort of space too and the eligibility space for sure very very similar

39:43 Olly Yeah, and many, many fields that gets overlooked that building the software is kind of the easy part and it's getting and it's You're watching language there. Watch your language, Scott. well, I'm just going to go out. then hear and

40:00 Brad Yeah.

40:00 Olly say that getting people to change their behavior is harder than writing software yeah

40:06 Brad Yeah. Okay, fine. I'll give you that. It's a lot easier to change the behavior of the software than it is to change the behavior of the humans using the software.

40:15 Scott yes yes it really is

40:17 Olly or or in the case of a business situation teach the humans how to use the new software because

40:24 Scott Mm-hmm.

40:25 Olly people learn i had a fascinating interaction this week i went and uh met with somebody who is um their company is still using a dos based inventory management and point of sale system now they're using that because it has some special features in it that are unique to their industry and nobody else has made a thing and there's but they recognize that they need to like modernize but it was fascinating I mean, it's a whole language of its own, figuratively speaking, how to use this thing. I mean, a simple example would be like, you know, they were showing me how you create an invoice. And then I was like, well, how do you do a refund? And he was like, well, you just type an R in this field and then it knows to do a refund. And so they had this system for 30 years and they've trained all their people on it and it's a whole world of its own and the idea and they're very fluid with it and it all just works,

41:18 Olly right? right and when I say it all just works I mean like they got to train their people but once they know how to use it like they can make it do everything they need it to do and it was fascinating to think about having to change that for them like if they brought in a new point of sale system like I'm sure they could retrain their people and make it all work out but like that's that's a big project just the transition

41:39 Scott We definitely found that, you know, in some cases we very much. scale we take we add a lot of scale of decisions that can be taken right and the pace at which they can be made but also added we've sort of replaced in the workflow quite a lot of human burden because we sort of create a combinatoric explosion of the content that can go into the decisions that get made over different channels and get operationalized and so because there's this need for human in the in the loop we've actually found in some cases the work goes up it's just a different emphasis of what work has to has to get done Whereas,

42:21 Olly Mm

42:21 Scott you

42:21 Olly -hmm.

42:22 Scott know, banks managing. quarterly promotions that are going to be on the side of the bus and in the window of the branch and that the banker will talk to you about right there was a lot of human mechanics that went into the creation of all that content and then the education now it's well there are tens hundreds or thousands of possible ways that we might drive this in this engagement the promotional opportunities are equally as dynamic and responsive to the competitive how does an environment and their fed and so on and so they're flowing more continuously and so that process itself is is happening all the time whereas it used to be there's an episodic sort of quarterly work cycle that happens

43:03 Olly In some ways, there's a similarity there to how our business as software engineers has changed. I just had in the last week or two someone at our company. suggests that we switch to monthly or delivering either monthly or quarterly like we just do continuous deployment we're always rolling stuff out right and they they're like maybe we should just kind of like have a monthly release or a quarterly release and then we can always

43:30 Brad Where did

43:30 Olly focus that not

43:31 Brad come from? Did that come from what part of the org did that come from?

43:35 Olly the engineering part of the org

43:37 Brad But they must have had a reason that they thought that was a good idea. What was the impetus

43:40 Olly Yeah. Oh,

43:41 Brad for

43:41 Olly well,

43:42 Brad that?

43:42 Olly we were looking at the on schedule accuracy of delivery and there was the idea in that because things are continuous, it just doesn't seem like that big of a deal to something just like takes a little longer. And this person was thinking, well, if we just had these periodic releases, then kind of everybody could rally around that and like really get the thing done and get it. than the release and then ship it and then go on to the next

44:06 Brad interesting

44:06 Olly one yeah

44:06 Brad like a sort of like a forced of sort of a deadline to help force attention because

44:12 Olly that's what if it or

44:13 Brad you miss it you're going to miss it by a month or

44:15 Olly a quarter yeah yeah

44:17 Brad a quarter

44:17 Olly right And my response was like, I mean, this is almost exactly what I said. I said, well, we could do that, but that's not how anybody really does it anymore.

44:28 Brad Right. Right.

44:30 Scott it's a really it's an interesting thing i mean in our industry you know with with our customers there really is one annual major event and that definitely does it For some of our releases, it definitely fixes that vertex of the triangle of constraints for us, which is, you know what,

44:48 Olly Mm

44:48 Scott if

44:48 Olly -hmm.

44:48 Scott we can't show it, if we can't show it and talk about it then, we really can't show it and talk about it in a way that scales like that for a year. And so we do have these things where it's like, okay. there's a major milestone we want to hit but at the same time we've are in our culture we've just allowed us to embrace agile there by saying well we're going to let the scope be flexible but I do sort of get this idea that many other functions do need to plan around some of these things because they are much more human driven or they they hit the human part of how your product gets sold or enabled yeah

45:19 Olly Yeah, they just need to make their plans.

45:23 Brad Yeah.

45:23 Olly It did cause me to reflect upon something, the continuous deployment versus what we call the release, it's not really the big release deployment. And this is the one piece that. it caused me to reflect on there's a certain like celebration that we lose with the continuous deployment it's like just a grind it's like you're just hiking up it's like sisyphus right you're just hiking up the hill forever and you never get the moment of uh release and no pun intended but sort of moment of release and joy when you get to the top of the hill and be like oh we made it right and now we gotta walk back down to against us like we're just trudging along um infinitely and i think this kind of connects back to what we talked about with camp back a little while ago brad like it it connect it we lost a little bit of that sort of like human part of doing this job and we lost those celebrations of i don't know if either of you ever worked at microsoft but you have to ship it the

46:17 Brad Yeah.

46:18 Olly ship it uh it's not a trophy

46:20 Brad You get the swag.

46:20 Olly what do you call

46:21 Brad You

46:21 Olly it get you

46:21 Brad the swag.

46:21 Olly get the shipping swag right

46:23 Brad Yeah,

46:23 Olly there

46:23 Brad there you go.

46:24 Olly i can't see it it's a little blurry but yeah exactly

46:26 Scott It

46:26 Olly right

46:26 Scott is.

46:28 Olly We kind of lost a little human element when we started continuously deploying.

46:34 Brad Yeah.

46:34 Olly I don't know if it's worth trying to recover, actually. Like, I guess we gave it up in the name of like delivering better software and delivering faster, but we lost something for ourselves as people in the joy of doing the work.

46:47 Scott There's definitely a little bit of that. I found in enterprise software, what we've pivoted that into is major... major customer contracts now

46:56 Olly Mm-hmm.

46:56 Scott with a new capability has emerged you couldn't quite put your finger on the moment when it emerged because of the continuous employment and delivery and those sorts of things but there was this moment where it was recognized and so for us in enterprise software we've sort of we sort of viewed it as wow we really We have valid external validation of product market fit and someone's written the check behind it and we've celebrated that milestone as the alternative, I would say, to the shipping.

47:23 Olly Yeah. Or that first, you know, when that integration launches is sort of another big milestone to go for as well. We've had that recently at Tassler. We have a partner business where we provide services to enterprises like Ikea and Wayfair and Best Buy and things like that. And, you know, when that first order comes through, it's not very much. It's not a big business yet, but it's just the beginning and that's an exciting moment.

47:48 Brad I would add to those types of rituals and those like celebrations, like the more that you can get the engineering team to feel connected. to the business problem then we know engineers engineers have higher satisfaction level when they really feel connected to the customer problem that they're solving right like that like that's that's we all know that's true and if anything like the oh we just did the big release that was very much like us with our propeller hats and um just being the nerds and like yes we did the thing here's the here's the release but it really only like if we like maybe connecting just to the business outcomes is the better way like ring the bell when you get your first you know million user day or whatever that would be you know to to and and if you can maybe that's that's how i that's how i want to frame it i want to frame

48:45 Brad I want to frame it as like, get the engineers really feeling connected to the business and then make those celebrations be the business outcomes.

48:56 Scott Yeah,

48:56 Olly Okay,

48:57 Scott absolutely.

48:57 Olly I want to... I want to propose a variation on that because we've done something like that at TaskRabbit. We sort of really try to talk about tasker earnings. So the people who work on TaskRabbit we refer to as taskers. And a few years ago we were talking about revenue. That's how much the company makes. And I was like, why are we talking about revenue? Why don't we talk about tasker earnings? That's the impact we're making with people. It's just the other side of the same coin. But it's still a little bit hollow. So my proposal, and I might try this out. The side is a little bit different, and it's really I'm going to have to bring this to our product management team, which is when you come with a project. You better come with some, not you better, I don't want to be threatening about it, but please,

49:35 Brad You better you better your bet

49:37 Olly please come with some videos of customer interviews that you did to understand this problem. So let's start at the beginning with the customer connection. You know, you've got a problem in the pipeline and because you've learned from talking to your customers that this doesn't work very well. Well, let's start there. As opposed to starting with the sort of celebrating the end, let's like really get people connected at the beginning so that. um you know they understand what they're doing it for that there's meaning behind it it all comes back to dan pink right purpose autonomy and mastery and you got to bring that purpose there

50:17 Scott It's really interesting when we were beginning to build Kiranice One, we were sort of thinking about building capabilities and our product management team really did what has been one of the best product discovery, customer discovery processes I've been a part of, I would say. They really did a very robust. process with our customers what our customers told us and remember i told you the conversation our primary buyers are not technologists secondary buyer for us they didn't tell us a single thing about capabilities they did they actually what emerged were five business outcomes that our primary buyers were looking for efficient acquisition relationship growth basically imbalances primacy which is that they're trying to drive that your bank your consumer relationship with that bank is your primary banking relationship customer and deposit balance retention and profitability

51:16 Scott And so it caused us to transform actually how we built Kieran Ice One and also how we're taking it to market, which is what you buy from us actually is a business outcome.

51:26 Olly Mm-hmm.

51:26 Scott The platform itself in this closed loop is instrumented to attributably measure progress on that outcome. And so it becomes self-business casing from a buyer perspective, but also for us, it becomes self-business casing from a did we make the right investment. perspective in how the technology works but we really now sell these outcomes not capabilities even though there are Yeah. sort of a set of core capabilities that underpin it yes

51:52 Olly Right. Because if you could get those outcomes with smoke signals or bubble gum, it doesn't matter. People want those outcomes.

51:58 Brad Right. Right. And those outcomes are the thing that you can celebrate above and beyond the just the release like. Yeah, like, look at that team. We achieved this outcome for

52:11 Olly Yeah,

52:11 Brad this customer.

52:12 Olly we did that.

52:12 Brad We

52:12 Olly We did did that.

52:13 Brad that. Yeah. I love it. Oli, this has been a wild conversation. We went from large-scale data analysis and strategy down to what actually connects with the developers to deliver the business outcomes. And we didn't even talk about your 1990s era paper on generative AI. We're going to apparently have to get you back to... to give us the full history of generative AI. But it was great talking to you and thank you for coming on the podcast.

52:48 Scott Guys, it's been a

52:49 Olly Yes.

52:49 Scott real pleasure and next time we'll do it over a beer somewhere in Seattle for sure.

52:53 Brad That sounds like a good time. That sounds like a good time. All right. Well, Scott, what do we say to all of our listeners out there?

52:59 Olly Tell your mom.

53:00 Brad Tell your mom. Rate and review. Smash that subscribe button. And we'll see you next week on another episode.

53:08 Olly Bye, everyone.

53:08 Brad This episode of the How Many CTOs Does It Take podcast was produced and edited by Saffron Hepta-Cal. Our theme song was AI generated by Brad. Please follow us on all the social media platforms at HowManyCTOsPod. You can find us on YouTube, LinkedIn. Instagram, Facebook, Threads, X, TikTok, and Patreon, or at our website at howmanyctospod.com. And remember to rate, review, and subscribe on the podcast platform of your choice. Thanks for listening!