Transcript: Adapt or Fade: Interviewing for Developers in the Age of AI Assisted Coding
0:00 Brad I want you to operate at the level of understanding and designing systems.
0:03 Brad And so I want you to use AI.
0:24 Brad Welcome to another episode of the How Many CTOs Does It Take podcast.
0:27 Brad I'm Brad Hefta-Gaub, and this is my co-host.
0:29 Scott Scott, all right.
0:31 Brad All right, Scott,
0:32 Brad welcome back.
0:33 Brad What do you want to talk about this week?
0:35 Scott Brad,
0:36 Scott only our loyal listeners might remember.
0:39 Scott A recent episode I talked about how I had been an interview,
0:44 Scott I've been interviewing somebody who was wearing a baseball cap.
0:47 Brad Yeah, I felt seen or being called out.
0:50 Scott It really threw me for a loop. And this is not about workplace fashion any longer. This is actually about a more modern problem,
0:58 Scott which is interviewing in the age.
1:04 Scott of AI-assisted coding.
1:06 Brad Yeah.
1:06 Scott And it actually takes on two dimensions.
1:08 Scott There's interviewing in person,
1:10 Scott and there's interviewing remotely.
1:11 Scott And even a lot of in-person companies now are still interviewing remotely because it is much more efficient.
1:19 Scott I'm curious to discuss with you best practices, so to speak, if there are best practices yet,
1:26 Scott for how to interview in an AI.
1:30 Scott assisted coding era and i think it's two parts of it one is interviewing for the ai skills meaning do you know how to use the tools and
1:40 Brad Yeah.
1:40 Scott the other part is well if somebody knows how to use the tools and get the ai to do it for them do they actually know the fundamentals right and so i'm just curious to know if how you're doing it these days if you're doing it or how you would hypothesize to do it if you're not doing
1:57 Brad Well,
1:58 Brad this I love this topic. This is a great topic.
2:00 Brad I think that, yeah, there's lots to lots to kind of dig in here.
2:04 Brad And I don't know that we can cover everything in a single podcast episode,
2:08 Brad but I think it's worth starting the conversation.
2:10 Brad Let me start by saying that I always thought that like leap code style interview questions were not a very constructive and useful interview technique.
2:21 Brad And I think that.
2:23 Brad One of the things about like the age of AI is that like that just like a remote, you know, remote call interviews where you ask lead code questions are just going to be a waste of time because every person would be able to just have their, you know, HUD overlay of heads up display overlay of,
2:43 Brad you know, the AI chatbot helping answer your lead code questions. In fact, there's like a.
2:48 Brad like a company that's like an Andreessen Horowitz funded startup of some engineering students from Columbia or something like that that got kicked out of Columbia because they they built a thing I think it's called Cluzi or something like that it's AI assisted lead code programming thing
3:04 Scott And I know that we have a few non-technical listeners to this podcast.
3:08 Scott So what is lead code?
3:10 Brad Yeah, elite code is basically like you stump you stump the interviewer stumps the interviewee by asking to perform,
3:18 Brad you know, coding gymnastics,
3:20 Brad you know, like a classic,
3:22 Brad a simple classic example is please implement in.
3:28 Brad the language of your choice, or maybe I give you specifically the language of Python,
3:31 Brad you know, or the language of JavaScript reverse a string in fastest possible time or something
3:38 Scott So
3:39 Brad like that.
3:39 Scott just some coding trick.
3:40 Brad Coding tricks.
3:41 Brad There's like leetcode.com or something like that has, you know, hundreds of leetcode tests and like score your test is like performance and memory usage.
3:53 Brad et cetera,
3:53 Brad right?
3:56 Scott So what do you think is a good interview?
3:58 Brad Well,
3:59 Brad I mean,
4:00 Brad my...
4:01 Brad My preferred interview technique is to get somebody to talk in depth technically about a project that they specifically worked on.
4:11 Brad Right.
4:12 Scott Mm-hmm.
4:12 Brad You know, I mean, I've been writing code for more than 40 years.
4:15 Brad So it's like I you can you'd better be able to explain any project you ever worked on to me in a way that you can make it clear to me what you actually did, what the architecture actually was.
4:26 Brad what the challenging problems of it were,
4:28 Brad what your technical choices were,
4:31 Brad and we ought to be able to have a conversation there that gives me a lot of information about what you actually know how to do as an engineer.
4:39 Brad That's how I typically interview people.
4:40 Scott In an AI coding world,
4:42 Scott that would get around one of the topics, which is, do people really know what they're talking about?
4:47 Scott If they had the AI robot write all the code,
4:49 Scott if they could then just describe what was happening there to you, then they would.
4:53 Scott That would illustrate that.
4:54 Brad Yeah, exactly.
4:55 Brad I'm a huge believer in like,
4:57 Brad show me your GitHub account,
4:59 Brad right?
4:59 Scott Yeah.
4:59 Brad And even if you only worked on private projects,
5:02 Brad right?
5:03 Brad Like your GitHub view still shows your activities across private projects and public projects,
5:09 Brad you know?
5:10 Brad And so, you know, okay,
5:12 Brad wow,
5:12 Brad this person has 78% commits,
5:15 Brad you know, of, you know, this big, huge green bar of work they've been doing.
5:20 Brad been doing on all these private projects you know now i know you're you're doing a bunch okay cool tell me about i don't want you to disclose anything that's confidential but let's
5:27 Scott Mm
5:27 Brad talk
5:27 Scott -hmm.
5:27 Brad about a pick a project pick a project you want to talk to me about and tell me about it and yeah to your point like you have to be able to understand the technology you'd be able to to handle that interview question um it can go a lot of different directions i find that it helps it should help people kind of like
5:45 Brad like get away from their nervousness like i mean some people are going to be naturally nervous but but like i don't care if you i'm not going to ding you because you're doing some lead code programming thing and you put a semicolon in the wrong place like compilers tell us that i don't need you to be a compiler compilers do that i want you to operate at the level of of understanding and designing systems and so
6:08 Brad I want you to use AI.
6:10 Scott Yeah, now what's your opinion?
6:13 Scott I think in an AI-assisted world,
6:15 Scott my opinion on this is changing.
6:16 Scott What is your opinion on the sample coding project?
6:19 Brad I mean,
6:20 Brad I think there's value in that still.
6:22 Brad I think that there certainly was a time like you and I were at a health tech company together and that was a big thing that they had. They kind of sent off people with go do the sample coding project.
6:34 Brad There's pros and cons to that. Like in a in a in a buyer's market,
6:39 Brad then.
6:39 Brad right where you're where you're you know or a hirer's market an employer market where employees are looking for work they're more willing to sort of like jump through hoops to prove that that they're worth hiring um in a in an employee sided market where there's a lot of um demand for employees i think that that kind of like do a coding project or i think you lose good candidates because they're just like i'm
7:06 Brad i'm i don't have time to write that code for you so to me i feel like it's like it can give you good information but it can cut both ways like sometimes it kind of gets in you it gets in your way
7:20 Scott Yeah, I was for a long, long time a big fan of the.
7:24 Scott the coding project um because i felt and i would say this even to the recruiters because the recruiters would always say oh this is really slow and step down we're losing candidates and i would sort of say look you could take away all the interviews and just give if i have the choice between no interviews and only coding project or interviews only and no coding project i take the coding project because i want to just see what somebody can do like i don't want to hear what they can do i want to see what they can do
7:51 Scott um and i sound that it was not the case that somebody who did a good coding project always got hired but it was almost always the case that when there was wasn't a great coding project and then we interviewed someone we ended up not wanting to hire them it
8:06 Brad Right.
8:07 Scott was so it was a really good signal um but it was a burden for people and i think that became an impediment but i do think that where my mind is changing is now in an ai coding world i mean
8:18 Scott We would basically would send someone a spec and say code this up and now you can just give it to a cursor and say, well, this company wants this thing. Can you make this for them?
8:27 Brad Well, but I, okay,
8:28 Brad so 100% agree with you on that. I have a couple of reactions to that. The first one is,
8:32 Brad well,
8:32 Brad isn't that how I'd want them to code it if they worked for me? Because I'd want them to go as fast as, like, if you're wasting your time pressing the enter key and the tab key and the bracket key,
8:42 Brad like,
8:42 Brad that's, you're slowing yourself.
8:43 Brad yourself down i want you to move faster than that anyway so you absolutely use the ai agent to solve this problem but then i need you to come back and you need to code review it you need to prove that it's working you need to have you need to have implemented your unit test you need to have written your your your er description and you need to come to me and tell me and and defend your code in front of me right like tell me how what was hard about that why was that a challenging question why was that challenging what
9:10 Brad And how did you solve that?
9:11 Brad What did you?
9:11 Scott So I have two questions,
9:13 Scott two comments leading into a question or something like that.
9:16 Scott One is something you also said on a previous podcast, and I agree with it, which is increasingly what's required when working with the AI coding assistance is like tech lead and engineering manager type skills,
9:30 Scott which is true. And I've always felt in interviewing, even interviewing for, you know, entry level and mid level IC engineers evaluating their sort of those types of skills is really important.
9:41 Scott So I guess my question is, if you were designing an interview loop,
9:46 Scott you're at a new company,
9:48 Scott you're going to hire some people,
9:50 Scott you're going to design an interview loop in this AI assisted coding world,
9:55 Scott how would you do it these days?
9:57 Scott How would you introduce this piece to it?
10:00 Brad Well,
10:00 Brad I already told you that I would I tend to lean away from like.
10:06 Brad code questions anyway
10:07 Scott Yeah.
10:07 Brad and I want to lean towards higher level thinking anyway right which I think lends itself to what I'm saying is the the better model to use AI agents anyway right
10:19 Brad is
10:19 Scott So is that like, tell me, you know, is it basically tell me, is it a star question?
10:24 Scott Tell me about a time where you developed something with an agent.
10:27 Scott What was the problem?
10:28 Scott It was, you know, star situation, task,
10:31 Scott action result.
10:32 Scott Is that basically,
10:33 Scott is it those kind of questions with
10:34 Brad Sure.
10:35 Scott an AI agent?
10:35 Brad I think you could totally make a bunch of star questions around how do you use AI agents, how do you use them to solve specific problems.
10:45 Brad uh you know and you could certainly use star like questions around all kinds of things like building new software refactoring old software finding a particularly challenging bug right um i do think that like you know we had one of our one of our early episodes uh last season was uh with ted neward where we talked about interviewing in general i thought that was there were some really great great nuggets that came out of that that conversation and one of the
11:12 Brad Points that I think Ted made that I 100% agree with is that, you know, you really have to have a very clear expectations for the role and your interviewers need to evaluate the person's fitness for those expectations of the role.
11:26 Scott Hmm
11:26 Brad Right.
11:27 Brad And so I think more and more the role of being a software engineer in the AI assisted coding world is all about,
11:33 Brad are you good at evaluating code written by another developer,
11:38 Brad aka the agent?
11:39 Brad Are you good at evaluating the code?
11:40 Brad So one thing that might be useful would be like,
11:43 Brad you know, I can imagine this being an interview loop thing,
11:47 Brad design an interview loop that includes like.
11:50 Brad We have a bunch of pull requests,
11:51 Brad a bunch of code written.
11:53 Brad Do a code review.
11:54 Scott Code review interview.
11:55 Scott I was thinking about just earlier this week was telling me about that I work with.
11:59 Scott We had some code written in Ruby and we wanted a TypeScript version and one person was bellyaching about it and the other person was like, well, let's just see if an agent can do it.
12:10 Scott And it was basically a hole in one, you know, they had to rework the prompt a little bit, it took them about an hour,
12:15 Scott which sidebar, I think that's one of the key things that people miss with this AI stuff is sometimes it takes a while to get the prompt right,
12:21 Scott you know, you have to iterate.
12:24 Scott But after a while, they got it.
12:27 Scott And so I wonder if that's an interesting thing to just almost,
12:31 Scott you know, here is, I don't know, something written in one language.
12:35 Scott Maybe we can sit down and pair together and you can just tell your AI of choice like to make it in this other language.
12:41 Scott And then we can look through it together and you can tell me if you think it's right.
12:45 Brad That has been a technique that I've used to varying degrees of success in interviews in the past,
12:50 Brad and I would like to get this to work well consistently,
12:53 Brad and that is a pair programming interview.
12:56 Brad You know, VS Code does support this, like, shared remote mode where you can basically,
13:01 Brad like.
13:02 Brad turn on like basically open up vs code in a way that you using your vs code can connect to my vs code and i literally can watch you type and follow your cursor around and you're looking at the code base which is my little sample code base and you're changing it directly right think cursor has this mode yet hey cursor team like let's turn this on like pair programming uh agents where we both are able to look at the same agent
13:28 Brad agent and type into it and chat with the agent together and I would love to basically see how you use the agent you know so here's some code it's got a bug in it or here's some old code in one language tell me how you use tell me how you use the agent to to implement this feature right
13:44 Scott I got a lot of questions in there.
13:46 Scott First to comment company I was at like a long time ago now we did this cool thing a pair programming interview.
13:52 Scott This is how it worked. I'm the interviewer.
13:54 Scott you are the interviewee and we're working on the same code and it starts like okay i'm going to write a test and it was a basic thing like we're going to make a calculator or something you know one of these we're going to make the elevator algorithm or the calculator i don't know what it was something basic like that right i'm going to write a test now you make a pass okay cool we did that one sweet now you write this test
14:15 Scott that person does it okay now i'm going to make that pass right back and forth back and forth ping pong and we sort of had a script to like escalate
14:22 Brad Yeah.
14:22 Scott it and make it go on and on for whatever this thing was we were building but it was a cool way of just
14:27 Scott working with somebody you see we've got to see like how do they write tests how they write code what is it like to work with them he was just sitting next to him sitting on the same computer sharing it that was a fun way to interview okay so here's the question about this agent thing I'm giving you some code I want you to make the agent make it do something different would you give them some good clean code or would you give them some really dinky code
14:51 Brad I like I've done this most recently literally with the code base.
14:54 Brad base of the enterprise that we're interviewing for so like like
14:56 Scott Okay.
14:57 Brad here's here's like a real project that needs to happen did exactly that project and it didn't like there were there were mechanical problems logistical problems with it that i think kind of got in the way of it so that's why i'm sort of saying like like i feel like it's a a very straightforward problem to say literally here's a ticket here's the ticket here's the code tell me how you solve this problem
15:20 Brad right
15:20 Scott Yeah.
15:20 Brad because that's what's going to happen on day one anyway and if the code base is big enough i mean i just fixed some bug today in some code base where it's like fuck i haven't looked at that file i have no i didn't write that code some other yahoo on the team wrote this code right or like i get dropped in as a fractional cto like when we were talking to uh scott bradley you know he's like his his firm like it's dropped into places and it's like their whole job is unfucking code bases right
15:46 Brad right it's like so you're dropped in on some random code and it's just like it's a mess well guess what that's the job like that's the job and i'm more certain that with ai coding like we said uh you know i think we said this in a couple episodes ago where i'm like i think 2026 is all going to be about this case-shaped economy for like the haves and the have-nots around understanding uh ai-assisted coding so i for the people that are good at ai coding
16:10 Brad There's going to be more and more work for them to do to go unscrew up code bases that were messed up by humans and AI,
16:19 Brad right?
16:19 Brad So expect that you're going to randomly be dropped in some code you've never seen before, and it's a real test to your ability as an engineer to be able to like quickly figure that out.
16:33 Brad Right.
16:34 Scott I create any intellectual property issues.
16:36 Scott You got interview candidates looking at your real code.
16:40 Brad Maybe.
16:40 Brad Obviously, you have to solve that as appropriate.
16:43 Brad I'm less I mean, most the average the companies companies I tend to work with are startups that are moving so fast that they that's not a problem to have them sign an NDA.
16:54 Scott Or I was thinking the other way around,
16:57 Scott which is we
16:57 Brad Oh, I'm not going to actually use the code, right?
17:00 Brad Like, besides,
17:01 Brad maybe I do.
17:02 Brad I don't know.
17:02 Brad Like, you mean like work for hire,
17:04 Brad intellectual property,
17:05 Scott Yeah,
17:05 Brad work for hire?
17:06 Scott yeah
17:07 Brad Yeah, okay.
17:08 Brad We're trying to evaluate their ability as a, yes,
17:11 Brad granted,
17:12 Brad solve that problem,
17:13 Brad have the legal team come up with something, but there's no reason why you can't solve that problem.
17:18 Brad Maybe you don't actually have them.
17:20 Brad you don't commit the code right like in the in the example that I am thinking of I knew it was a very doable project because literally a developer had just done it and
17:30 Scott Mm
17:30 Brad that developer
17:30 Scott -hmm.
17:30 Brad told me oh yeah that took me like you know 20 minutes
17:34 Brad Right.
17:35 Brad So I know it's a do so that's an example where it's like it's a real example of the code that they'd be in.
17:41 Brad It's a project that I know is a solvable problem.
17:44 Brad Right.
17:44 Brad And so really we're just testing that we're directly testing the engineer's ability to do exactly the types of problems that we need to have them do. And.
17:54 Brad It tells it shows me how they solve problems,
17:56 Brad right?
17:57 Brad The tools that they reach for.
17:58 Scott And you're just looking for all the normal stuff you kind of look for which is are they breaking it down into small tasks?
18:04 Brad Yeah.
18:04 Scott Are they normal kind of things interesting?
18:07 Brad So it's interesting.
18:08 Brad I just I literally just did this interview with a candidate for another firm and it was like, to be fair,
18:14 Brad the person was relatively junior.
18:16 Brad And,
18:17 Brad you know, I asked my,
18:19 Brad you know, talk to me about a project you've worked on.
18:21 Brad John and this person had very very limited experience and what was really interesting is effectively they said man I I thought you were going to be grilling me on react native and and so they were kind of like this interview is a lot easier than I thought it would be you know and you know at the same time I was like
18:44 Brad if you're not really experienced enough for the role that we're trying to hire so but keep up the good work and talk to us in the future you know um but i just think that these like coding quiz questions are a waste of everybody's time
18:59 Scott There was a trend.
19:01 Scott I'd say it was about seven, eight years ago of companies that, you know, made online platforms for sort of coding quiz questions.
19:11 Scott You know, you'd screen a resume and someone passed that, then you'd say, OK,
19:14 Scott go do these coding tests and then it would give you a score kind
19:16 Brad Yep.
19:17 Scott of thing.
19:18 Scott And we tried that and it was OK.
19:21 Scott I mean, it certainly it was these it was as you say,
19:23 Scott it was these leak questions,
19:24 Scott right?
19:25 Scott and got to see if there was some proficiency. But I think that assumes that there's something on the other side, which is, for lack of a better way of describing it,
19:36 Scott that there are perfectly well-described tickets that you want someone to do,
19:40 Scott right?
19:40 Brad Yeah, exactly.
19:42 Scott And in at least in the realities I've lived in, there are never perfectly well described tickets,
19:49 Scott despite our best efforts.
19:51 Scott In fact, I would actually say differently, there are never perfectly well described tickets because there hasn't been a lot.
19:59 Brad it's like i'm going to detour a little bit i was talking with one of our engineering managers yesterday um we don't use we most of the people i work with use a tool called augment code which is a competitor to cursor Yep we bought an enterprise license and at the time we did is because they had a bigger context
20:20 Brad window uh i think chris has probably pulled ahead but that's beside the point anyway uh this engineering manager he is uh he's plugged our github our confluence our jira our company objectives all sorts of stuff into this thing he essentially he refers to it as a problem brief but like you know he was showing yesterday there's like this one new error showing up in the log and it's like what is this and someone might
20:45 Brad i file a ticket which just has the street of the air like hey we started getting these on tuesdays
20:49 Scott Right.
20:50 Brad and you know some engineer would go well i guess i'm gonna go start digging into this and he said he sent an agent about it and wrote like i don't know like two or three pages of like information and context of like where this was coming from and might be what might be causing it and
21:05 Brad yeah potential ways to fix it and all
21:07 Scott Yeah.
21:08 Brad sorts of things like that and so ultimately you get a much more robust ticket which potentially you could handle hand to an engineer say now let now fix this or now let the robot fix this I'm not actually sure why you wasn't just coming up with answers to those things
21:23 Brad When I say answers, I mean solutions, but it might be that there were some choices that had to be made. I don't know what I've digressed a ton from those leak code questions or those coding platforms assume that there's like these very rote code things on the other end and those days are gone with the agents. If there are,
21:39 Brad the agents are doing them.
21:41 Scott Well,
21:41 Scott one,
21:41 Scott those days are gone.
21:43 Scott And two,
21:43 Scott I think the teams that hired those types of engineers are just focused on that.
21:48 Scott that for hiring engineers they weren't really they were getting they were getting uh coders they weren't getting engineers they were getting people that would clickety clack on a keyboard that's not the same as like actually thinking about the business problem understanding it you know understanding how to solve the business problem and and how to understand the business description like there's i mean like
22:10 Scott That's the hardest part about being an engineer is translating the non-technical description that you inevitably get that is not very useful into something that's actionable and testable and implementable.
22:25 Brad I've thought about this so many times on this podcast and elsewhere.
22:28 Brad That is the value.
22:30 Brad If you can translate something from an idea to action.
22:34 Scott So let's go full circle back to the original question you asked.
22:37 Scott In the day, in the age of AI coding,
22:40 Scott how do we interview for engineers?
22:44 Scott And I feel like the answer is we have to lean that much more heavily on how good are you at understanding.
22:53 Scott vaguely describe business problems and then describing them in you know describing them in a manner that allows them to be broken down into sufficiently executable tasks by an agent and then that you can evaluate the quality of the plan and ultimately the implementation done by that agent
23:17 Brad Mm-hmm.
23:18 Scott That's what matters.
23:19 Brad That's it. That's the interview loop right there.
23:21 Scott That's the interview.
23:22 Scott And so tell me examples of that.
23:24 Scott Star questions that relate to that.
23:26 Brad We're going to have four sessions and focus on each one of them.
23:29 Scott I love it.
23:30 Brad I interviewed at Microsoft once.
23:32 Scott Uh-huh. Did they ask you leak code questions or ask you how many ping pongs it takes to fill up the Smith Tower?
23:37 Brad They did not.
23:38 Brad I remember a question that was asked of me though.
23:41 Scott Okay.
23:42 Brad you are designing a key fob for a car what do you need to know and it was pretty interesting because i knew nothing about key fobs and i was a i was applying for a job working on the visual studio team actually so like i had nothing to do at all with what they were doing and it was a little bit of a trick interview question but it was you know just one of those questions that they were getting at trying to understand how do you think and
24:04 Scott Yeah, totally.
24:07 Brad It only comes to mind now because I think actually in these days, that's actually might be kind of a handy question.
24:13 Brad Somebody comes to you and asks you to build them a key fob and you know you can get the agent to build the key fob, but you got to ask all the right questions so
24:19 Scott Exactly.
24:20 Brad you can tell the agent the right thing to do.
24:21 Brad What do you need to know?
24:23 Brad First question is, what is this key fob opening?
24:26 Scott Right.
24:27 Scott Right.
24:27 Brad What's the key fob for?
24:29 Scott Right.
24:29 Brad Um,
24:30 Brad so maybe those kinds of questions, which I think sometimes can be a little bit of a trick question.
24:35 Brad I was also asked the question,
24:36 Brad how many gas stations are there in the United States, you know, but maybe some of those questions are a little bit more relevant these days.
24:42 Scott Yeah,
24:43 Scott I feel like some of those kind of are just a little bit around trickery and just like estimations and strategies for estimations. But I agree with you that that that there's like it does.
24:53 Scott If you explain your answer to me, then I get.
24:55 Scott I get to see how you think the problem through.
24:58 Scott I just would like to turn those into questions that are more relevant to the task at hand,
25:03 Scott right?
25:03 Brad Yes,
25:03 Scott So
25:03 Brad I agree
25:04 Scott I like
25:04 Brad with your you.
25:04 Scott key fob questions a better one than how many ping pong balls will fill Smith Tower or how many gas
25:11 Brad That's stupid.
25:11 Scott stations are there in the United States.
25:15 Scott You know, like to me,
25:16 Brad Gas station ping pong are stupid, but I think the key fob is closer, but key fobs have a stretch with its visual studio.
25:22 Brad studio if somebody said to me we want to add spell checking in visual studio doesn't believe it or not it doesn't have spell checking right
25:29 Scott Yeah,
25:29 Brad now
25:29 Scott there are some plugins for it.
25:31 Brad so someone's put it in a ticket put out of the spell checker what
25:35 Scott Yeah, how do we do it?
25:36 Brad what do you need to know
25:38 Brad Right.
25:38 Scott Yeah.
25:38 Brad That would probably be kind of maybe irrelevant this question.
25:41 Brad I don't know.
25:42 Scott By the way, I only because, you know, I'm a big believer in like if you're going to like have coding standard rules,
25:48 Scott then if, you know, make sure you can codify them into your linting and your and your static analysis toolset. And I had a developer on a team that was like really adamant about like he would like refuse pull requests that had like spelling mistakes in the comments or.
26:05 Scott grammatical mistakes in the comments and stuff like that and i was just like oh this is driving me crazy so i installed like a linting plugin that does spell checking and
26:16 Scott it was it's pretty good but it's surprising how many technical words it doesn't or how many times you would like like even if you kind of like are anti-abbreviation there's still times where it's like so this c spell thing just has like this extra extra thing to add your ignored words or your special words you know it's just like man so spell checking that's a fun that's a fun one
26:39 Brad So
26:39 Scott i
26:39 Brad not a good one.
26:40 Scott mean no it's a good one it's it's got some it's
26:43 Scott some it's got some uh it's got some sharp edges to it so you know yeah
26:47 Brad Yeah, that's an obvious edge,
26:49 Brad right? You're going to get all these non-English words.
26:52 Scott so i actually like that one that's a that that right there does sound like a good example like if you if someone for if someone was interviewing for you know
27:00 Scott VS code or something like that or any kind of an editorial product then say oh yeah we're going to add a spell checker what do we need to do what do we need to know what are the what are the shapes and what are the boundaries of this thing how are you going to prompt your AI coding agent to build this etc
27:15 Brad Very relevant.
27:16 Brad I could use that in TaskRabbit. When you go to create a task,
27:19 Brad there's a message box where you can type in a free text description of what you want.
27:25 Brad And, yeah, hey, we want to add, there is no spell checker in there now.
27:28 Brad We want to add a spell checker button or actually I led the witness.
27:35 Brad We want to add, I said spell checker button.
27:37 Brad We want to add spell checking
27:39 Scott Yeah.
27:39 Brad in there.
27:40 Scott Yeah.
27:40 Brad because it minimizes confusion and if we can minimize confusion we can increase conversion and as we've talked about on a recent episode decrease conversion increases customer conversion which lowers CAC and
27:54 Scott Yep.
27:54 Scott Yep.
27:55 Scott Love it. There you go.
27:56 Scott There's the business reason.
27:57 Scott Now tell me how you're going to build it.
27:58 Scott Or
27:59 Brad you build
27:59 Scott feel free to tell me your favorite AI tool that you're going to use to do it.
28:03 Scott Yeah.
28:04 Scott I love it. There you go.
28:05 Brad Do you think for this specific,
28:07 Brad I'm going to interview you for a second,
28:08 Scott Okay.
28:08 Brad for this specific implementation,
28:10 Brad I'm really just getting at a pet peeve I have lately,
28:12 Brad which is people using AI models for things which AI models are not appropriate
28:17 Scott Oh,
28:17 Brad for.
28:17 Scott wait,
28:17 Scott should you use like an AI LLM to be your spell checker?
28:20 Brad Yes.
28:21 Scott No,
28:21 Scott not at all.
28:23 Scott That's a bad idea.
28:24 Scott Why is it a bad idea?
28:26 Scott I'd say that's a bad idea.
28:28 Brad Because I think spell checking is fairly deterministic.
28:30 Scott Yes, and it's a well-solved problem that can be done in a like there are various spell checking libraries that are compete on speed and performance.
28:41 Brad Oh boy, yeah.
28:42 Brad So we did an AI hackathon,
28:43 Brad lots of cool stuff,
28:45 Brad but there were a handful of things where I was watching the demos and I was like, that's not a good use of AI.
28:51 Brad That's a deterministic problem.
28:53 Scott Exactly. You don't want a probability machine solving a determinant. I don't know.
28:59 Scott Sometimes strawberry does have two R's in it and sometimes it has three R's in it. It depends.
29:06 Brad Well, this actually begs a different question. Now we're way off of the interviewing topic.
29:10 Brad This is bonus episode for the listeners who stayed to the end of this thing.
29:14 Brad This has to do with the perfect is the enemy of good.
29:20 Scott Yeah
29:20 Brad Now maybe in spellchecking it's not,
29:22 Brad but for other solutions where it's like,
29:24 Brad well,
29:25 Brad maybe probabilistic is good enough because you know why?
29:29 Brad They could build it fast.
29:31 Brad It would have taken them more time to go figure out what these different libraries were and then to get a license for them and to figure out how to get it into the build blah, blah, blah, blah, blah, blah, blah, blah.
29:41 Brad Like, look, we had an opening eye token already.
29:44 Brad It was really easy.
29:44 Scott I don't know. If they gave me that answer,
29:46 Scott then I would sort of be like, it's spell checking.
29:49 Scott I'm not sold on that one.
29:52 Brad Fair enough, fair enough.
29:54 Brad Well,
29:54 Brad Brad,
29:55 Brad I feel like I learned something today.
29:57 Scott Well,
29:57 Scott thanks for playing.
29:59 Brad It's not a podcast.
30:01 Scott Unless we say rate
30:03 Brad Smash that subscribe button.
30:05 Scott and review.
30:05 Scott All right.
30:06 Brad Thanks for playing along.
30:07 Scott Until next week.
30:09 Brad Until next week.
30:10 Scott Good
30:10 Brad Bye,
30:10 Scott to
30:11 Brad everyone.
30:11 Scott see you, Scott.
30:11 Scott Bye, everybody.
30:12 Scott This episode of How Many CTOs Does It Take Podcast was edited by Saffron Epigal.
30:17 Scott Our theme song was AI generated by Brad.
30:21 Scott Please follow us on all the social media platforms at at How Many CDOs Pod.
30:26 Scott You can find us on YouTube,
30:28 Scott LinkedIn,
30:28 Scott Instagram,
30:29 Scott Facebook,
30:30 Scott Threads, X,
30:30 Scott TikTok,
30:31 Scott and Patreon or at our website at HowManyCDOsPod.com and remember to rate,
30:38 Scott review,
30:39 Scott and subscribe on the podcast platform of your choice.
30:43 Scott Thanks for listening.