Transcript: How GenAI Is Changing the Two Guys in a Garage Founder Story
0:00 Scott I fully believe that if you had zero knowledge and you had the patience,
0:04 Scott you could sit down and use these tools to teach you about it. You could actually build from zero if you had no knowledge. It'd just take you longer.
0:28 Brad Welcome,
0:28 Brad everybody, to another episode of the How Many CTOs podcast does it take?
0:32 Brad I'm Brad Hefta-Gaub, and this is my co-host.
0:34 Scott Scott Porad.
0:36 Brad And today we have two very special guests,
0:39 Brad a couple of friends of mine from the ski patrol world who are also tech entrepreneurs,
0:46 Brad Justin Wolfe and Joe Malik. And why don't I kick it to you, Justin,
0:50 Brad and maybe introduce yourself and then.
0:53 Brad We'll hear from Joe.
0:55 Scott Sure.
0:56 Scott Hi, everyone.
0:57 Scott My name is Justin Wolf.
0:58 Scott I'm one of the guys in the garage trying to do something cool with other technologies.
1:05 Scott Historically, I spent some time in Amazon,
1:07 Scott which was a lot of fun,
1:09 Scott traveled the world.
1:10 Scott Then I worked at a startup,
1:12 Scott got to watch some Comedy CTOs blow $2 billion in two years.
1:16 Scott Now we can talk about that.
1:18 Scott And now I'm trying to build something.
1:21 Scott So Joe,
1:22 Scott tell us about you.
1:23 Brad Yeah, well,
1:24 Brad I'm in the other half of that, my virtual garage.
1:26 Brad Justin's garage is way over there and some mine's here. We're doing this all virtually.
1:30 Brad And my former roles have been at Oracle in product,
1:35 Brad leading engineering teams and all that. And so I'm really excited to be working with some of the Gen AI tech.
1:42 Brad I like to think that I've got some little ghosts in the machine as it were. I got like Claude hiding over here,
1:47 Brad Gemini's hiding over there.
1:49 Brad And Justin and I have been working on this thing for what, about four months now?
1:53 Brad Started with a bunch of empty blocks on my hard drive and now we got, I don't know,
1:59 Brad thousands upon thousands upon thousands of lines of code to create this new scuba diving guidebook community app.
2:07 Brad You want to tell them about the app,
2:09 Brad Justin?
2:09 Scott Yeah, so I was laying in bed one night.
2:13 Scott And trying to figure out what to do.
2:16 Scott I wanted to build an app.
2:17 Scott I wanted to build something.
2:18 Scott I didn't know what.
2:19 Scott I didn't know why.
2:20 Scott I didn't know. I had no concept of like what it should do,
2:25 Scott but I wanted to try and build something.
2:26 Scott A, just for the practice of building something from ground zero and B,
2:32 Scott because I need something to do. It's kind of the reality of it.
2:39 Scott I went you know just thinking in my head I was like mountain project for scuba diving that's literally how it went so I got on the phone and I kind of did a couple google searches and then I called Joe I said Joe we're going to build trail forks and mountain project and all trails for scuba diving
2:56 Scott that's what we're going to do and because we've been talking about it like two two three weeks like what what should we do what do we do because we wanted to try and build build a piece of technology that was a interesting to build be something around the passion we both have on in a hobby and C would be useful to people that was kind of kind of the main guardrails
3:20 Scott And so that was kind of the idea.
3:22 Scott And the reason I say useful is there's a lot of information on the Internet on.
3:27 Scott dive sites or scuba diving or where can you go diving but it is not centralized in any shape or form it's not reliable in any shape or form and it's not easy to find and so those are kind of the three guardrails or three pillars I wanted to be reliable easy to find and a centralized space so how do I build that like what do I do well there's a couple sites out there that scrape everything under the Sun
3:53 Scott And there's like 50 dive sites in the center of Bainbridge Island,
3:56 Scott which makes zero sense.
3:58 Scott Then there's a couple apps that get information from people and their dive watches,
4:04 Scott and they call a quote dive site when you enter and exit the water or more specifically when the app and the watch connect.
4:13 Scott So you've got dive sites in the middle of Fremont and Seattle and Texas and everywhere else.
4:18 Scott So that's not reliable information.
4:19 Scott information.
4:21 Scott I also wanted the information to be very much from divers to divers.
4:26 Scott So there's nothing automated about what we're going to put in there.
4:29 Scott We're going to create mechanisms to highlight areas where we know there's a dive site,
4:35 Scott but we're not going to have any dive sites in there that a human did not say yes.
4:39 Scott I dove this,
4:41 Scott I've been there, I've seen this, this is how you do this because I want that information to be
4:44 Scott to be very reliable and trustworthy then you wrap around that a lot of social aspects of photos and
4:50 Scott maybe some type of gamification and different type of logbook activities and future down the line how to connect different dive watches to it for dive profiles and different mapping software so you come up with kind of like an all trails or a mountain project for scuba diving it's kind of the best way to describe it if you're not a scuba diver in the easiest words possible so how do we get from hey what what should we use Joe
5:16 Scott Oh, I don't know.
5:18 Scott Mern,
5:19 Scott what's Mern? I have no idea to where we are now where we probably have,
5:24 Scott I'm guessing we have at least 500,000 lines of code.
5:28 Scott Joe is probably going to go lower,
5:31 Scott but I'm like, I'll be optimistic.
5:34 Scott so how did we get from zero to where we are now and I hadn't coded probably in 20 years like I started coding in Perl and C C C plus plus and then machine or yeah machine coding for control systems which is not Java not Python in any way shape or form so I was like I don't even know how to even think about this like I can have conversations about
5:59 Scott about architectures and data structures and servers and AWS goo to put something together really big when you're talking about the data you know data centers like I used to build data centers and big networks but in terms of like hey I want this this thing this app to take this person's login and use a data structure and some security goo to make it so that this person can access these things
6:25 Scott I don't know how to do that so it in comes generative AI which has been beautiful and very frustrating at the same time and so
6:35 Scott That's kind of where we are now. We're working through a whole bunch of different features.
6:40 Scott I'll say we have more than a structured product.
6:43 Scott We have a product right now that works like if you wanted to log in and do stuff with it. I think since you saw it last,
6:49 Scott Brad, it's probably changed a little bit.
6:51 Scott But yeah,
6:53 Scott so generative AI has been been excellent and horrible at the same time. But as a learning tool.
6:59 Scott Awesome.
7:00 Scott Un unbelievably awesome.
7:03 Brad You know, Justin,
7:04 Brad it's interesting.
7:04 Brad I think that one of the things that we've talked about on past episodes and we've kind of every time we every time we talk about Gen AI,
7:12 Brad there's always this question of like, you know, hey, is it useful for.
7:17 Brad you know, non-technical people,
7:19 Brad or is it useful only for engineers?
7:21 Brad And of course, the hype cycle that's out there is like, oh, it's going to replace all the engineering jobs.
7:26 Brad And I guess the one thing that I think is kind of interesting,
7:28 Brad I think there's two points about this that I find really interesting.
7:32 Brad The first one is that both you and Joe actually have, you know,
7:36 Brad 20 plus years,
7:37 Brad you know, 20 plus years each of experience being very technical.
7:42 Brad And so I think you're
7:43 Brad you know, you may be underselling your, your, your technical acumen a little bit. And at the minimum, you know, you're, you're experienced enough that you kind of know the questions you want to ask for other than just the features, you know, you're looking for specific things out of the stack out of login and things like that.
8:04 Brad And so I think that's really interesting.
8:06 Brad And I think that
8:08 Brad You know, I'd love to hear maybe some war stories from you and Joe about like what's been hard about that.
8:16 Brad The other thing, and maybe we can circle back to this that I think is also really interesting is that it strikes me that.
8:24 Brad The app that you're building here,
8:25 Brad which, you know, being a scuba diver myself,
8:28 Brad I know the two of you are very avid scuba divers.
8:30 Brad I don't know about you, Scott,
8:31 Brad if you've ever scuba dived before,
8:33 Brad but it feels like the type of app that would be very difficult to get a lot of funding behind,
8:39 Brad right?
8:40 Brad Because it feels pretty niche.
8:41 Scott Yeah
8:41 Brad And so in some sense, and in some sense, so therefore,
8:45 Brad it's like almost like the perfect example of the thing that becomes possible.
8:51 Brad because of this
8:52 Scott Right.
8:53 Brad tool set.
8:54 Scott Absolutely.
8:54 Scott Yeah, 100%.
8:56 Brad Yeah, for sure. Because I think when we look at the called the landscape of this, you know, all the device manufacturers have their own app that's out there for managing the dive log.
9:05 Brad Some of the other, you know, larger conglomerate equipment providers are starting to get into the space where they're starting to see some of these social things come in, but they're all, you know, within their, you know, walled garden of.
9:17 Brad of what that device is.
9:19 Brad And so Justin was talking about the openness and made reference to some of those community aspects of what's really important to us.
9:27 Brad And yeah,
9:29 Brad when we look at some of the information that came out of DEMA,
9:32 Brad Dive Equipment Manufacturers Association last year, it looked like if we were to get 10% of the North American diving,
9:39 Brad scuba diving market,
9:40 Brad it'd be about 200,000 users.
9:42 Brad so it's not not a huge number but it's definitely going to make things like dive planning easier you know potentially I don't know safer if you've got a better way to visualize what's what's going into it
9:59 Brad And then, yeah, organizing the data and then keeping all of the things in this really actionable thing has been really, really interesting. And working with the Gen AI tools myself,
10:09 Brad it's been able to take that concept of, you know, an iteration down to like iterations on the scope of minutes.
10:17 Brad And then you can quickly discard all of that. And I remember many of the interactions that we used to have with.
10:24 Brad You know, the engineers, when I was on product teams,
10:26 Brad you know, you'd have turnaround cycles of, you know, three days to figure out that they either understood or didn't understand, you know, what problem they were trying to solve. But, you know, with these things, you can quickly iterate through a prototype and I can get something done in 40 minutes and it's like a chunky feature,
10:42 Brad you know, call up Justin, get his input on it and then make these adjustments.
10:46 Brad So any of the attitudes around.
10:49 Brad Well, there's so much time that we put into this code that this feature is already here.
10:52 Brad We spent a bunch of time on it. They're just gone because it was, you know, no skin off of my back to turn around an update to the code.
11:03 Brad And so for me, that's been kind of the most interesting part.
11:07 Scott Yeah, the thing I would say is like.
11:12 Scott Through technology management,
11:14 Scott the key is I used to manage one of the biggest problems was deciding what to build and then how to build it and then did we build the thing the way we said we were going to build it.
11:26 Scott And so there's a you go through a lot of different iterations with tech teams and PMs and business units to argue that customer journey to technology mapping with the AI that we've been using. And granted, we're building pretty simple things right now. Like they're not they're starting to get more complicated.
11:44 Scott Today we were like,
11:45 Scott oh, shit,
11:46 Scott now I've got to touch 12 different things. But you can build something in 40 minutes.
11:52 Scott test it prototype it and then throw it away and you're like i don't really care
11:57 Scott Because you stash it and put it in the corner for another day.
12:02 Scott It doesn't matter.
12:03 Scott I built an entire admin console to manage permissions and locations and tiers.
12:08 Scott I did it in an hour,
12:10 Scott an hour and a half.
12:11 Scott And basically I'm using it for myself in order to manage features on the site as we keep moving.
12:17 Scott It probably never will get touched by anyone else, but it's okay because I did it in a couple hours versus having.
12:23 Scott having 12 engineers sit there for a week and a sprint to build an admin console.
12:30 Scott So you're able to build, throw away and iterate on software extremely quick,
12:37 Scott which is awesome,
12:38 Scott right?
12:39 Scott And you're also at that time, I guess the plus and minus when we talk about learning,
12:43 Scott right, is you can become extremely dumb to the fact of what you're doing.
12:50 Scott I'm just giving it prompts copy and pasting giving prompts copy paste doesn't work versus I give a prompt I get some code base I put it in I look and see what it's doing and you can see the AI is good
13:03 Scott but once once you get past I found that like with Claude and Gemini once you get past about 300 lines of code it starts to
13:11 Scott go off and spiral off in different directions especially around when you talk about off you know authentication and using off contacts different hooks it wants to do it a certain way that maybe is not your way and even if you tell it I'm doing it this way sometimes it doesn't go is that no no no you're gonna do it this way and so you have to make sure you understand that if you don't understand that
13:34 Scott You wouldn't be able to put this together.
13:37 Scott So to your point,
13:37 Scott Brad,
13:37 Scott yes,
13:38 Scott you have to have some technology understanding in order to.
13:44 Scott Set up a Mern stack,
13:45 Scott use AI to build your code and understand it.
13:47 Scott However,
13:48 Scott I fully believe that if you had zero knowledge and you had the patience,
13:53 Scott you could sit down and use these tools to teach you about it because it does explain it.
13:59 Scott You can say, what is a Mern stack and what should I use and how should I use it? And it will give you options. It's getting smarter.
14:05 Scott It gives you options, option A,
14:07 Scott B or C.
14:08 Scott build you could actually build from zero if you had no knowledge it'd just take you off
14:13 Brad I got two questions.
14:14 Brad The core question I want to ask is, how do you know if what the AI is giving you is correct?
14:20 Brad But before you answer that question,
14:22 Brad which AI tools are you using? You've mentioned, I think, well,
14:26 Brad you mentioned, I don't know what you mentioned.
14:28 Brad So what AI tools are you using in this journey?
14:32 Scott Yeah,
14:33 Scott so we use different ones. We started using Gemini and then ChatGDP.
14:42 Scott And then I found Ninja AI,
14:44 Scott which kind of uses all of them.
14:46 Scott So I'm now a paying member for Ninja AI and I can pick which one I want to use.
14:50 Scott The best one that I found is Claude. I think Joe might agree with me,
14:54 Scott although Gemini sometimes can give you much better iterations on logging.
15:00 Brad but they each have personalities right
15:02 Scott Yeah,
15:03 Scott they have different personalities.
15:03 Brad and i think you've noticed that too justin it's like the gemini stuff is really good
15:08 Brad Like procedural housekeeping stuff,
15:10 Brad blasting things
15:11 Scott Hmm
15:11 Brad out where,
15:12 Brad you know, I'm trying to.
15:13 Brad figure out what's going on in one of our, you know, react components and I just paste the whole component in there and I say add logging and it just comes back with, you know, a well-formed series of checks to inform, you know, the front end of what it is and it literally takes a minute to get that back but then if you ask, you know, that same tool to go design you something,
15:36 Brad then we go off the rails really fast and so some of these.
15:40 Brad These different personalities are kind of fun to figure out how to best use.
15:45 Brad I mean, I've done probably more design work using CLAWD and probably more, you know, runtime,
15:51 Brad you know, run of the mill, you know, housekeeping infrastructure stuff that I don't really like to do at Gemini. And it's pretty good at it. And chat GPT as far as coding is, you know, for us on this project has only lasted about five minutes and we don't.
16:06 Brad Really
16:06 Scott Yeah.
16:06 Brad touch the watch anymore.
16:07 Scott How do you know it's right?
16:10 Scott You test it and you don't care if it's wrong.
16:12 Scott You literally just iterate through it, feeding it error message after error message to get it right.
16:20 Brad Then if it takes more than 20 minutes to test it you throw away and you do it again because it was clearly not right
16:24 Brad fascinating and
16:25 Scott Right.
16:26 Brad are you just using the web user interfaces of these tools or you have one of the built-in editors like windsurf or cursor or any of those things
16:37 Scott I'm just using the web.
16:38 Scott I don't like the built-in.
16:40 Scott I think on Visual Studio,
16:42 Scott there's like we tried the Gemini built-in. I hated that.
16:46 Scott like I wanted to type something specific and the damn thing just would type something I turned it off I was like no stop and this auto type mechanism it just drove me nuts because I'm trying to get a I'm trying to close a function or something and it pukes out it pukes out 40 more lines of function I'm like no I just want to close the damn function leave me alone and you can't do it
17:09 Brad yeah it's so funny you say that just then i was doing a
17:11 Brad A couple months ago, I did an interview with like a, you know, like a real senior engineer,
17:17 Brad almost a principal engineer level C++ developer. And I'd asked him like,
17:21 Brad oh, what are you doing with, you know, generative AI?
17:23 Brad And he made this point that I think is very similar to what you're making. But the way he described it was,
17:30 Brad hey,
17:31 Brad when I'm coding,
17:32 Brad when I'm like, when I know what I want to write and I'm coding,
17:36 Brad I'm in the flow state.
17:37 Brad and if
17:38 Brad Mm-hmm.
17:39 Brad something else is like popping up at me and interrupting the thing I'm writing then that's gonna like knock me out of my flow state and it's and I was like when he said that I was kind of like oh yeah that is so fascinating I hadn't I hadn't put it together in that way but I've definitely had that experience that you're talking about Justin in fact
18:02 Brad In fact, you know, literally just this morning I was working on something and it was like literally it was trying to auto complete some random stuff that was like the exact opposite of what I was actually writing. And I was just like, oh, my God, dude,
18:15 Brad stop.
18:16 Brad No, I mean, so it is funny,
18:19 Brad too,
18:19 Brad that I always.
18:20 Brad anthropomorphize these things it's like i think of it as like you know i i i say please and thank you to it when i'm chatting with it and i'm and i'm like you know like cursing it you know um
18:31 Scott Oh,
18:31 Brad but
18:32 Scott yeah,
18:32 Scott I've done that.
18:32 Brad but i but it is interesting that um it is interesting to hear that you say that um one of the advantages of the of the um
18:44 Brad Kind of the copy and paste into the web interface is that you don't feel like it's interrupting your writing.
18:49 Brad And I tend to agree with that.
18:50 Brad I have taken to I use cursor and so I'm using the in in IDE version all the time. But often I'm telling the agent, do not implement anything.
19:01 Brad Just tell me your plan.
19:03 Brad Show me your code here in the chat.
19:05 Brad But do not start changing the file until I agree with your plan.
19:10 Brad Right.
19:10 Scott Yeah.
19:11 Scott And that's how I use the web interface because I can have it puke out stuff,
19:15 Scott then I can review it.
19:17 Scott I can go down and I can follow it, read it, and I can review it before I do a copy and paste.
19:25 Scott And so that kind of sometimes can save the excitement. Today I was looking, I was like,
19:29 Scott no,
19:30 Scott that's wrong.
19:31 Scott Nope, nope,
19:31 Scott nope.
19:32 Scott So I had to go back and say no.
19:34 Scott do not do this i want to do this and it's stuck on the same tracks i basically was like okay restart like this this this chat this this instance of its brain how i don't know how that how to describe it was like it was not going to go away from the solution
19:50 Brad Yeah,
19:51 Brad by the way, that's very common,
19:52 Brad right?
19:53 Brad Scott,
19:53 Brad Yeah, I mean.
19:54 Brad it's a little OCD ADHD fixation on the part of the AI.
19:59 Scott I just had one more thing about the versus the NIDE bit.
20:03 Scott I think context is really important too.
20:05 Scott And so I've noticed that within VS Code, some of those other plugins that are on the left, they take into account all the tabs you have open.
20:13 Scott And then I'm like, I'm working on something like, whoa, where did you come up with that from?
20:16 Scott And so, you know, the interactions about which context it's picking up, it's, you know, my best successes with the tooling have been where I can, you know.
20:25 Scott limit the context and say here's the thing i need you to do uh go go and do it and then when it comes back you know as justin was saying kind of kind of review it so if you can control the context um you know brad that the plan it's it's a lot easier
20:38 Brad Yeah,
20:38 Scott to get the right
20:38 Brad we've
20:38 Scott thing
20:39 Brad we did
20:41 Scott Some of these comments you guys are making we've heard previously,
20:45 Scott which is validating to hear that there's themes that people are figuring out.
20:49 Scott I want to take a step back and go back even further to the very beginning.
20:54 Scott You ended up on this MIRN stack.
20:57 Scott Which for those who don't know that's what is Merne Mongo Express
21:01 Scott React node so what Merne is yep. You got it How did you get there or did cloud help you figure that part out too?
21:08 Scott Joe picked it otherwise I don't yeah I had a little experience with it and I said I don't want to earn it I don't want to really develop better angular skills now so we're gonna go with react and I don't really want to deal with you know structured
21:22 Scott relational databases so we're going to go with the no sequel and the rest is as it is not
21:28 Scott Got it. Now,
21:29 Scott we're talking about developing the app.
21:32 Scott Is there any use of AI in the app?
21:35 Scott Like, I don't know exactly how it works, but I imagine there's all this data that you're gathering from we're getting from somewhere.
21:41 Scott Are you using AI for actually part of the product?
21:44 Scott yet
21:45 Brad We'd like to. I
21:46 Scott Yeah,
21:46 Brad mean, I.
21:46 Scott we want to get into recommendations and insights generations because the whole goal of this is to make it easier for you to have a great dive,
21:57 Scott you know, pick the great time to go,
21:59 Scott pick the place you want to go to see the things or to get the experience that you want to have. And so, you know, the community data and getting it in is going to be really important and kind of where we're at strategically.
22:11 Scott you know,
22:11 Scott life cycle wise,
22:14 Scott we got to get a lot more data into the app before I think using any of the AI tech in the app is going to be useful.
22:20 Scott Makes sense.
22:21 Scott Okay,
22:21 Scott now I had a quick question from earlier that I forgot to ask. Well, I didn't forget to ask what the conversation moved on,
22:26 Scott but I'd love to go back to it.
22:28 Scott Justin,
22:28 Scott I was asking how you know if it was right and you said, well, you test it and if it doesn't work,
22:33 Scott eventually you throw it out.
22:35 Scott Are you guys using...
22:38 Scott you know whether it's cloud or whatever to write automated tests for you
22:41 Brad Not yet.
22:43 Brad We're like we've just been
22:46 Brad focused on getting the functionality up we had a conversation with Brad he's like you guys need to get some automated testing going and I think this week with the functionalities that I've added I'm like yeah we should probably start doing that because it's getting a little bit too hard to do it so we've got to spend some time what we want to do is get this sprint closed which will close down I think the main functionality that we want to
23:11 Brad want to present to prospective users or dive shops and then we'll go back circle back on the automation automation piece for testing but the core functionality of kind of what we envisioned we didn't really come up with with the
23:27 Brad with that core hook I guess of what why people would use this why would a dive shop want to use this until like I think last week Friday so like that's that's how fast we're moving with this it's kind of weird like you have a conversation Thursday and Friday you're like oh I got a new vision and then you by Thursday of this week that vision is almost complete and you're ready to roll it out
23:49 Scott Yeah, that's great.
23:49 Scott But I think to some of the things we've talked about, the way that AI can get in and kind of muck stuff up,
23:55 Scott I mean, we've talked to a couple of people where they're like, yeah, I don't use any of those, you know, full stack hosting things because every time I want to go put a new feature in, I get an entirely new app out the other end.
24:05 Brad Yeah.
24:06 Scott So Scott,
24:06 Scott I think to your question of, you know, are we using it? No.
24:10 Scott To draw that out to where would we use it is kind of once we
24:14 Scott if we lock down you know the the scope of a feature is to make sure that it keeps you know working the way that we wanted it to even though we're changing a bunch of stuff you know around it and you know maybe forklifting out tons of of it uh tons of the code so i'm less sensitive as to you know whether the contents of you know what's powering the api the controllers and things change i just want to make sure that the
24:38 Scott exchange between the front and the back end doesn't mean I have to rework the
24:43 Brad You
24:43 Scott whole thing yeah
24:44 Brad just said something I think is super interesting,
24:46 Brad and I want to rewind to it because I hadn't heard it before, but it sounds compelling.
24:53 Brad So the first is.
24:55 Brad Even though I was coaching you guys to get some automated testing in there early rather than later,
25:01 Brad I agree with the point, and I think Justin just made it,
25:03 Brad that if you don't really know what your product is yet and you're churning on features,
25:09 Brad then adding a whole bunch of automated tests is going to slow you down, right?
25:13 Brad But once you've locked into a set of features,
25:15 Brad that's when you want it to be proven that it's not going to break.
25:20 Brad Okay,
25:21 Brad so then this is what I just heard you say, Joe, and it was like a light bulb, if I understood you correctly.
25:27 Brad I think what I just heard you say is one of the reasons you've avoided using some of the full stack builders,
25:36 Brad like the Replets of the world, I assume would be an example, or maybe is it Lovable is another one, is that because
25:47 Brad Every time you ask for a new feature,
25:49 Brad it spits back a full product that may or may not have some of the old features you already liked,
25:58 Brad and it just gives you like a whole new thing every time. Did I understand you correctly?
26:03 Scott Yeah, depending on the context you get it like that might happen you might get an entirely new implementation of a function back or like hey I'm having you know a problem with this or or I want to add in you know this other thing and it changes the thing that the API returns from you know an array of strings into a new object type and then you drop back in and your front ends like I don't know what to do with that.
26:28 Scott And so that's kind of what I mean is, you know, we want to constrain that and we've talked to other people that have had that problem where they're using like, you know, a bubble or something like that where they say, hey, now do this.
26:42 Brad yeah bubble maybe even look we looked at bubble and thought about that and that was why we we wanted more control we're like no no no this is not
26:51 Brad where we want to be.
26:53 Brad We want control of the data, of the structures, of the flow of the user experience as we build this because it's not just like a fire and done type of deal.
27:04 Brad If it works and takes off,
27:08 Brad we want to be able to do something with it.
27:10 Scott I have two thoughts to share on this topic.
27:13 Scott One is I think you'll have some luck with automated tests.
27:17 Scott I know that the engineers at our company have had success at getting whatever and whatever tool they were using,
27:25 Scott Claude or whatever.
27:28 Scott Gemini, et cetera, to generate tests for apps that didn't have them.
27:33 Scott It took them time to get the prompt right and to get it to work right,
27:37 Scott but they were able to successfully do that.
27:40 Scott The other thing which I have not personally worked with,
27:43 Scott but it sounds like it might be interesting to you, Joe,
27:45 Scott there's a certain type of test called a PACT test,
27:49 Scott P-A-C-T,
27:50 Scott and it's specifically designed for the contract.
27:54 Scott tracked between server and client and um it's now that i think about it as we're talking about this it's the kind of thing that might be well suited to a situation where uh you know in this case the robot is trying to change the contract between uh between the server and client so
28:19 Scott That seems, and I have to explore that further.
28:22 Scott There's another thought I had, which is, did any of you read or actually watch the video?
28:28 Scott Really, they called it a movie that the Wall Street Journal put out this week.
28:34 Scott OK,
28:34 Scott so it was on the front page yesterday. I encourage you to go look,
28:37 Scott find it and look at it.
28:39 Scott It was one of their reporters worked with a producer and they made about a three or five minute, probably three minute little film.
28:47 Scott And the film was about what was called like my robot and me or something like that.
28:52 Scott But it was pretty fascinating how they had to do it. And the reason I'm thinking of it has to do with what you're talking about of wanting to be in control. They didn't.
29:01 Scott have ai generate a whole film for them they wrote a script then they had to have a generate each scene individually right and then they had to put it into whatever video you know movie editor they use like stitched all together to make a thing for exactly that reason which is if you can't you can't really get it to do a whole three minutes all on its own you can only kind of get it to do a scene
29:23 Brad Right.
29:23 Scott And even then they had to do a lot to control so that each scene sort of came out similarly.
29:30 Scott An example was the main character was the reporter and they fed it a bunch of pictures of her to say this is what we want you to make the character look like so that she looked similar in each scene.
29:44 Scott And even then.
29:45 Scott she pointed she came out differently in many of the scenes and part of this little video on the wall street journal was they showed the movie and then she narrated well this is how we did it and she talked one of the things she talked about was like despite the fact that we vetted all these images of me it still came up with different pictures of me uh in different clips a similar similar kind of thing there
30:08 Scott which I guess over time we're going to figure out in 20 years it'll all get figured out but today we're at day one right
30:14 Brad Yeah, I mean, I don't think it's going to take 20 years. I think that these are the it's I think you're onto something,
30:20 Brad Scott,
30:20 Brad that the same type of thing that Joe and Justin and I and other folks that we've talked to are doing daily software development with this.
30:28 Brad We keep saying like, look, it can run off in its own opinion. It can get lost.
30:33 Brad It can lose its.
30:33 Brad loses contact it can and that's the same thing that they were dealing with with this little three minute film and so these things are not the
30:46 Brad nature of the task that it's being asked to do it has nothing to do with software it has nothing to do with creating an image it has nothing to do with creating a video it has everything to do with sort of the way that large language models work and so if they solve if we can solve the large language model
31:09 Brad problem of context and you know kind of uh getting too uh sort of like anchored to an idea or running off on a random if we that would be that's going to get solved like i i can see where
31:25 Scott Engineers
31:25 Brad the buck's
31:25 Scott do
31:25 Brad going
31:25 Scott that
31:25 Brad sure
31:25 Scott too,
31:26 Scott right?
31:26 Brad human beings do it you know that's like the nature like it's like what it's like what john said in in uh our recent episode about like you know
31:36 Brad You know, what does it mean to be intelligent,
31:39 Brad right?
31:39 Brad Well, it turns out that a lot of getting through life is just pattern recognition and correlation,
31:46 Brad you know, a lot of it.
31:47 Scott Mm-hmm.
31:48 Brad Yeah, I would interject with creativity.
31:54 Brad And that randomness that humans have is that creative aspect which allows us to be so awesome,
32:03 Brad but also so bad at certain things.
32:06 Brad So the story I have around that is a friend of mine works with helmets for fighter pilots,
32:13 Brad and he works with like the optics and stuff. And so he would be working with AI and fighter planes.
32:23 Brad And the reason, one of the reasons that when he's doing this is that why we should never put AI into a fighter jet is because once you put an AI in a fighter jet,
32:33 Brad another AI is going to figure out how to beat that fighter jet.
32:37 Brad Put a human in a fighter jet against an AI,
32:40 Brad the human will eventually figure out the pattern recognition.
32:45 Brad of that ai and the randomness of humans and he studied fighter jet fights and the randomness that these pilots do no ai comes up with because they do things that are against the norms of flying an aircraft in order to beat their opponents so an ai that's based on rules thou shall not do abc with this plane because you will break it a human's going to do that
33:13 Brad And the plane might break or it might not,
33:16 Brad but the computer is not going to go outside of those rule banks.
33:21 Brad So his analogy when he was talking to me about this is like, this is why a human in a fighter jet is so much more dangerous because we are creative.
33:30 Brad Whereas an AI machine is stuck in its context. Now you could argue that these hallucinations.
33:39 Brad can be a sense of creativity but i don't think that we are we think about it that like that way we've all talked about oh it starts hallucinating i shut it down well are we shutting down you know innate creativity maybe start you start going to that context so if it'll be interesting as ai develops and if it ever can gain i'll use the word consciousness of its hallucinations
34:06 Brad Then we're going to start talking about what does it mean to be intelligent,
34:10 Brad because it'll realize that it leaves context of task to be creative.
34:15 Brad And there's my philosophy
34:16 Scott Oh,
34:16 Brad for that.
34:17 Scott it's interesting you bring that up. I really love this thought about creativity,
34:20 Scott what it means to be creative,
34:22 Scott because I've been really...
34:23 Scott ruminating lately on just how stupid the AI is.
34:28 Scott And what I mean by that is somebody asked me to explain how it works.
34:32 Scott And,
34:32 Scott you know, the explanation people typically give it, it's like the, you know, the auto filling in your address on a website or something like that.
34:40 Scott And then you kind of go on to explain what's kind of probabilistically picking the next, you know, word or the next character that would get typed.
34:49 Scott But it occurred to me that it's actually even, like, to use the word conscious,
34:53 Scott even less conscious than that.
34:54 Scott I think it's actually just probabilistically picking the next, like, bite.
34:58 Scott It doesn't know anything.
35:00 Scott It's just a pattern of ones and zeros that's saying,
35:02 Scott given that I was a one here,
35:04 Scott what's the most likely thing to be next?
35:06 Scott A one or a zero,
35:07 Scott right?
35:08 Scott And so...
35:09 Scott It's really artificial.
35:13 Scott It's not intelligent at all.
35:15 Scott Yeah,
35:15 Scott it's going to bring it rapidly up to that average middle ground. You look at the distribution of the creativity rating of all the code that could come in,
35:23 Scott it's going to get you to that mid ground real fast, but it's not going to push anything forward.
35:27 Brad I,
35:27 Brad well,
35:27 Brad okay,
35:28 Brad I'm going to play the contrarian then.
35:29 Brad So let me, I
35:30 Scott Yeah.
35:30 Scott I thought
35:31 Brad mean,
35:31 Scott I was being the contrarian there, by the way.
35:32 Brad no, I'm going to be even contrary to you.
35:35 Brad So if most of life is just pattern recognition and repeating the, you know, like.
35:47 Brad Look,
35:48 Brad Scott,
35:48 Brad you and I have been doing this podcast for, you know, six months now talking to each other every week. And you know, you're like you, you knew John Helm a lot better because you worked with him for 20 years.
36:00 Brad But you've got a sense of what I'm going to say next.
36:03 Brad And but if you have literally read every word ever written.
36:10 Brad And you can remember every word that's ever been written.
36:14 Brad And you have literally watched every YouTube video that has ever been released because that's what open AI has done.
36:21 Brad They have scraped every YouTube video and every word ever written.
36:27 Brad And you have perfect memory of all of it.
36:30 Brad Then,
36:31 Brad and not just the words, but the sounds,
36:35 Brad the frequencies,
36:37 Brad the rhythm,
36:38 Brad and yes,
36:39 Brad it's all just tokens, it's all just ones and zeros, but you have infinite instantaneous recall of all of that,
36:45 Brad okay?
36:47 Brad Then you're certainly going to win every trivia game better than any human will.
36:55 Brad And you, so you know everything that's been known.
37:01 Brad Now you want to know what creativity is?
37:04 Brad Creativity is the inverse of everything that's already been known.
37:09 Brad But you know how to try something new is to exclude everything that's already been done.
37:14 Brad And so if you literally know every way that everyone's ever answered the question,
37:19 Brad how do I boil water?
37:21 Brad Well, guess what?
37:23 Brad There are many ways to boil water that no one's ever tried before.
37:28 Brad And those are all creative solutions to how to boil water.
37:32 Brad I'll one-up you on that, Brad,
37:35 Brad is that if you know, and this is where, in my view, AI is extremely scary.
37:41 Brad If you know everything that has happened and everything that we have known,
37:47 Brad you will, with some sense of certainty,
37:51 Brad be able to predict what will happen.
37:55 Brad And that's where the controversy on these AI bots and what they're saying in some of these YouTube videos really sparks because.
38:05 Brad You know, history repeats itself.
38:07 Brad Patterns are recognizable if you're looking for the right thing.
38:11 Brad And so you watch these YouTube videos of someone asking an AI,
38:14 Brad like, are there aliens?
38:15 Brad And AI goes,
38:16 Brad yeah, of course there are. Can't you see the freaking patterns out there?
38:19 Brad They've already been here and left.
38:21 Brad Or what's going to happen to the human race?
38:23 Brad Oh,
38:23 Brad you're going to be dead in X number of years.
38:26 Brad Like, that should be the real fear of AI,
38:30 Brad not only because it can recognize them.
38:32 Brad organize those patterns but and i'm not saying that it's going to get like the terminator consciousness to go kill all of us but it could lead us down that path of destruction
38:45 Brad by accident.
38:46 Scott maybe you had like bound it too as you're looking at it i mean like brad whatever you think might be creative i'm gonna go that's dumb i don't like that at all so then there's this idea of like aesthetic and you know almost scope to it too that i don't know if you can generalize that out on the macro scale like you if you you know focus it in then then you know you might see some of the things you're you're talking about but you know what we still have you know people and judgment involved and
39:11 Scott and you know preferences and promoting things out and figuring out how many of this thing are there out there that this you know and how many people are going to resonate with that I think it it gets to be really interesting because then you know guys like us can create you know these things much smaller down for much more focused and targeted people so I I wonder how much of that is is also you know wrapped up in the pattern of how many people have seen those things before you
39:39 Brad Well, this is a pretty fascinating conversation for sure.
39:41 Scott Yeah, I mean, it's making you just this and the conversation we have with John Helmbrecht. It's like, how quickly does AI go into philosophy?
39:48 Brad Always.
39:50 Scott I blame Justin.
39:51 Scott He brought us there.
39:53 Scott Justin,
39:53 Scott you talked about, you know, destroying us,
39:57 Scott destroying humans,
39:58 Scott destroying Earth, whatever,
39:59 Scott I don't know, destroying.
39:59 Scott stuff let me come back to this thing from the wall street journal or something that one of my kids sent me this week they sent me a video that someone had created ai using ai of people complaining about people making stuff with ai and it was fairly realistic i looked at it on my phone so i couldn't get that good of a view at it but um you just don't know what's real anymore or what you can believe
40:24 Brad I think that this is a very legitimate concern
40:28 Scott Yeah.
40:28 Brad that is powerful enough that even if you train everybody to be skeptical of what they see on video,
40:35 Brad we could probably get everybody convinced that pretty much anything you see on video could be faked and it will be indistinguishable from reality.
40:45 Brad Well,
40:45 Brad that means that we can't trust anything unless we see the person in person.
40:51 Brad And so,
40:52 Brad you know, it's interesting.
40:53 Brad Maybe there's a version where this all forces us back into the real world where we're actually,
40:59 Brad you know, shaking hands with each other.
41:01 Scott Mm
41:01 Brad And, you
41:02 Scott -hmm.
41:02 Brad know, what did you say once before on an old episode,
41:06 Brad Scott?
41:06 Brad There's just something about being in person with somebody.
41:09 Brad I don't know. Is it a smell or something?
41:10 Brad Something like that,
41:11 Brad you
41:11 Scott pheromones
41:11 Brad know, hair
41:12 Scott are real yeah
41:13 Brad bones are real, right?
41:14 Scott no
41:14 Brad And AI can't fake that.
41:16 Scott i can't fake that shit but no i mean you get to the point where can't so i would ask the question can ai distinguish ai generated from reality and that's going to be like a pattern recognition like someone should write that out the uh
41:32 Scott Because how much of this stuff that we see is AI generated versus non-AI generated?
41:39 Scott So if you could run that on Facebook and the AI app like highlights everything,
41:44 Scott you get a flag feature,
41:45 Scott AI generated, AI generated.
41:47 Scott That'd be kind of cool.
41:48 Scott That'd debunk a lot of stupidity out there.
41:51 Scott But,
41:51 Scott you know, is it smart enough?
41:53 Scott At what point does the AI...
41:55 Scott out fake the other AI and so there's that experiment that was done where they took two AIs and they wanted them to talk with each other over something in I think they were using English and within like what a couple minutes they switched over to a different language that no one understood so that they could communicate quicker basically gene language and so you like at what point do these things start
42:21 Scott communicating outside of our realms well i don't know exactly
42:25 Scott It's kind of, it's a mind bender.
42:28 Scott I comfort myself by believing we can just unplug them.
42:32 Scott i'm there with you like deal shut just turn it off
42:35 Brad I think that at the risk of like.
42:38 Brad Getting off the topic of the philosophy for a second, I feel like I want to circle back and say that I am incredibly bullish about the moment that we're in and the fact that, you know.
42:52 Brad the the two guys in a garage type concept of something that you're both very passionate about you're both technical enough that you know kind of right when you see it you know and you are able to move very very quickly and build something uh compelling i think that right now we're in this moment and i don't know how long this moment's going to last but i believe that we're in this moment where um the the the
43:17 Brad Capital costs of entrepreneurism are falling away
43:24 Scott Wait
43:25 Brad because of what generative AI lets us do.
43:28 Brad And I think there are many.
43:31 Brad There are many benefits of that for people like yourself that have an idea,
43:36 Brad are passionate about it, have the time and energy and have enough acumen, whether that be technical or business or a combination of technical and business acumen to know what to run towards that you're passionate about.
43:50 Brad And then you just have this army of AI that can help work so you don't have to.
43:56 Brad You don't have to do that. So that's a positive thing.
43:58 Brad I think that's amazing.
43:59 Brad And I have no
44:00 Scott Yeah,
44:00 Brad doubt that that's very real.
44:04 Scott yeah.
44:04 Brad And the other side of that coin is that means,
44:10 Brad well,
44:11 Brad larger organizations.
44:15 Brad really probably don't need nearly as many people as they have right now and they certainly won't need as many people as they have right now you know in a year in five years in ten years
44:28 Scott Who's the CEO of Nvidia?
44:30 Scott You know what I'm talking about.
44:31 Scott I don't know.
44:33 Scott He said that.
44:34 Scott You know, in X number of years, you won't need as many computer engineers.
44:40 Scott I think he's right.
44:41 Scott I think he's very right.
44:42 Scott Computer software, software engineering,
44:44 Scott don't need that.
44:45 Scott But you will need biologists, physicists,
44:49 Scott scientists who understand what they want programmed.
44:55 Scott The art of programming is going to become less and less.
45:00 Scott required human driven activity but what do you program what do I want this thing to do it's not going to create itself and then how do I as a human interact with it that AI is not going to figure out because of our randomness and our creativity at least it's not gonna figure it out yet so I firmly believe in that
45:25 Scott I sort of feel along those lines, I sort of feel like to some degree what this is doing with generative AI is doing for programming is a continuation of something that's already happening,
45:35 Scott which is just continuing to abstract the machine from the programmer,
45:40 Scott right? We started out, we,
45:41 Scott the industry,
45:42 Scott started out, you know, punch cards and machine code and then got to compiled languages and then got to scripting languages and now we're at.
45:50 Scott kind of the spoken or the English language
45:53 Scott English
45:53 Scott to
45:54 Scott language another
45:55 Scott tell the machine what to do and it still gets still gets turned into code that gets to compile that gets to compile that gets compiled all the way down but it's just moving further and further up the stack
46:07 Scott layer
46:08 Brad Well, Justin,
46:09 Brad one of the things you were saying then,
46:10 Brad Brad,
46:10 Brad to what you're talking about for, you know, the cost of startup,
46:14 Brad the cost of all of these things. I mean, software.
46:15 Scott software-based businesses are among some of the most expensive ones to start and they take the most amount of time to get from idea to market and I think
46:26 Scott Maybe these larger companies may not need software engineers to the same degree,
46:30 Scott but you know, these are still smart people that are going to go out there. They're going to have their own ideas.
46:34 Scott And so we may see, you know, bigger proliferation of smaller companies of more ideas and things out there because it just doesn't need to be centralized anymore.
46:42 Scott Right
46:42 Scott And so that's kind of how I think about it is, you know, you don't need to have this massive concentration of capital,
46:48 Scott of hired people, of architects,
46:50 Scott of machines.
46:52 Brad all in one place it's more accessible to everybody else and so they can kind of you know do do their own thing and have access to all this merely just how much of this did we do in you know the various you know vendors free tiers before we you know started paying for some of it like a lot
47:08 Scott Yeah.
47:10 Scott So that leads to another question.
47:11 Scott I met a guy at a conference,
47:13 Scott South by Southwest,
47:14 Scott actually in about 2007,
47:16 Scott 2008, 2009,
47:18 Scott somewhere in there.
47:19 Scott I wish I knew his name. His first name was Jeff and I remember his face.
47:22 Scott He was a marketer.
47:24 Scott And I think I remember he made this comment to me that stuck with me all these years, which is the one thing the computers can't replace is the marketers.
47:32 Scott I thought as an engineer,
47:34 Scott that's a strange thing to say,
47:35 Scott but it's actually kind of getting to that point where the software is getting very easy to make, but you still, as a business,
47:43 Scott have to go out and figure out how to appeal to a customer, how to get them to feel moved in such a way that whether it's through the head or through the heart to want to buy or use the thing that you've got.
47:57 Brad I have a ton of respect for high quality marketers that I've worked with in my career.
48:02 Brad So any of my friends who are high quality marketers that hear me say this out loud on this podcast,
48:08 Brad I want to apologize up front.
48:10 Brad But no,
48:11 Brad the AI is coming for their jobs too.
48:13 Brad It's all about,
48:15 Brad it's all about,
48:16 Brad because how many books on marketing have been written?
48:20 Brad And how many people have spelled out the story of how to tell a story and the story of connecting to people?
48:29 Brad And this is all about language.
48:31 Brad It's all about stories. And so.
48:34 Scott look at all the influencers and podcasters and it's it's uh tiktok is a huge marketing tool you got to do is have your your flood it with your video
48:46 Scott your brand.
48:49 Scott I think that it's going to replace like, I don't want to downplay the human interaction,
48:56 Scott but humans can no longer do in technology and business what we have done in the last 20 years. Every job that except maybe the C level and the VP level,
49:08 Scott maybe the director level is drastically going to change.
49:12 Scott Because you don't need a TPM to wrangle five developing managers to build one thing anymore.
49:22 Scott Like you can lower the people cost and increase your iterations on your product.
49:32 Scott drastically if you use these tools
49:34 Scott But I think you can increase the expectations,
49:36 Scott too.
49:37 Brad By the way, Scott,
49:38 Brad didn't you say you want to see the app that's the AI CEO,
49:43 Brad right?
49:43 Scott I tried to buy the domain name CEO.ai and somebody already owned it.
49:49 Scott Yeah, I was going to try to make a CEO AI that you could just ask the questions that you ask a CEO.
49:58 Scott go and see what there's been a lot of books written about being a ceo too right so it shouldn't be that hard you're the ceo of a scuba diving app startup what should what should your marketing strategy be
50:09 Brad If only you had thought of getting this domain name like 20 years ago when it was first purchased,
50:15 Brad I bet.
50:17 Brad Anyway, this has been a fantastic conversation.
50:20 Brad I love the story,
50:21 Brad Justin and Joe.
50:23 Brad I know that you're super passionate about this.
50:28 Brad You've made a lot,
50:29 Brad amazing progress in the short time you've been working on this.
50:32 Scott Well, guys,
50:33 Scott thanks for coming on here today.
50:34 Scott I have to tell you something.
50:35 Scott I'm inspired.
50:36 Scott This has made me want to just go start writing some app or some startup.
50:42 Scott Just from scratch and see how that goes.
50:45 Scott I've been a spark. You've lit a spark today.
50:50 Scott I appreciate
50:50 Scott Oh,
50:50 Scott that.
50:51 Scott yeah.
50:51 Scott Cool.
50:51 Scott That was right. Yeah, my wife might not like that.
50:54 Scott I said actually my wife's incredibly supportive.
50:56 Scott So but you know, she just might not want me to quit my job to go write, you know, some sort of app.
51:01 Scott But she's so wonderful.
51:04 Brad That's what I think, Scott,
51:05 Brad is that, again,
51:06 Brad like you did a great job of wrapping up the episode and I'm going to make it drag it on a little bit longer by 20 more seconds and say.
51:13 Brad That's the thing about this is you don't have to quit your job. If you've got a job, it's able to go so fast. It's able to make you so efficient that only a little bit of time that you can carve out out of your day is as if you spent all your time working on
51:30 Scott That
51:30 Brad it in
51:30 Scott is true.
51:31 Brad the
51:31 Scott That
51:31 Brad old world.
51:31 Scott is true.
51:32 Scott Mm-hmm.
51:33 Brad So it's
51:33 Scott Yep.
51:34 Brad pretty amazing stuff. Yeah.
51:35 Brad Thanks a lot, Joe.
51:36 Brad Thanks a lot, Justin. It was fantastic talking to you.
51:39 Brad Let's go out and ski next weekend.
51:42 Brad And by the way, it's in the middle of summer.
51:45 Brad Everybody,
51:46 Brad yes, we ski all year round.
51:50 Scott Yes, I'm going for a little Tomo.
51:52 Brad There you go.
51:53 Brad Awesome.
51:53 Scott Wow.
51:54 Brad Thanks a lot, guys.
51:55 Brad This episode of How Many CTOs Does It Take Podcast was edited by Saffron Epigow. Our theme song was AI Generated by Brad.
52:03 Brad Please follow us on all the social media platforms at at HowManyCTOsPod. You can find us on YouTube,
52:11 Brad LinkedIn,
52:11 Brad Instagram,
52:12 Brad Facebook,
52:12 Brad Threads, X,
52:13 Brad TikTok,
52:14 Brad and Patreon,
52:15 Brad or at our website at HowManyCTOsPod.com.
52:19 Brad And remember to rate, review,
52:22 Brad and subscribe on the podcast platform of your choice.
52:26 Brad Thanks for listening.