Setting the Tone: Building Trust Through Security
In this episode of "How Many CTOs Does It Take?" podcast, hosts Scott Porad and Brad Hefta-Gaub discuss the role of CTOs in setting a security-minded culture and provide real-life anecdotes of security breaches and incidents. The conversation covers critical frameworks like CIS, SOC 2, and tools like Vanta, Wiz, Drata, and AWS Security Hub. They emphasize the importance of incident response, continuous vulnerability management, and making security practices an effortless part of development workflows. The episode concludes by highlighting the urgency of adapting to new security challenges posed by AI and other technological advances.
Resources:
- How Many CTOs Pod: https://howmanyctospod.com
- Scott Porad: https://www.linkedin.com/in/scottporad/
- Brad Hefta-Gaub: https://www.linkedin.com/in/bradheftagaub/
#TechPodcast #EngineeringPodcast #DevTalks #PodcastForDevs #HowManyCTOs #Podcast #CTOs #CTOPodcast #ChiefTechnologyOfficer #Technology #Engineering #SoftwareDevelopment #SoftwareEngineering #TechLeadership #EngineeringLeadership #EngineeringCulture #TechDebates #Security #SecurityIncidents #IncidentResponse #SecurityCulture #SoftwareSecurity #SupplyChainAttacks #SecurityChecks #ComplianceAndSecurity #AI #Vanta #Wiz #Drata #AWSSecurityHub #CIS #SOC2
Scott: We don't put it on our balance sheets as businesses,
but security is an asset or liability on that balance sheet. And all those vulnerabilities that aren't fixed are really, you can convert them to dollars and put them as a liability.
Welcome everybody to yet another episode of how many CTOs does it take today?
Uh,
my name is Scott Poirier. It's always Scott Poirier, and I'm with my trusty co-host.
Brad: Brad Hefta-Gaub, who sometimes is Brad Hefta-Gaub.
Scott: Correct.
And today, the topic is security.
Brad sent me a text earlier this week. He said,
hey,
let's talk about security on the podcast this week.
Brad: Yeah.
Scott: I said, sure.
So, Brad,
what do you want to talk about?
Brad: Well,
I want to talk about...
Not just let's not talk about security best practices, although that will certainly come up as things we talk about.
But I'm just kind of curious about your experience in as a CTO, what have you seen as sort of the various cultures around security and technologies around securities and ways leaders we can make sure we're doing the right thing? And, you know, how.
What's the role of the CTO in security?
Scott: That is a great question.
Well,
I think the first thing that one needs to ask is,
does the security function like the information security function roll up into the CTO? Well, let's for the purposes of this conversation,
assume that it does.
And some companies it's separate because some people see that's a little bit like the
Fox guarding the hen
Brad: Yeah.
Scott: house, if that makes sense.
And so I would say the first role of the CTO in security is mindset or attitude or culture,
the tone at the top.
What is the tone that the head of engineering is setting?
And let me, before I go any further,
let me say this.
I am not a security expert.
I don't have any of the security certifications or anything like that.
But I have worked at three or four different companies where security for different reasons was important,
either because we had scale where it mattered to be secure.
We had external partners who wanted some sort of security.
sort of trust that their data was being handled properly or we were working where you and I work together in health care and we had confidential patient data so I'm not an expert but I've been around it and I've had to lead a security function on a few different occasions
So, step number one, tone at the top. And I would say tone at the top goes both ways. And what I mean by both ways is toward one's own engineering team setting the tone for the engineering team, but also out and up towards the senior leadership.
That it has to, as a CTO, it's your job to set that tone and educate and train and guide in both directions.
because you know I often say most people don't know what we do inside of engineering and they really don't know what we do inside of engineering when it comes to security but
Brad: Yeah.
Scott: but they will know they will know when there's a problem
Brad: Yeah,
exactly.
It's funny.
I just as I realized as I asked you that question and you started answering it,
I was like, maybe the better maybe the better place for us to start. First of all, love your point about tone.
100% agree with that.
Because I've seen places where, you know, to your point,
they don't care about it until all of a sudden it matters.
Right.
Like if you have a breach,
everyone's going to care about security.
But it probably was it's very it's it's it's probably the breach probably occurred because.
of the tone like it's very likely that we had a lax security posture because we had a unclear tone or a clear tone that was security is not important that we
led to the choices that ended up resulting in a breach.
So does
Scott: Yes,
Brad: that ring true to you?
Scott: that rings true.
Let me tell you, I've been involved in what I would call three real security incidents.
Okay,
but one was only one was really a breach. So let me tell you about these three because I won't be revealing anything.
One of them was when I was working at Rover.
This was not a breach.
It was a situation.
Rover was a two-sided marketplace.
So you had buyers and sellers who were communicating via messages.
And we had, I can't remember if it was fake buyers defrauding.
Sellers are fake sellers defrauding buyers,
but there was people were getting defrauded through the system. So there was nothing about our technology that was broken,
but it was to the end user,
they felt like something was wrong.
That was one thing.
The second thing was at another company,
not Rover,
this was when I was at DALI, which DALI got acquired by TaskRabbit.
We had an account credential stuffing attack.
So this is where some rogue actor has found
a database of usernames and passwords that's been leaked on the internet and they come and try them on your website and people who have used the same username and password on multiple websites,
then they're able to get into that account and they're able to do stuff, right?
Again,
our system was working. Someone used a legitimate username and password.
Brad: Yeah.
Scott: To access the system. Now, you could say, well,
we didn't have two-factor authentication,
which was true,
and we then thereafter introduced two-factor authentication,
and that solves that problem.
The third actually was also at Dolly, and it was a real breach. Now, the breach predated my time there.
by a couple of years that but the consequence was thereafter so what happened was we received an email from someone one day that said hey we have access to your systems and we want a bunch of bitcoin right
Brad: Oh, no.
Scott: and yeah that's right oh no and we were able to diagnose that
two key things here.
They had gotten access to our systems through an insecure password.
Somebody had a password like password123 and that let them get in.
And we were also able to diagnose that they,
whoever this person was,
had actually not been inside,
to whatever extent they were able to show us proof that they really had been inside our system.
But they had not been in our system for at least a year or two based on the proof that they were showing us. They were showing us database schemas and we were able to say, oh, we know when certain tables were added or deleted.
And so this is the time frame when they were in the vector that they got in.
That was a server that had this crappy password that had been out of commission for a long time.
So those are kind of, that was the real incident that I was involved in.
And I will tell you, you get real heart palpitations until.
You find out, oh,
okay,
they got in a while ago and they're not in right now.
So at least the door was unlocked, but now it's locked.
So that was a little scary.
Brad: And by the way, you just mentioned something, you used the I word incident.
And one of the things that definitely is on my mind about security is one of the things about having an attitude of security and an attitude and a sort of like a.
culture and systems that help you solve for security is do you have a robust incident response process do you have a documented
Scott: Mm-hmm.
Brad: incident response process do you have you practiced it have you had a chance to you know work those muscles and you know do you have a culture that's good at incident response because certainly any incident that's a security incident
It's going to be the type of incidents it's going to really lean on your culture of excellence and your high performance team dynamic because it's going to be stressful and there's going to be a lot of opportunity for finger pointing.
And have you established a great quality of ego-less,
retrospective incident management type team coordination? Do you have systems in place? Do you have systems in place for incident management in general?
Scott: Yes, now I'm able to describe these events succinctly, but at the time they're happening.
I don't want to make it seem overly dramatic.
There's bullets flying in every direction.
I mean, people are shooting each other. There's just a lot going on. You don't understand the situation.
You're like, what's happening?
Who's in our system?
And then you're talking to one of your people and they're like, wait, this CC2 instance,
we don't even have this instance anymore.
Like, you know, you're slowly unpeeling and like trying, you know, getting an understanding.
I forgot a fourth incident that I've been involved with.
this was an internal embezzlement yeah
this was a situation where again
Brad: No one will notice if we just take a penny per transaction.
Scott: kind of it wasn't exactly like that this was again at one of the market I've worked at three marketplaces so this is at one of the marketplaces I was at and somebody
There was a cadence,
I don't remember exactly the rhythm,
but let's just say it was once a week where workers got paid and there was like a cron job that paid these workers.
And somebody inside the company who was figured out that they could insert rows into this table that did the payouts.
And so it would pay
Brad: Hmm.
Scott: them.
But there was no check and balance on that.
The only way we discovered it was at the end of the year, you have to issue 1099s for all these people. And the person who was doing that was just kind of spot checking and just noticed like unusual payments, like either they were kind of large.
or they didn't have they would usually be there was a piece of the the idea of the person who you know there's like a a user id and they're like but this isn't a real user id these are fake like what's so what's going on they so if that person hadn't been double checking
Brad: You
Scott: we
Brad: wouldn't do it.
Scott: might have we wouldn't have noticed and then through quite a bit of investigation we were able to figure out it was somebody internally um and we were you know we got variety lawyers involved in that stuff and uh they not to say it made it better but this is what this is what the lawyer told us actually this will be good this will be a fun story to say look
In my experience, we were able to get ourselves in touch with this person who dealt with this kind of problem.
They were a former federal prosecutor who is now in private practice and we were just like, what should we do?
Like, do we call the cops? Like,
you know, what is, how do you handle this situation?
And he said, look, in my experience, when somebody is stealing from their company,
they're having money problems.
And those money problems are coming from a divorce,
gambling or drugs.
Brad: Yeah,
yeah
Scott: OK, that's probably most likely that's what you have going on here.
And that's what was going on. We had somebody who had gambling losses and was trying to cover it up from his spouse.
And this is kind of how he.
Brad: Yeah.
Scott: doing that anyway
Brad: So back to security.
So yeah,
Scott: okay so tone at the top I
Brad: tone at the top, obviously. What else?
Scott: would I'll offer two frameworks to our listeners and I'll say one thing overarching both of these frameworks
And I say this to people all the time.
Lots of times people,
when they go to address these frameworks,
they're just trying to check boxes.
Brad: Yeah.
Scott: And I try to, this is a tone at the top thing, I always try to say, look, what's actually important?
I don't care what this framework says.
I don't care what, you know, the control that we're supposed to put in place is.
What actually matters is that we're more secure.
That's what matters.
Brad: Right.
Scott: And in fact, I just had this conversation yesterday with a general counsel at TaskRavit.
He asked me the question we were talking about some security thing where there's a little bit of risk and he's like, he was trying to understand, Scott, are we just trying to check the box here or is this a real thing to make us more secure?
I was like,
no, this is, I want to do this because it will make us more secure. Like I'm not doing this to just check the box.
If you're doing it to check the box, you might as well.
not do it okay so what are my two frameworks one is a framework i invented and then another one is a is real world so here what's the what's the one i invented very simple three stages it's like it's like a castle you're the king of your castle brad secure the perimeter make sure nobody can get in is number one number two make sure everybody who does get in you know who they are
And number three, make sure the people who do get in can only go to the rooms of the castle that you let them go into.
Brad: Yeah, minimum exposure,
minimum.
Scott: Yeah,
so number one is,
you know, have yourself inside some sort of.
private network and have all the outside of that network locked down real tight number two is you know don't have shared user accounts and stuff like that in your network everyone you know have all that stuff on the surface and number three is role-based access control and if you do those things and part of securing the perimeter stuff like
you know all the different types of security things you can use ssh keys and two-factor authentication and all that kind of stuff and to me like if you follow that framework you're going to be pretty solid right
Brad: So I love it. That's all great stuff.
I feel like that's really speaking to,
we haven't really talked about the code.
So you're
Scott: Oh, yeah.
Brad: in the zone of access to systems,
access to information,
controlling the humans,
controlling the humans that you know about.
Right.
Scott: Yeah.
Brad: And I think you're right, like you're you're enumerating like all the best practices that we all hear.
Right.
You know, like, you know, actual authentication of the person who they are, who they say they are. And that's like quality passwords and two factor authentication and appropriately secure systems.
Don't ever write your own cryptography. Right.
Like and and then, you know, role based accesses, that's like clear.
Like we really need to know that it's you,
Brad,
and we need to know that Brad has access to do these things. Right.
As opposed to all of dev uses.
dev and just logged in with root and the root password is password um
Scott: Right. Yeah,
exactly.
Brad: and of course you know modern systems like all of our cloud providers uh all of them have you know user account management systems that allow us to do really fine grain controls there um and i think that i'll just say like uh
Because I've worked so often with startups and growth stage companies,
right?
Like one of the things that often you find is like in the early days, you're just go,
go, go and you're taking shortcuts all over the place, right?
And,
you know, you and I like we've show up after the fact, right?
And maybe shortcuts have already been taken. Maybe it's been years of shortcuts have been taken.
And so you've got a little bit of a.
figuring out what you what your exposures are can be you kind of come to it late right um and i feel like that mistake of maybe
i just turned on a bunch of privileges you know maybe i've just i've got these agents that do this work for me you know
that i've got keys stored all over the place right especially in today's now this is going to pivot into the software conversation right because a lot of times our software is built on privileged components talking to other components and negotiating privilege at the software layer right i kind of threw out a couple things there one we've got this like often we're confronted with this legacy of choices that were made that we may need to come and clean up
And then I'm also sort of starting to pull at the thread of what about software security?
So what are your thoughts on those two topics?
Scott: I got thoughts on those two topics. I want to say one thing before I finish a previous thought and then get into this.
I talked about two frameworks.
Brad: Oh yeah, you said your framework,
which is the...
Scott: That's my personal little framework,
and it didn't talk about software.
I'm going to come back to that.
The other is these standard frameworks like SOC 2,
ISO 9000 or whatever.
And the one that I've found is the best to start with, and by best I mean just easiest to start with,
is called CIS,
Center for Information Security.
already um they have uh what they call uh implementation groups so they kind of give you there's a lot of there's like 200 controls in this thing and they sort of say okay we'll do these 50 first
Brad: Yeah.
Scott: right so it's just a good it's a good starter kit and um the re so that's one of the reasons i like it and i also like it because it's a little more prescriptive it sort of just says do this thing right a lot of the
frameworks that are out there are more like guidance like you shall have some way to ensure that people log in securely right and so when you're getting this gets to your question about you're coming in a little bit later i tend to start with that ig1 i'm sorry cis and then you've got the implementation group one it just provides a roadmap
Because you're overwhelmed. You got so much. You walk into a scenario and you're like,
oh, my God, there's no security here at all.
What do we even do?
And it just provides a, you know, I hate to reuse the word framework, but a framework or a roadmap for like,
OK,
well,
we just work our way down this checklist and one by one do all these things.
And I have a specific philosophy there,
which is it's better to go shallow and broad.
and keep on doing that as opposed to you can take one of these controls and you can go really really deep on it right it's it's better it's better to get a simple little lock on every door and then come back and put a better lock on each door and then a better lock on each door as opposed to putting spending all your time to making sure door number one is rock solid but doors two through 20 are wide open
Brad: So for people's benefit,
I just pulled up the CIS IG1 framework and I'll just like read through.
through kind of their various controls they have just as an example to your point about how you could go deep on these or you could go broad so like control one is inventory and control of enterprise assets you can imagine how like yeah do i have
Scott: inventory of all my control of my enterprise assets.
Inventory and control of software assets.
Data protection.
That's control three.
Control four,
secure configuration of enterprise assets and software.
Control five,
account management.
Right, do I have accounts?
Control six,
account control management,
access control management.
That's the roles-based control,
right? Control seven,
continuous vulnerability management.
control eight audit log management control nine email and web browser protections and there's 18 of these I'm going to stop here at nine but malware data recovery network infrastructure network monitoring security awareness
service provider management application software security incident response management and pen testing so yeah like you can imagine that all of those you can go each of those each of those you could go super deep on but you're going to way more benefit if you do if you do something for everyone
Brad: And those are actually called control families,
and within each of those that they have.
many controls underneath there i think the total
Scott: safeguards maybe underneath or something like that
Brad: i think they do call them safeguards yeah uh i believe there are 100 about 150 total controls or safeguards in that framework in those 18 families and then they've broken those up into those three layers implementation group or ig123 yeah and so my strategy is to go shallow and get a little everything just like painting the golden gate bridge you paint one coat
and then just go back over and paint another coat you keep on painting coats endlessly because by the time you get to the one end of the golden gate bridge the other end is already faded you need to start over
Scott: Yeah.
Brad: okay so when we talked about we wanted to get into software yeah so two thoughts about this i first went to humans and it's because i was influenced once
The first job I had where I had to do security,
we hired someone,
and this was a guy who came out of some bigger companies.
I think he'd been at Walmart and some other places, and my inclination as an engineer was to start with the software.
And he was telling me, you've got to start with user access,
and I didn't understand why.
And he's like,
well,
look,
I've been doing this a while,
and the reality is most of your problems.
come because a human made an error Yeah, right
Scott: right
Brad: and you know it's not as frequent that someone finds some clever way to exploit the software that you wrote it's much more the social engineering that people do to get someone's password out of them or that someone used a weak password and then when they used a weak password and somebody got in you didn't have the roles and set set the right way so
I learned that from him and then that sort of changed my mind. My mindset was always like,
oh, protect against SQL injection attacks and stuff like that. That's where I sort of started out in this journey.
And then I learned like,
oh, it turns out most of the problems are human.
And then those four incidents that I told you
Scott: All
Brad: about,
Scott: human.
Brad: all human, right?
Scott: Yeah.
Brad: And so, and in fact, I've been at some companies where we have done, we have done.
You know, we phished our own employees. You know, that's when you do something to try and get an employee to give you their password.
We phished our own employees as part of our training.
And one place I had 26% of the employees gave up their password.
Scott: Yeah.
Brad: Right.
I mean, this is, you know, this is crazy.
And this was the classic, you know.
hey we need you to go take this survey or something click this link and the person clicks the link and we set this whole thing it was a page that looked like you had to log into google but you know the url was like google spelled wrong or something like
Scott: Right.
Brad: that and then they typed in their username and password to log into google and boom you know we have it so um it again comes back to that tone at the top and setting the culture okay but then you come back to the software
Well, let's talk about the software stuff.
My experience, and this is not again,
not my biggest area of expertise. And by the way, it is all changing with AI.
I mean, it's like a whole new game.
The things I lean on are one is coming back to that perimeter is your network setup.
securely basically if you're running a web app or you just locked down to port 80 right do you have all these other port 443 but like is that 443 is that your sso port i can't remember other people remember that stuff these days better than i do it uh do you just have a lockdown remember i told you about that incident we had where there was this one ec2 instance with a lousy password that was at a company that was not using a network and all of their instances were on
the public internet
Scott: I
Brad: you
Scott: see.
They didn't have any kind of an application load balance or any kind of a...
Brad: nothing protecting it so they had this one container that someone had set up probably to experiment with something and then that person set a weak password and then it was out there and someone found it who knows how they found it so um that's one piece and then the other piece is uh standards-based like you know
I've been at companies that have had their own homegrown authentication systems and it's just like what are you doing these days man
Scott: Yeah.
Brad: like we should not be you unless you're
off gov or off zero like you're not getting paid to have good security to make your own security system so like you know rely on a third party for this kind of stuff and whether it's all the stuff that matters which is databases and money and users like that's all stuff where you can find whether it's you know
open source or commercial systems to use where you got a community of people to help you there.
Scott: So I love that point, and I think that's a great segue actually into one of the other topics that we've become a little bit more aware of, and that is supply chain attacks.
Brad: Yeah.
Scott: Right.
And in fact, we just had one recently,
React had a supply chain attack with Next.js and React.
So it's like pretty big, common,
large,
large open source products that end up with a, you know, significant vulnerability in a supply chain that a lot of people.
You know, so there are best practices for making sure you're ready for that kind of a thing,
right?
Like software inventory,
you know, like do you really know what software is being used to build your software?
With NPM packages these days, it's just like so easy to be like,
oh, I need a new, like what was the thing that was like, there was one that was not even an attack,
but there was like a famous one like a couple years ago where this guy had this open source package that was for.
for colors and everyone
Brad: Yep.
Scott: like used that color package and then one day it was just like he took it down it's like no one's software could build anymore because you know he got rid of the package
Brad: Well, you talk about NPM,
you know, you type in NPM, I can't remember the words for NPM at the moment,
but like NPM install color package and you watch it sit there and install 100 dependencies and you're just thinking to yourself, what is this stuff?
Scott: Right.
Brad: So, okay,
I said there were two things on the software side,
but I fibbed and I actually have a third thing on the software side.
Scott: Yeah.
Brad: the system you know npm audit in the npm space will tell you uh where your packages are at in terms of security vulnerability stuff like that make that a part of your build and if you've got a critical or if you've got a high don't let it deploy you know set in a set an sla for your team um it says look any high vulnerability this is how i like to set these slas i like to
I have, I'm going to go deep here for a second, when I say deep, a little bit of a tangent,
whether it's a bug or a vulnerability or a performance issue,
any of these kinds of things.
Here's how I think about it. You've got critical,
high,
and medium luck.
Critical is, actually you have one on top of that.
You've got the top one, which is above critical. Let's call it red alert.
Red alert is stop, drop,
and roll.
Stop what you are doing now and work on this thing until it is fixed day or night.
Okay.
Critical is, hey,
break your current sprint to work on this.
You had some stuff in your sprint,
pull this thing into your sprint,
and disrupt your sprint to make it happen.
High is...
this needs to go in the next sprint.
Medium is this needs to go in the next sprint or the sprint thereafter.
And low is you need to do it within the next 90 days.
And when it comes to low,
if you don't do it in 90 days, we're just going to close this ticket.
We've decided that if it's not important for us to do it in 90 days, we're just going to get rid of this ticket.
So those are priorities.
You can use those priorities for bugs.
for performance for vulnerabilities.
So step two there is for each of those categories.
you decide on criteria now vulnerabilities is the easiest because something like NPM audit is telling you
Scott: Yeah.
Brad: and you can just map it right oh we got a high high high severity vulnerability that's going to be put into our bug system our ticketing system JIRA be marked as a high priority and that'll just go in the next sprint and then it's really easy to build a report in JIRA which shows you things that are out of SLA
And as a manager,
you can just manage against that.
Scott: By the way, I love what you're saying here.
I want to pull on a thread, though, that I think you're ending up hinting at,
which is.
And you mentioned this at the beginning where you said, hey,
you're not just setting the tone to the engineers and the teams below you, but you're also setting the tone up and out to your peers and to the other leadership team.
And I think that that is hinting at a really common thing where.
How do you change the philosophy or how do you help manage your stakeholders into thinking of security as more than just a cost center?
Because certainly when you say,
hey, I've got these tickets and you just use some great language like critical, we stop what we're doing to it breaks our sprint.
We work on it.
Hi,
we got to get it into this next sprint.
Right.
Like you're making statements that the average product owner,
if they don't have a security mindset, don't believe in security mindset,
they're going to they might bristle at the idea that you're saying these things trump their feature ideas.
Brad: They do bristle. And let me tell you, they bristle in two ways.
I mentioned that you can apply this also to bugs and to...
Scott: performance
Brad: So, yes,
product managers can really bristle because when you start to say,
hey, any endpoint that's slower than, let's say,
500 milliseconds,
that's going to be a high priority bug. And anything that's slower than a thousand milliseconds is high priority and anything that's slower than 500 is medium priority,
then they're going to say that's filling up their sprint. And when you say,
hey, anything that's, you know, we're going to set some standards for what's a high priority bug and a medium priority.
And they say, well, gosh, if we set those standards,
my sprints are going to get filled up with my sprints. Of course, the sprints are going to get filled up with all these bugs.
And do you know what I'm going to say?
I'm going to say, well, then we need to do better QA, don't we?
Because we're letting all these things out.
But coming back to the security point,
I would say I try to impress upon them a few things.
One is that.
We don't want to have a breach.
A breach is incredibly expensive.
I've done some cost analysis on some of these breaches I talked to you about before.
And when you start to get the lawyers involved,
and I've never been in one of those cases where we've had to disclose to our customers,
but you've received a letter in the mail just like I have,
which says so and so company was breached. I mean, these are hundreds of thousands, if not millions of dollars in expense.
Scott: They've
Brad: And
Scott: gotten hundreds so of
Brad: that's
Scott: millions of
Brad: real.
Scott: dollars.
Brad: Right.
That's real.
The other thing I try to impress upon them is we have to have a culture of maintenance because when we get behind on these things,
it's not a hole we can really ever dig out of.
Scott: Yep.
Brad: So, I mean, OK,
do you want to blow off this high?
severity vulnerability a sprint okay blow it off a sprint but if you blow them off six months then you're going to have 20 of them and it's going to take us six sprints to get it figured out right so we have to just always stay up with it it's just a cost of doing business and the other thing i try to impress upon people is we don't put it on our balance sheets as businesses
But security is an asset or a liability on that balance sheet.
And all those vulnerabilities that aren't fixed are really, you can convert them to dollars and put them as a liability on.
Scott: Yeah, they're almost like what Jon Helm talked about.
They're like.
Naked calls,
right?
Brad: Yeah.
Let's assume if one of those high severity vulnerabilities is exploited, it's worth half a million dollars.
Let's assume that one out of a hundred of those is going to happen.
So that means that.
It's $50,000. That's a $50,000 liability on average on your sheet, on your balance sheet.
And so we can decide as a business if we want to carry those risks.
And lots and lots of business people will say yes.
But the first time they go through a real incident,
then they will regret it.
So I try to do that.
And then the other piece, and this is the last thing I'll say is.
is when you set it up is just create that culture of oh it's a high and we put it in the next sprint then it just becomes a non-topic any longer it's a little rough at first to get people to buy into it but once you get them to buy into it then nobody thinks about it anymore it's just a
cut course matter doing
Scott: Yeah,
Brad: things.
Scott: I think that's fair.
I also think and this I kind of opened the door a little bit to the question of like, what do you do when you kind of arrive and it's and you've got like a legacy of problems,
but I'll just take a moment here. If you have if you are lucky enough to be starting from scratch.
Then this is something that you can build into your processes to be completely automated.
Like you mentioned,
you know, you just make a CICD rule that like requires that there are no vulnerabilities.
And, you know, the current state of the art in static analysis is really quite good.
And so, you know, you have all these different things. It's like, you know, DependBot can tell you if you've got vulnerabilities in your dependencies,
you can run vulnerability scans.
You can be running static analysis against your code base to see that you're accidentally doing stuff like being vulnerable to SQL injection or being vulnerable to API injection of various sorts. It's like.
The static analysis tools are pretty damn good for that stuff right now.
And so set it up as part of your build process to just not let you check in code that has the problem.
Brad: Exactly right.
The build process,
and I'm a big fan of something called danger system,
danger system, something like that,
which is a way to build rules into your build process and your deploy process.
It is really where you can put your guardrails, right?
Whether it's everything from like don't deploy a critical kind of thing or no PR is bigger than 300 lines or no PRs after 430,
no pushes after 430 on Friday.
You know, you can put all those rules in your build system.
And when you build a culture that does that,
then it's basically a straight line from incident retrospective to let's put a rule in the build to prevent that from happening.
I'll put it again.
Scott: Yeah, totally.
Now, you also mentioned the notion of checking boxes and things like that. And I'll just make a quick, I'm not going to give a sales pitch for this product, but I will tell you, I just recently started working with a product called Vanta.
And Vanta is essentially an automated system for compliance regimes.
And so it actually,
so I'm using it with one of my clients to do SOC 2 compliance.
And it's so far so good. It's a pretty powerful system.
So you you actually hook it up to your GitHub and you hook it up to your JIRA ticketing system and you can hook it up to your various cloud providers. And it does analysis for you to see if you're meeting all the controls.
Brad: Mm-hmm.
Scott: And I think that and so,
for example, a good example of this is, you know, it will analyze your controls around your logins and your credentials.
Do you have do you have appropriate does everybody who has.
has an account in github are they a mapped person on your accounts in your you know your google workspace are they are is there a correlation there and and do you have you know checks there these are examples of the types of controls that you know would be in your cis you know ig1 but
A tool like Vanta,
and there's probably others out there that do the same thing. I actually, in fact, know there are others out there that do the same thing.
They actually automate that control checking,
and I think that makes it possible for, makes it
Brad: Yeah,
Scott: it a lot easier.
Brad: let me offer two other tools that are out there.
Three other tools that are out there, actually.
We use a tool called Wiz,
which does a lot of that.
We use a tool called Drata,
which is a competitor of Aventa, and then built into AWS for any of you who are using AWS,
and I would assume that Azure and Google Cloud Platform have the same thing.
Within AWS, it's called AWS Security Hub.
You just go in there and you say,
run a check, use this framework to run a check for me, and they've got dozens of, they've got, you know, SOC 2 and HIPAA and you name it, they've got them all in there,
and it'll just run through whatever AWS setup you have and tell you how you're doing against that.
Oh, here you've got this bucket exposed publicly.
Oh, here you've got this container that's not part of the VPCs, etc. And so...
It gives you then that's your kind of your roadmap for getting started on what to fix.
Let me offer one other framing and I'm going to say this is what I'm about to say is a little bit in evolution in my mind.
I used to frame it like this. In most of our companies, you have to think about we actually have two technology environments.
We have what we call our production environment,
which is often an AWS or Azure or Google Cloud or something like that,
which is like our web service that we.
serve people with and then we have our corporate environment which is like our work in our laptops our workstations our google workspace or whatever you use all those kinds of things and so you have to cover both and i tend to when i look at these control frameworks i tend to apply it separately to both of them because it's easy to say okay we'll do user access control oh sweet we got aws sort it out i was like okay well that's only half
That's it. You got this stuff over here.
Where I've started to evolve in my mind of that framing a little bit lately is I actually think there's a third environment.
It's a portion which bridges the two and it's the data environment.
Right. Is that we often have data coming from our production environment or data coming from our corporate environment that is going into, you know, some sort of data warehouse kind of thing, which is a third space. And sometimes you own that third space and it lives like in your cloud place,
Scott: Yeah
Brad: your cloud place. That's funny.
Scott: your cloud environment or sometimes there's a third party like a snowflake or something like that but it's kind of this third piece that you have to think about that stuff as well CIS does have a control family about data but again I tend to think about the control families or the rows of the different controls and then you have three columns your corporate your production your data and the only reason I do that is to help people think
think more clearly about all the areas we have to
Brad: Or
Scott: cover
Brad: the gaps.
Scott: so that we're all yes we're covering all the ground and that's kind of the three main the three main areas that tend to cover it all we've
Brad: Totally.
Scott: gone on a huge tangent from software i don't know how we got from software to talking about
a control environment matrix.
Brad: Here's how I tie those things together in my mind is that typically as CTOs, right,
you're typically working for an organization that is building software.
And I think that you're right that it's easy to get hyper focused on that part of it.
And to leave out, you know, like your your colleague said,
you know, the human factor,
right,
or to leave out the corporate factor or to leave out the, you know, like a, you know, like a good example of this would be like, oh.
You know, if I have let's say I'm doing a good job of not checking in secrets into my code base. Yay me.
But my developers have, you know, secrets on their development machines that, you know, are used for their development environment.
Right. Well,
now that means that that that machine and that individual corporate user is an attack vector.
Right.
Scott: Yeah,
Brad: And so you could say, oh,
I'm doing such a great job of doing the software part securely.
But then if you've missed your if you miss coverage on the attack vector of humans that work for your company,
then you're not really doesn't matter that you did all that work to secure the software.
Scott: that's right.
You know, we were talking about the software side and I didn't think of it, but I should have now that I hear what you said.
One of the most basic things on the software side is a security, a secrets manager.
Brad: Yeah.
Scott: You're not using, you know, one password and AWS secrets manager or whatever you want to use. We use a system called Doppler at our company.
Like you're just missing one of the low hanging fruits for keeping things more secure.
And there's a lot of different choices out there you can use.
One of the things I did run into,
I was at a company where we had a lot of smart engineers,
and they often wanted to use the best tool for the job.
And so there was some secret manager they were using.
When they started to put in a secrets manager,
they started to put in this really good one, right?
But it didn't have the ease of use and developer tooling around it.
So then it just took developers a long time.
And so they
Brad: So I know what happened.
Scott: just skipped it.
Brad: They skipped it and they put the secrets right
Scott: Yeah.
Brad: in the code.
Scott: Yeah.
So this is one of the cases where I think you really do have to think about what is your engineering or your developer workflow,
because you want it to be such that it needs to be.
The cost-benefit equation,
it has to be easier to use the secret manager than to not,
because if it isn't, people won't do You it.
Brad: know, it's so funny you say that. I literally just heard a podcast with a company that's basically working on using AI to basically do dynamic.
pen testing against all these different you know surface areas that you're talking about and this is you know the founder of that company it's very much like a longtime security expert and was making that was telling a story about a time that he was working consulting with a CISO that basically said oh I want my secret manager to be really really hard to use and he was like what why would you do that and he's like
Well,
because I would like to try to encourage the developers to simply not use secrets, to come up with solutions that don't require secrets.
And he was like, that's not going to work,
buddy.
Like, it's not going to work.
Like your developers are going to, I mean, you can't avoid secrets like all of the tooling that we use nowadays,
all the API packages that we have to use, they're always going to have secrets involved in them. So to your point, if you.
And I think,
by the way, I think we could rewind this to the very top first comment that you made,
which is it's all about the tone that you set,
right?
And so if you set a tone that says,
like, look.
security matters and of course you're going to have to have software that talks to software and therefore you're going to have to have credentials for software and that means you're going to have to have secrets and that matters and we're not going to not do that we're going to do that right and that matters and then and then you have developers who will at least voice the concern like hey I'm trying to use the secrets manager and it's really hard to use can we find a better secrets manager that has better
dev tooling as opposed to like fuck it i'm just going to go around the policy agreed
Scott: Yeah.
Here's the last thought I'll share.
I mean, maybe we'll have more to discuss, but it seems like for me, this is the last thought that I have.
If you follow good security practices,
you follow the CIS framework or the SOC2 framework or whatever,
you do these things,
you're just following good software engineering practices.
And if you follow them.
It is work to set them up. That matrix I was just describing of your control families going down the rows in your environments.
So let's say you've got 18 control families and you've got three environments.
You've got 56.
Did I get that right?
Whatever,
you've got a lot.
That's like 56.
It isn't 56.
Three times 18.
I don't know what it is, but whatever it is,
let's just say it's 56 for the purposes of my illustration.
That's 56 projects you have to do.
And if there's three implementation groups,
that's actually 160.
big something projects
Brad: Big
Scott: you
Brad: number.
Scott: got to do it's a big so it's a lot of work it's a lot of work to set it up but when you do get it going it actually greases the rails of your software engineer um it just makes things go more smoothly because a lot of getting it set up is having um you know procedures and processes for how something get done and so you're just streamlining the work that you're doing
And so I see a lot of people see security in opposition to software engineering. I see security as, you know, accelerant or catalyst to software engineering.
And actually, I said that was less than, but I didn't say the last last thing.
This is going to be maybe the nerdiest thing I ever say on this podcast in
Brad: I'm I'm I'm
Scott: all of our episodes.
Brad: on edge
Scott: It's almost spiritual.
Information security and specifically information security frameworks is the grease that makes capitalism go.
It's what allows individuals and businesses to trust each other.
And if we didn't trust each other,
we would have a harder time transacting with each other.
And, you know, I trust that this dollar I'm going to give you is worth a dollar,
right?
I trust that this data I'm going to give you, you're going to keep secure.
And if all that stuff didn't happen, if all those trust frameworks weren't there.
You would only do business with people who you could see and shake their hand and think of how different our world would be. And so in a certain way,
I have almost like it's to me, it's almost like.
I'm a little bit in awe of it, whether it's our security frameworks we're talking about,
or you got the, what was that one having to do with public companies that came out after Enron?
Brad: Yeah,
Sorbain's
Scott: Sox, Sarbanes-Oxley.
Brad: -Oxley.
Scott: Yep. Or you got HIPAA with your health information.
You got PCI with your credit card information, all this stuff.
All these are the things that allow all of the actors,
meaning all the people in businesses in the economy,
to trust each other.
And trust is what makes the whole thing go.
Brad: I think the way to sum this up, actually,
Scott,
rewinding to the whole, we basically like summarize the whole podcast as trust is fundamental to being able to be in business.
And the tone that we need to set as leaders.
is that establishing good engineering practices will establish good security practices,
which therefore allow us to implement the trust and to demonstrate the trust that makes us an effective business.
Scott: Yeah.
Brad: And I don't think that you could say it any better than that.
Scott: No.
You said it well.
Brad: There you go.
Well,
this was fun. I think we covered a lot of ground.
I think we talked about some important things.
Obviously, there's way more that we could talk about it. Like,
I think that like this is right there in the in the mix of like, what do you do when you show up and there's been, you know, a fair amount of technical debt or security debt,
and then you kind of try to clean it up and you touched on many of those topics.
But I love the idea of if you can start from if you get to start with a clean.
clean slate then for sure put security in place right away just make it part of your just make it your culture make it part of your engineering practices and your life's going to be so much easier yeah
Scott: agreed and brad i'm going to go a little bit out on a limb here i don't think you'll object to what i'm about to say but for any of the 40 people who are listening to this because our listenership increased by one person this week brad
Brad: there we go nice
Scott: I would be happy to, and I think you would be happy to also take a call with anybody if you're struggling with this at your company and need some guidance.
Brad: Yeah.
Scott: I'd be happy to help out in part because I think good security serves us all as an industry.
You know, if all of our websites and all of our mobile apps and all of our software systems were flaky, nobody would trust this industry and we'd all be out of jobs and so on and so forth.
And so I think it's good for us as an industry.
Brad: And I think that's a great way to sum it up. But gosh darn it,
you made me think of one last thing. And I'm going to say this and then we're going to sum this up.
Just because a breach happens,
that's not the end of the story,
right?
Like we said earlier,
like having high quality teams that understand how to respond to incidents,
that's what that's that's like such an important part of this.
And so, you know, a part of that tone is like security matters. And also we are professionals such that when something goes wrong, we deal with it in a professional manner.
Scott: Absolutely.
Now, there's a whole topic we didn't discuss,
and there's another podcast from this.
And we actually should find the right guest for it because I'm not the right person.
AI is changing everything.
Brad: AI is changing everything.
Scott: You've got data getting exposed in ways that we never had it exposed before.
You've got agents crawling around your environment.
I was just talking to someone this week.
He was telling me how he basically gave.
some sort of AI agent root access to it was a problem in their environment he basically gave this agent root access and said can you help me figure this out and it figured it out and he was like it was amazing and so scary at the same time right
Brad: Totally.
Scott: and so there's this whole new world out there that to be honest I don't even have a concept of how to cope with but there are going to be agents that
Do the wrong like I've heard of them already agents deleting databases right and all this stuff and It comes back to that second plank of actually I've known it comes back to but the whole that's a whole other podcast We're gonna have to talk about
Brad: All right.
We've beaten this one. This one's a good one.
We covered this one.
Let's wrap it up with.
Tone is what matters.
Trust is what matters. And if you set the right tone for security and trust,
it's just the same as good engineering practices.
All right.
Well,
what do we say?
Scott: smash that subscribe button tell all of your friends and leave us a great review say this is these are the best the best podcast ever
Better than Joe Rogan or Mark Maron or any of those podcasters out there.
And also you can tell all your friends that you knew about it first.
Brad: You knew about it first.
That's right.
All right. Thanks,
Scott.
It was great talking to you. We'll talk to you next week.
Scott: See you, Brad.
Brad: This episode of How Many CTOs Does It Take Podcast was edited by Safran Epbegaard.
Our theme song was AI Generated by Brad.
Please follow us on all the social media platforms at at HowManyCTOsPod. You can find us on YouTube,
LinkedIn,
Instagram,
Facebook,
Threads, X,
TikTok,
and Patreon,
or at our website at HowManyCTOsPod.com.
And remember to rate, review,
and subscribe on the podcast platform of your choice.
How many CTOs Thank you for listening.