Welcome to Conversations with CISOs, Security Leaders and Technology Executives, where we sit down with the leaders shaping cybersecurity and enterprise technology. Today I'm joined by Christophe Foulon to discuss who owns AI risk, building an AI governance model that actually works. Christophe, thank you for the taking the time to join us today. To get us started, could you introduce yourself and tell us a little bit about your background and current role?

Yeah, so Christophe Foulon been in tech and IT for over twenty years, currently serving as virtual CISOs for several companies through an MSSP as well as providing consulting and advisory services for startups, additionally myself.

Excellent. So when it comes to AI risk, who should actually own it inside an enterprise and where should ultimate accountability sit?

Well, from risk it always comes back to who Who wants the AI, who wants the data, who wants the application, the risk owner always goes back to that person. So the risk owner ultimately always becomes the business. The business should always be responsible for ensuring that they're aware of what's needed to be implemented. to secure and facilitate their data.

Now they might have to triage with their counterparts in technology, in security to ensure that they're putting the proper protections in place and ensuring controls like access controls and other controls for security at rest and who ultimately ends up with access to the final product, but it's the business that is accountable for that in the end.

Okay. So when AI governance spans those different departments across the business, like security, IT, legal compliance, how do you prevent that shared responsibility from becoming no responsibility?

The owner becomes the risk owner no matter what. So they're the ones that decide who gets access to the application. How the risk flows down and when it comes to if it's a shared use application then it becomes a matter of the organization then has to centralize the risk owner to potentially BIT who then owns that application versus the business because it becomes a centralized service that IT and or securities offering for the lines of business. So they become the risk owner in that case, because they're providing the AI service for the organization rather than the organization wanting to bring in an AI service to be used by them.

So just a clarifying point, is this risk owner are these risk owners within departments or is there a single risk owner for the entire business?

Ideally you should have a single risk owner for the entire business. You can have risk stewards that help facilitate how data should be managed, how the applications should be managed, how processes should be managed down at the department level, but there should be one owner for the organization.

Okay. So should that be the CISO? Where does the CISO's responsibility around AI kind of begin and end?

Yeah. Like I said, it doesn't always have to be the CISO. If for example, the business is the one saying, Hey, I want to bring in Claude to use Claude as the service because I need it to enable my business, then it's the business that is the risk owner, not the CISO. The CISO is simply the one helping to facilitate that the business can be enabled and use that service in a secure way, and they're the ones that are ensuring that there's potentially no data loss and that the business is f following all the other policies and regulations that they need to follow, but the risk owner for that application still lies within the business.

So can you walk us through what an effective AI governance structure should actually look like, particularly in maybe a like a regulated organization, highly regulated organization?

So you would tend to have a governance structure that sits up at the ELT level. You'd have a risk owner. You typically have a governance committee that decides on whether an AI application is right for the organization, deciding whether it meets all the organizational needs, whether there's duplicative services already provided, whether the risk of using that that AI solution is too great based on previous exposures or the inherent risk of using that solution, that the potential business impact of enablement is sufficiently great enough to take the risk of using that. The AI governance committee would typically be the ones to help make that decision.

They would also then help with the AI architecture around setting the these are the controls that we need to have enabled for AI within our enterprise. Whether that be hey, these are the DLP rules that we need to have followed. These are the secure sign-ins that we need to have followed. Or you can only use it through API, you can only use it through this approach, managing the the risk that of using that application through a defined approach because that allows them to control the risk of that application and data exposure from that application.

So that AI governance committee and that architecture that gets defined there sits up at that level and then the risk owner from there has to then go out and work with IT, their developers to go implement the solution based on the architecture model and implement it within their line of business.

So how should organizations as they're coming up with this kind of AI governance structure, most organizations are already gonna have AI that's been used or being used by employees. How should they manage that as they're developing their governance structure and policy?

So start with defining the use cases for AI. Whether if you say for example the organization has a default AI that comes with their productivity suite. They get a huge discount for using that AI and therefore inherently they might be biased to use that just for cost reasons. So that might be the default AI that they would push most of their employees to use. But that might not be the only use case for AI.

So then through the AI governance committee, a developer can say, hey, I have a defined use case of doing XYZ with this type of AI, I would like to be able to use this kind of AI to do XYZ, and then the governance committee will decide whether that's an appropriate use case for the organization. They also don't want to have a hundred and one different use cases of different AIs within their organization and control it.

So what they would have to do is they'd have to evaluate can this be done with the AI solutions that they already have and is the use case that they're presenting that innately unique from what is already enabled or does it justify the additional cost to the organization from a productivity gain perspective to get that additional AI implemented in the organization from a costing perspective to to make that decision as well.

So before we move on to more questions about kind of the AI approval process, I did want to ask about the risk committee. Are there people Mm-hmm. that should definitely be on it from your perspective? Yeah.

Yes. I mean so from the risk committee you would want IT, you would potentially want someone from legal, you'd want someone from HR, and you'd want someone from the participating line of business. So you don't necessarily always need someone that same person from the line of business every time, but you'd want the participating line of business to be able to weigh in on these are the risks, and this is how we're going to compensate for the risks. And that way they can have a revolving set of individuals within that committee, but you want to have a standard set of individuals in there from IT, legal, HR, compliance to kinda set those standard set of items that you want to consider.

Got it. So as the committee is working through these approvals for AI use cases, how do they determine which ones require more rigorous governance and which can move through a more a lighter approval process?

Well, th that depends on the organization. So you you start with, okay, does it touch sensitive business information? Yes or no. yes, then potentially a more rigorous process. does it affect critical business systems, yes or no. Does it have a blast radius to affect downstream processes of critical business systems or critical sensitive data, yes or no? And then those become your your decision points as to how critically you want it to be screened.

So what should a AI risk assessment evaluate before that system or use case is approved?

Really you're you're looking at what in the business that AI solution is doing. So for example, are you bringing in an AI solution, hey, just to create images for marketing or is this going to also analyze all the data that's associated with the marketing campaigns, take that all in, and then be able to generate an image, then that becomes a different risk profile. If you're just giving it a prompt to create an image, that's a much lower risk profile than if it has the ability to ingest all of your marketing data sits outside of your guardrails.

For example, you can't bring it into your foundational AI system, whether that be Azure Foundry, AWS Bedrock, Google GCP, so it sits outside with that third party. That becomes one of those critical decisions.

Post approval, every organization is gonna need to implement AI guardrails. How do they do how do they implement those guardrails in a way that meaningfully reduces risk without making the process so restrictive that employees start working around them?

Well, that's becoming a challenge right now that a lot of our organizations are starting to have to deal with. And there's not I would say there's not as many good solutions for that. You typically have sub five millisecond time frame to make a decision on whether an AI should allow processing of a a piece of data and then a sub 30 millisecond decision on whether if it was able to process that data, what could it do at the output of that data? so your guardrails need to be able to take that into consideration.

And then when you're deciding on those guardrails, you have to decide okay, are you trying to bring them in and use guardrails that are designed around the cloud foundation that you're based in, or are you open to using a a guardrail system that is open to all clouds and allowing your developers to develop within that so that you can allow them more flexibility but now you have to be able to try to govern that and manage spend manage data loss And so now the the levers become a little bit harder to do, when you have an omni cloud experience.

How should governance models change as organizations move from employees just using those AI tools to autonomous or semi autonomous AI agents that can actually start to take action?

Well, so you always have to think of every action as a decision point between a human identity and a non human identity. and Then you have to decide whether the action was spawned by a human action or the action w was spawned by a non human action. And based on those, you can make decisions or rules at your guardrails on how you want to make some of those decision points. So I'm starting to see technologies that will say, okay, at a database, this request was spawned by a human Natalie, she wants access this data in here, I will let her through. This other request was spawned by Jake's agent, but Jake didn't spawn it. Jake shouldn't have access to this database. I will not let that through.

So it can make those sorts of decisions like that.

Okay. are there any specific tools you can recommend for that? Or you'd rather not Okay.

I'd I'd rather not not recommend tools. Yeah. Not a marketing piece.

So with this AI governance committee, they're they're governing the AI, they're gonna need to show to auditors that AI is being governed,

right? So Mm-hmm.

what evidence should they maintain to demonstrate, not to just the auditors but executives and regulators and even customers that AI is being governed?

So well, first to executives, you're gonna wanna see where how effective AI has been to achieving the mission. And that's gonna come down to the spend, right? Was this two million dollar investment of tokens worth the output? And of the two million dollars of output who spent the most and where was these projects spent. You're starting to see a lot of guardrails that are putting controls in to monitor spend and analyze spend based on the type of model, the type of model used for each query. If they have access to multiple models, they're smart enough now to use the right type of model for the type of query.

So if they know say you stolen you've done this query using sonnet but I use opus all the time after a while they're gonna downgrade me to sonnet because that's a more efficient model to use. So you're getting smart smarter guardrails that are starting to be able to optimize around spend. So that's one of the things that you're gonna want to see in governance. The other thing that you're gonna want to see in governance is the ability to control data loss and data exfil. Now this is a challenge for many organizations because when it comes to identifying data within your organization and tagging data within the organization for what is and isn't sensitive.

Has always been a challenge, and then now you're going, okay, well, what is and isn't sensitive? Plus, do I want AI to be able to touch that? some of the sensitive data you may want AI to touch that because it's part of an enterprise process and you want AI to touch that. But then Mm-hmm. there might be other sensitive information that you don't want AI to touch that. So now there's this extra layer of tagging and extra rules that you need to create in your DLP process that hey AI shouldn't be able to interact with this type of data. So it becomes an extra layer of guardrails that you need to put in from the DLP process to further protect your data from that.

Then there's the second, third, and fourth hand processing of data. A lot of organizations might have openly shared shares that they don't realize are openly shared. And that will cause data that has been say, we're in a team, you have access to data that maybe I shouldn't have access to, but now I'm doing a query and it's able to query your queries and now I can see the data that you queried that I shouldn't have access to in my query. So being able to control that and being able to put limits on hey, if individual A doesn't have access to the data, even if it's in a shared environment, prevent that oversharing from happening.

And those are from a co-work perspective or a work perspective, extra controls that you have to put in and think about on how you want your work context to develop because from an organizational perspective context is really helpful for your generalized AI because that helps your team's function because now they have that extra ability to see things at a larger picture which they might not have been able to see but you want to ensure that they only see the information that they're supposed to see and not the information that they weren't supposed to see. So it becomes that that balancing of that.

And then the the last item is making sure that your AIs are protected from outside access, from malicious use, from prompt injections and things like that, from being poisoned, from ensuring that there's some level of confidence in the data that they're providing back. That there's some validation there, that the teams don't inherently trust everything that comes out of it, 'cause it could be wrong. And teams still have to realize that while these AI are pulling in a lot of information, it could be wrong. So you still have to try to double check that information.

Excellent. All right. So as we wrap up, that was a lot of excellent advice on AI governance. For organizations that already use AI extensively without a formal model, how do you wrap that into a kind of a thirty, sixty, ninety day plan for them? What the first steps that they need to take to govern their properly?

Yes, start with identifying what's available in your organization, making a decision not to allow anything more to enter the environment, put in controls to block any additional OAuth or any additional purchases of AI.

And then now starting to screen down any unwanted or undesired AI that has been used down to a reasonable set of AI that you do want to use while developing that that AI governance team from an enterprise perspective as you go along so that by the time you get to your ninety day you have that AI governance team and you've whittled down that list to what you think would be a desirable list, that you can work with the business and go, okay, we're gonna start to cut off access to all these applications. If you have a justified business reason to keep them, let's talk about that now and discuss that with the AI governance committee as to why we should keep them.

All right, excellent. Well thank you everybody for watching. If you enjoyed this conversation, be sure to like, subscribe, and leave a comment with thoughts or suggestions for future guests and topics. And thank you again to Christophe for joining us today.

Thank you so much.