5 Underestimations Security Teams Are Making About AI Adoption

The Gap Between AI Deployment and AI Security Is Widening. Here Is How to Close It.

The conversation most security teams are having about AI focuses on whether to allow it. The more important conversation is about what it takes to govern AI that is already in use, proliferating faster than most organizations have mapped, and creating exposures that existing security frameworks were not built to address.

The problem is not one of technology maturity but of assumptions: security teams are approaching AI adoption with mental models built for a previous generation of technology risk, and those models are causing them to underinvest in the right places.

Underestimation #1: Shadow AI Is Not a Future Problem. It Is an Active, Present One.

Most security teams think of shadow AI the way they thought about shadow IT a decade ago: a problem that needs a policy, a communication campaign, and an approved tool list. That framing is wrong for AI in the same way it was wrong for cloud storage; by the time the policy is in place, the behavior is already entrenched.

Employees at every organizational level are using public-facing AI tools (large language models, AI writing assistants, AI image tools, AI-assisted development environments) and they are entering organizational data into those tools routinely. Customer data gets pasted into a summarization prompt. A finance analyst uses an AI to reformat a revenue forecast. A developer uses a code assistant to get unstuck on a complex function. None of these users believes they are doing anything wrong. Many of them are not aware that their inputs may be retained, used for model training, or accessible to the model provider’s personnel.

The security team’s response to this cannot be limited to policy. Policy alone has never stopped a convenient behavior. The corrective actions that actually work are:

- Browser-level controls that enforce data classification at the point of submission. Proxy configurations or endpoint agents can inspect traffic to known AI endpoints and block or alert when content patterns, SSNs, account numbers, internal project identifiers, and customer records are detected.

- An approved AI tool registry with actively maintained data handling documentation for each platform. The registry needs to specify, for each tool: what data classifications are permitted as inputs, whether the organization has an enterprise agreement with appropriate data processing terms, and whether employees must use organizational accounts (not personal accounts) to access the tool.

- AI-specific employee training that explains the actual data handling mechanics, not just the policy. Employees who understand that their company does not have an enterprise agreement with the AI platform they are using, and that their inputs may be used for training, make different decisions than employees who have only read a policy document.

Underestimation #2: AI Vendors Are a Supply Chain Risk, and Most Vendor Reviews Are Not Catching It

Third-party risk management programs have matured significantly. Most organizations have vendor questionnaires, contractual security requirements, and tiered review processes. The problem is not that these programs are poorly run. The problem is that AI is arriving through channels the programs were not designed to evaluate.

When a productivity suite adds an AI assistant as a feature toggle, it does not trigger a new vendor review. When a development IDE ships an AI autocomplete update, it is treated as a software update, not a new third-party data processor. When a CRM platform enables AI-powered deal forecasting, the procurement team sees a feature release note, not a security review request. In each case, a system that the AI feature now has access to, whether email, code, or customer data, is being processed by a model whose data handling practices were never independently reviewed in that context.

Closing this gap requires two changes to how vendor risk is managed. First, existing vendor reviews for major platforms need to be reopened to assess AI capabilities that have been added since the original review. This is not a trivial ask, but it is a necessary one for platforms that have access to sensitive organizational data. Second, the vendor questionnaire itself needs to be updated. Standard questionnaires ask about encryption, access controls, incident response, and SOC 2 or ISO 27001 certifications. Almost none of them ask the questions that matter for AI specifically:

- Are user inputs used to train or improve the model, and does this apply to the organization’s contracted tier?

- How long are conversation histories or input logs retained, and who can access them?

- Is there a documented process for handling a data exposure incident that originates from AI model behavior rather than infrastructure compromise?

- Does the vendor provide telemetry (logs, API access records, behavioral events) that allows the organization’s security team to monitor AI system activity?

Organizations that have answered these questions for their five most-used AI-enabled platforms are ahead of the majority.

"AI-powered SaaS tools and third-party integrations expand the attack surface in ways most teams miss because the risk is not in the tool itself but in the data flows it creates. Every integration is a potential egress path for sensitive organizational data, often routed through foundation model providers, subprocessors, and logging infrastructure across multiple jurisdictions, none of which appear on the security team's radar.

The failure is structural: procurement and security review processes were built for a different generation of software and have not caught up to the speed, depth, and data appetite of AI-native SaaS."

Ashish Garg

Founder, Managing Partner - RIGA Cyber

Underestimation #3: AI Agents Need Identity Governance, and Almost No One Has Built It Yet

The shift from AI tools to AI agents, systems that take autonomous actions on behalf of users, interact with APIs, read and write data, and operate continuously, creates an identity problem that most organizations have not yet confronted. AI agents are principals in the IAM sense. They have access scopes. They act with credentials. They execute workflows. But they are not being governed like principals.

The practical consequence is that the access an AI agent holds is often far broader than the access required to do its job. An AI assistant provisioned to support a sales team may have been granted access to the CRM, the email platform, the shared document repository, and the calendar system, not because all of that access is required for every task, but because it was easier to grant broad access than to scope it precisely during deployment. Most organizations would not tolerate that access pattern for a human contractor. They are tolerating it for AI agents without realizing it.

The attack surface this creates is significant. A successful prompt injection attack, in which a malicious instruction embedded in content that the agent processes causes it to act in unintended ways, has a blast radius equal to the union of every system the agent can reach. If the agent has write access to documents and send access to email, a prompt injection payload can exfiltrate data and cover its tracks simultaneously. The broader the agent’s access, the more severe the potential impact.

What Proper AI Agent Identity Governance Looks Like

Applying the same principles used for service accounts and privileged credentials to AI agents requires:

Scope access to the minimum necessary for the agent’s defined function. An agent that summarizes inbound emails does not need write access to the CRM. An agent that drafts documents does not need access to financial systems. Define the access scope in the same way you would define it for a service account, and enforce it technically rather than relying on the agent to self-limit.
Treat AI agent credentials as privileged credentials. They should be subject to the same rotation schedules, vault storage, and monitoring as other privileged credentials, not stored as environment variables or hardcoded in configuration files.
Log everything the agent does. AI agent actions should produce audit logs that are ingested by the SIEM and reviewed on the same schedule as other privileged account activity. Anomalies in agent behavior, unexpected access patterns, unusual data volumes, and out-of-hours activity should trigger the same alerting logic as they would for a human account.
Conduct access reviews for AI agents on a regular cadence. The access an agent was granted at deployment may have become excessive as its function evolved. AI agents should be included in the IAM governance cycle, not excluded from it.

Underestimation #4: The Threat Landscape Has Changed, and Most Detection Programs Have Not Caught Up

The defensive security community understands, in principle, that AI has changed what attacks look like. Phishing emails are more convincing. Social engineering scripts are more personalized. Vulnerability research is faster. Malware development that previously required specialized skills can be partially automated. What is underappreciated is how dramatically the tempo of attack has accelerated, and what that means for detection programs that were calibrated for a slower threat environment.

Organizations that do weekly threat reviews are reviewing intelligence that may already be stale relative to how quickly AI-assisted campaigns can move. Alert triage processes designed for a certain volume of suspicious activity are struggling as AI allows attackers to operate at scales previously reserved for nation-state actors. Tabletop exercises built around historical attack patterns are not incorporating AI-specific scenarios that have already been observed in the wild.

There are four places where security programs need to recalibrate specifically because of AI:

Phishing Detection and Employee Awareness

The grammatical and stylistic signals that phishing training programs have relied on for years are largely gone. AI-generated phishing content is fluent, contextually appropriate, and personalized using data scraped from LinkedIn, company websites, and public databases.

Technical controls (DMARC, DKIM, SPF, link inspection, attachment sandboxing) need to carry more of the detection burden that human judgment used to carry. At the same time, employee awareness training needs to shift from “identify suspicious language” to “verify unexpected requests through a second channel regardless of how legitimate they appear.”

Prompt Injection in AI Workflows

Prompt injection is the AI-specific attack vector that deserves the most immediate attention from security teams that have deployed agentic workflows. The mechanism is straightforward: if an AI agent processes external content (emails, web pages, documents, database records) a malicious actor can embed instructions into that content that the model will execute. The agent receives what looks like content and treats embedded instructions as legitimate direction.

Organizations that have deployed AI agents with access to email, document management systems, or external data sources should specifically test for prompt injection susceptibility. Mitigations include input/output validation layers that filter instruction-like content from data inputs, privilege separation that limits what the agent can do even if its behavior is influenced, and monitoring that flags behavioral anomalies in agent outputs.

Vulnerability Chaining and Accelerated Exploitation

AI has significantly lowered the barrier to chaining vulnerabilities into working exploit sequences. Traditionally, developing a multi-stage exploit required deep technical expertise: identifying an initial foothold, understanding how that access could be pivoted to a privilege escalation, and connecting that to a path toward the target. That process is now partially automatable. AI-assisted tools can scan public vulnerability databases, correlate CVEs that are frequently co-exploited, and generate candidate exploit chains faster than a human analyst can review them.

The practical implication is that the window between vulnerability disclosure and weaponized exploitation is shrinking. Security teams that previously had days or weeks to prioritize and patch a disclosed vulnerability are increasingly operating in a window measured in hours. AI allows attackers to rapidly prototype exploits against newly disclosed CVEs, test them against common configurations, and deploy them at scale before defenders have completed their initial triage.

Detection programs need to account for this compression. Vulnerability prioritization based on CVSS scores alone is no longer sufficient; exploitability in the current threat environment, whether the vulnerability is actively being targeted with AI-generated exploit tooling, needs to factor into remediation sequencing. Patch SLAs that were reasonable against a manual exploitation timeline may need to be revisited for vulnerabilities in exposed, high-value systems.

"I believe there already have been drastic changes in supply chain risks, code dependencies, and code validation implications due to the exponential advancements in AI tooling. Mythos is not just 10% better at detecting zero days vulnerabilities than previous models; it is 250% better.

Advanced AI models can chain often low risk vulnerabilities together in unprecedented ways, so we will no longer be able to ship vulnerable code and expect to wait two years before critical vulnerabilities will be found; they will be found in days. At the same time, these same AI models can be used to completely re-write legacy libraries and code dependencies in days, whereas previously, it took months or even years to do so. Mythos can rewrite proprietary code in days, using Rust, for example, which provides extremely fast performance, with memory safe & data race safe benefits."

Dean Sapp

CISO at Filevine

AI-Assisted Threat Detection

The same AI capabilities that are accelerating attacks are available on the defensive side, and security teams that are not using them are accepting an asymmetry that is entirely avoidable. AI-assisted SIEM analysis can surface correlations across event volumes that manual triage cannot process in a reasonable time. AI-assisted threat intelligence summarization can reduce the time between a new threat becoming known and the security team understanding its relevance to their environment.

These are not theoretical capabilities; they are available in commercial platforms today. The question is whether the security team has invested in learning to use them effectively, and whether the output of those tools is being acted on, or just generated.

Underestimation #5: AI Governance That Exists Only on Paper Is Worse Than No Governance at All

This is the underestimation that has the most immediate organizational consequence. A well-written AI use policy that is not operationally enforced does not reduce risk. It creates a documented record that the organization was aware of the risk and failed to control it. When an AI-related incident eventually occurs, and for organizations with significant AI adoption, an incident is not a question of if, the gap between what the policy said and what was actually happening will be examined by regulators, auditors, insurers, and potentially legal counsel.

The signs that AI governance exists on paper rather than in practice are recognizable. Employees regularly use AI tools that are not on the approved list, not because they are defiant, but because the approved tools do not meet their needs, and no escalation path exists to request new tools through the security review process. The security review process for new AI tools has no defined SLA, so business units route around it. Policies prohibit inputting sensitive data into AI tools but no technical controls enforce the prohibition. AI systems are deployed in production without completing the security review the policy requires, because the review is not integrated into the deployment pipeline.

Closing the gap between policy and practice requires acknowledging that policy without process is not governance. The specific operational changes that move an AI governance program from theoretical to functional are:

- Integrate AI security review into the systems procurement and deployment pipeline at the process level. This means that a new AI tool cannot reach production without a completed review in the same way a new vendor cannot receive a purchase order without a completed security questionnaire. The friction has to be built into the workflow, not applied after the fact.

- Create a fast-track review process for low-risk AI tools. One reason business units route around security review is that the review process is slow and opaque. A 48-hour track for tools that meet predefined criteria, no sensitive data inputs, no persistent access, and no integrations with internal systems removes the incentive to bypass the process entirely.

- Assign clear ownership for the AI inventory and the governance program. If no one is accountable for knowing what AI is in use, the inventory will be incomplete by default. A named owner with a defined mandate is more durable than a committee with a shared responsibility.

- Test the controls, not just the policy. Quarterly assessments of whether AI use in practice matches AI use as described in the policy should be routine. Employees who are using unapproved tools should surface through monitoring, not through incident investigations.

Start Implementing Strong AI Governance Today

Strong AI governance is not a future state to work toward once AI adoption stabilizes. It is the precondition for managing AI adoption safely at all. That means establishing clear ownership, enforceable policies, integration with procurement and deployment pipelines, and regular control validation, not as a one-time project but as an ongoing operational discipline. The organizations that will navigate AI adoption without a major security incident are the ones that decide now that AI warrants the same governance rigor as any other enterprise technology.

The regulatory environment, the threat landscape, and the pace of AI deployment are all moving in the same direction. The cost of implementing strong governance now is a fraction of what it will be after an incident forces it. The question is no longer whether AI governance is necessary; it is whether your organization will build it proactively or reactively.

FAQs About AI Security Risks

How do I find out what AI tools are already in use across my organization?

Start with three data sources that most organizations already have access to: network proxy or firewall logs filtered for known AI service domains, browser extension inventories from your endpoint management platform, and a short anonymous survey distributed to department heads asking what AI tools their teams use regularly.

Cross-referencing these three sources typically surfaces the majority of active AI tool usage without requiring specialized tooling. The more important finding from this exercise is usually not the tools themselves but the categories of data being submitted to them, and whether any of that data is regulated, sensitive, or subject to contractual restrictions on third-party processing.

What makes AI vendors a different kind of supply chain risk than traditional software vendors?

Traditional software vendors receive data to process or store on your behalf, and the risk is primarily about their security posture and your contractual terms. AI vendors do something additional: they use submitted data to influence model behavior, which may include training, fine-tuning, or improving outputs for other customers.

The risk is not just that your data could be exposed through a breach, it is that your data may be retained, used, and accessible to personnel at the AI vendor in ways that your existing vendor review process was not designed to evaluate. The specific questions your vendor review should answer for any AI tool are: what data is retained, for how long, for what purposes, and who has access to it internally.

What does governing AI agents like privileged accounts actually look like in practice?

It means treating an AI agent’s access credentials and permissions with the same controls you apply to a service account with elevated privileges. Concretely:

Document the specific access each agent requires to perform its defined function and deny everything else.
Provision dedicated credentials for each agent rather than reusing human user credentials.
Connect the agent’s activity logs to your SIEM.
Set calendar reminders for quarterly access reviews
Establish a revocation procedure that can be executed in under an hour if an agent behaves unexpectedly.

The goal is that any given AI agent’s access is no more than what it needs, is fully documented, and is continuously monitored, the same standard you would apply to a system account with write access to production data.

How has AI changed phishing and social engineering, and what should our awareness training cover now?

The stylistic signals that phishing detection has relied on for years (grammatical errors, awkward phrasing, generic salutations) are no longer reliable indicators. AI-generated phishing content can be fluent, contextually accurate, and personalized using publicly available information about the target, their role, and their organization.

Training should shift from “look for mistakes” to “verify the request through a separate channel.” The operative question employees should be trained to ask is not “does this look legitimate?” but “does this request require me to take an action that I can independently verify before completing?” For high-risk actions (credential changes, wire transfers, access grants) verification through a known-good channel should be a procedural requirement, not a judgment call.

Our organization has an AI use policy. Is that sufficient for managing AI security risk?

A policy is necessary but not sufficient. The gap between what a policy says and what employees actually do tends to be wide for AI tools, because AI use happens at the individual level through browser-based tools that leave no procurement trail, and the personal productivity benefits are immediate enough that employees route around review processes they perceive as slow.

The test for whether your policy functions as governance rather than documentation is whether you can answer three questions: Can you identify employees using AI tools that are not on the approved list? Is the security review process integrated into your procurement and deployment workflow at the process level, not just referenced in a policy document? And have the controls referenced in the policy (data classification enforcement, access restrictions) been technically implemented, or are they voluntary? If the honest answer to any of these is no, the policy is not yet governance.