What AI Adoption Actually Does to Your Attack Surface

What AI Adoption Actually Does to Your Attack Surface

Deploying an LLM or AI tool isn't just an IT decision. It's a risk decision. A practical look at the new exposure categories AI introduces and what governance controls actually matter.

During planning meetings for AI adoption, the most common thing I hear from leadership is some version of: "It's just another SaaS tool. We already have a procurement process for this."

That comparison feels reasonable on the surface. You are licensing a cloud service, provisioning user accounts, and agreeing to a vendor's terms. The procurement workflow is familiar. But the comparison breaks down quickly once you look at what these tools actually do inside your environment.

A traditional SaaS application stores and retrieves data within defined boundaries. An AI tool does something fundamentally different. It ingests data from across your environment, processes it through an external model, and generates new outputs that people act on. The data flows are bidirectional. The permission surface is broader than what anyone designed it to be. And the supply chain you are trusting now includes training data, model behaviour, and inference infrastructure that you cannot audit the same way you audit a database vendor.

This is not an argument against adoption. AI tools deliver real productivity gains and organizations that refuse to engage with them will fall behind. Treating adoption as a procurement exercise rather than a risk exercise is how organizations get blindsided by exposure categories that did not exist in their environment 2 years ago.

NIST recognized this distinction explicitly. Appendix B of the AI Risk Management Framework (AI 100-1) catalogs how AI risks differ from traditional software risks: dependency on training data that may not represent the intended context, system complexity that makes failure modes difficult to predict, pre-trained models that introduce statistical uncertainty and bias, enhanced data aggregation capabilities that create new privacy risks, and underdeveloped testing standards that do not meet the expectations applied to traditionally engineered software [9]. That list alone should tell you this is not the same procurement conversation.

Here is what actually changes when you deploy these tools, and what governance controls matter enough to implement before you do.

1. Your Permissions Were Not Built for This

Microsoft 365 Copilot is the clearest example because it is the most commonly deployed AI tool in enterprise environments. Copilot does not create new access. It surfaces the access you already granted. That distinction is the entire problem.

Most organizations accumulate years of over-provisioned SharePoint permissions. Sites configured as public by default. "Everyone except external users" groups that were never scoped down. OneDrive folders shared broadly for a one-time collaboration and never revoked. Legacy Teams channels where sensitive documents were dropped once and forgotten.

Before Copilot, this was a latent risk. An employee would have to navigate to the specific site, know the file existed, and open it. The permission sprawl was technically there, but the practical exposure was limited by obscurity.

Copilot removes that obscurity entirely. A user can now ask, in natural language, for executive compensation data, salary bands, legal strategy documents, or financial forecasts, and if their permission graph allows access, the AI will retrieve it in seconds. The model is doing exactly what it was designed to do. The problem is that your permission structure was never designed to withstand that kind of query capability.

This is not a Copilot vulnerability. It is an architectural debt that AI made visible and exploitable. Microsoft has acknowledged this directly by releasing SharePoint Advanced Management tooling specifically to help organizations identify and remediate oversharing before Copilot deployment [1].

If your organization is deploying any AI tool that indexes internal data, a permission audit is a recommended activity your team can start planning.

2. Shadow AI Is Already in Your Environment

While leadership debates formal AI adoption strategies, employees have already made their own decisions.

Research consistently shows that approximately half of employees are using unauthorized AI tools at work. A 2024 study by Software AG, surveying 6,000 knowledge workers across the US, UK, and Germany, found that 50% of workers use unapproved AI tools, and 46% indicated they would continue doing so even if explicitly banned [2]. The Verizon 2026 DBIR found that 45% of employees are now regular users of generative AI on corporate devices, nearly tripling from 15% the year before, with 67% accessing these services through non-corporate accounts rather than governed enterprise channels [3]. Menlo Security's 2025 report found that 68% of employees use free-tier generative AI tools through personal accounts, and recorded a 50% jump in web traffic to generative AI sites across the organizations it monitors [4].

This is not future risk. This is current, unmanaged data exfiltration.

Every time an employee pastes a contract clause, a client list, a financial projection, or a strategy document into a free-tier AI tool, that data leaves your environment. It enters a third-party system with terms of service you have not reviewed, retention policies you do not control, and training practices you cannot verify. In some cases, those inputs are explicitly used to improve the vendor's model, which means your proprietary data becomes part of someone else's product.

The important distinction from traditional shadow IT is the direction of data flow. When someone used unauthorized Dropbox, they stored company files externally. That was a bounded risk. When someone uses unauthorized AI, they actively transmit data to a third-party model for processing. The data does not just sit in a container somewhere. It is consumed.

Prohibition does not work. Research shows that banning AI tools drives shadow usage deeper rather than eliminating it. Organizations that provide sanctioned enterprise alternatives see unauthorized usage drop significantly [5]. The governance model that works is: provide approved tools with guardrails, classify your data so employees understand what cannot be entered into any AI system, implement data loss prevention policies specific to AI interactions, and monitor for unsanctioned usage.

3. Where the Data Gets Processed Matters

When an organization evaluates a traditional SaaS vendor, data residency is a familiar checkbox. Where is the data stored? Is it in Canada? Does the provider meet our residency requirements?

With AI tools, storage is only half the question. The other half is processing. Your data may be stored in a Canadian data centre, but when a user submits a prompt, that input may be routed to an inference endpoint in another jurisdiction for processing. The result is returned to the Canadian tenant, but the data briefly transited through infrastructure subject to a different legal framework.

This is not a hypothetical architecture. It reflects how large-scale AI services distribute inference workloads across regional compute clusters based on availability and load balancing.

For federal public sector organizations subject to the Treasury Board Directive on Service and Digital, which establishes that Canadian data residency must be evaluated as the principal delivery option for Protected B data in the cloud [6], this creates a compliance question that the standard vendor assessment questionnaire may not ask. It is not enough to confirm where data is stored. You need to understand where it is processed, what jurisdictions the inference infrastructure touches, and whether the provider's terms contractually guarantee processing locality.

For Indigenous communities operating under data sovereignty frameworks, the implications are even more direct. If community data is being processed through infrastructure that is subject to foreign legal access, that is a sovereignty issue regardless of where the storage layer resides. Nitap has written separately about Indigenous data sovereignty and why the distinction between residency and sovereignty matters [7].

4. Prompt Injection Is a Threat Class That Did Not Exist Before

Traditional software vulnerabilities are well understood. SQL injection, cross-site scripting, buffer overflows. These are categories with decades of defensive tooling and established remediation patterns.

Prompt injection is something different. It is a fundamentally new attack vector that exists because of how large language models process input. LLMs do not have a reliable mechanism to distinguish between instructions and data. They process both in the same channel. That means an attacker can embed malicious instructions inside what appears to be ordinary content, and when the AI processes that content, it follows the embedded instructions as if they were legitimate commands [8].

The OWASP Top 10 for LLM Applications (2025) ranks prompt injection as the number one risk for the second consecutive year [8]. There are two forms. Direct prompt injection is when a user crafts input specifically to override the model's intended behaviour. Indirect prompt injection is when malicious instructions are embedded in external data sources that the AI retrieves and processes during normal operation.

Indirect injection is the more dangerous of the two for enterprise environments. Consider a scenario: your organization deploys an AI assistant that can read and summarize email. A threat actor sends an email to someone in your organization that contains hidden instructions in the message body. The email looks normal to a human reader. But when the AI processes it, the embedded instructions tell the model to exfiltrate the contents of the user's inbox summary to an external endpoint, or to modify its responses in ways that serve the attacker's objectives.

OWASP's guidance is direct: system prompts are not security controls [8]. Because LLMs are probabilistic rather than deterministic, they cannot function as auditable security boundaries. Any secrets, credentials, or role structures placed in prompts should be considered exposed. Security controls must be enforced at the application layer, independently of the model.

This has practical implications for how you evaluate and deploy AI tooling. Any AI system that accesses internal data, processes external content, or takes actions on behalf of users needs application-layer controls that do not rely on the model's own behaviour to enforce security.

5. You Are Trusting More Than Code

When you onboard a traditional SaaS vendor, the risk assessment covers familiar ground. Uptime guarantees, patching cadence, data handling and retention, incident response, SOC 2 certification. The supply chain risk is essentially about whether the vendor maintains their software and infrastructure responsibly.

AI vendor risk adds categories that most third-party risk assessment frameworks were not designed to evaluate.

Training data provenance. What data was used to train the model? Does it include copyrighted material, biased datasets, or data that was collected without consent? If the model was trained on data that creates legal liability, that liability can extend to the outputs your organization generates with it.

Model hosting and multi-tenancy. Where does the model run? Is it a shared multi-tenant instance, or is it isolated? In shared environments, are there adequate controls to prevent data leakage between tenants?

Input retention and training policies. Does the vendor retain your prompts and inputs? Are those inputs used to improve the model? If your employees are submitting sensitive operational data, and that data is being fed back into a model that serves other customers, your information boundary has collapsed.

Behavioural predictability. Traditional software is deterministic. Given the same input, it produces the same output. LLMs are probabilistic. The same prompt can produce different outputs on different days. This means that testing and validation must be continuous, not just pre-deployment.

The NIST AI Risk Management Framework (AI 100-1) addresses this by extending its risk model beyond technical performance to include organizational and societal dimensions [9]. The companion Generative AI Profile (NIST AI 600-1) goes further, identifying twelve risk categories specific to generative AI, including data privacy, information security, confabulation, and value chain and component integration [10]. The value chain category is particularly relevant here. It explicitly recognizes that AI supply chain risk includes the provenance of training data, the security of model development pipelines, and the integrity of third-party components.

Your existing vendor risk assessment template likely does not ask these questions. It needs to.

This is not an abstract concern. In February 2026, the Cyber Risk Institute (CRI), in collaboration with the U.S. Department of the Treasury and NIST, published the Financial Services AI Risk Management Framework (FS AI RMF) [14]. The framework exists because 108 financial institutions concluded that the NIST AI RMF alone, while foundational, was not operationally sufficient for managing AI risk in practice. CRI built 230 additional Control Objectives on top of NIST's Govern, Map, Measure, and Manage functions to bridge the gap between abstract guidance and actionable governance. The framework also introduces an AI Adoption Stage model that scales governance requirements to an organization's level of AI maturity, from Initial (exploring use cases) through Embedded (enterprise-wide autonomous decision-making).

If the financial sector, with some of the most mature risk management programs in any industry, determined that existing frameworks left gaps too large to ignore, consider what that means for organizations whose AI governance today consists of a policy that has not been written yet.

6. Output Risk and Decision Integrity

Confabulation, commonly called hallucination, is not a bug that will be patched in the next release. It is a structural characteristic of how large language models generate text. The model produces outputs that are statistically probable given its training data, not outputs that are verified against ground truth. This means it can generate responses that are fluent, confident, and completely wrong.

NIST AI 600-1 identifies confabulation as one of its twelve risk categories for generative AI, noting that confidently stated but inaccurate outputs can undermine the reliability and safe use of generative AI, and are particularly dangerous when they inform decision-making [10].

For an executive audience, this matters because AI outputs are increasingly being used as inputs to operational decisions. If a financial summary is generated by AI and contains fabricated figures, or a policy analysis references regulations that do not exist, or a risk assessment overstates or understates a finding, the person acting on that output has no reliable audit trail to verify its accuracy.

The risk is compounded by automation bias. Research in human factors consistently shows that people tend to over-trust automated systems, especially systems that present information in a confident, articulate format. An AI-generated briefing note looks the same as a human-generated one. The difference is that the human author knows when they are uncertain. The model does not.

The governance control here is not to ban AI from decision support. It is to require human validation before AI-generated outputs are used for material decisions, and to ensure that people understand the limitations of the tool they are relying on. The Government of Canada's Guide on the Use of Generative AI for federal institutions recommends exactly this: that AI-generated content be reviewed for accuracy and that users exercise professional judgement before acting on it [11]. The Canadian Centre for Cyber Security's guidance on generative AI (ITSAP.00.041) reinforces the point, listing output validation among the core mitigations alongside controls for poisoned datasets and data privacy [13].

AI Governance Readiness Scorecard

Evaluate your AI deployment against six objective, technical criteria to determine whether your adoption has outpaced your governance. Rate your organization on each criterion below:

What To Do About It

Audit your permissions before you deploy. If you are rolling out Copilot, or any AI tool that indexes your internal data, run a comprehensive data access governance review first. Identify sites and repositories with overly broad sharing configurations and remediate them before the tool goes live. This is not a nice-to-have. It is a prerequisite.

Publish an acceptable use policy for AI. Define what data classifications are permitted in AI interactions, what tools are sanctioned, and what the consequences are for using unauthorized tools with organizational data. Make the policy short, clear, and specific. A twelve-page document that nobody reads is not governance.

Provide approved alternatives. Banning AI does not eliminate shadow AI. It drives it underground and removes any remaining organizational visibility. Providing sanctioned tools with appropriate controls gives employees what they need while maintaining the data boundaries you require.

Add AI-specific questions to your vendor risk assessment. At minimum, your assessment should cover: training data provenance, input retention and reuse policies, processing jurisdiction (not just storage), multi-tenancy isolation, and the vendor's approach to output reliability and confabulation mitigation.

Require human validation for material outputs. AI should augment decision-making, not replace it. Establish clear processes that require human review before AI-generated content is used for financial decisions, legal analysis, policy recommendations, or any other context where accuracy is consequential. For federal institutions, the Treasury Board Directive on Automated Decision-Making formalizes this expectation, requiring Algorithmic Impact Assessments and human-in-the-loop safeguards scaled to the assessed impact of the system [12].

Run a tabletop exercise. Before the scenario finds you, simulate an AI-related incident: a data leakage event through shadow AI, a prompt injection attack against an internal assistant, or a decision made on fabricated AI output. Walk through your response. You may be surprised by what is uncovered.

AI adoption is not a question of whether. It is a question of how. The organizations that treat it as a procurement exercise will discover their exposure after the incident. The organizations that treat it as a risk exercise will deploy the same tools with controls that actually protect them.

This is not about slowing down adoption. It is about making sure your governance catches up to your technology before something forces it to.

If you need help assessing your AI readiness, mapping your exposure, or building governance controls that align with how these tools actually work, start with a risk assessment.