The AI Attack Surface

Executive summary

The AI attack surface is the complete set of components an attacker can target to manipulate an AI system, extract data from it, or turn it against the business that runs it. It is not a single product or endpoint; it is a new layer that spans models, prompts, retrieval systems, autonomous agents, training pipelines, third-party dependencies, and the APIs and logs that connect them all.

For most enterprises, this surface is expanding faster than any other part of their technology estate. Every team that adds a chatbot, a copilot, a document summarizer, or an autonomous agent extends it — often without a security review, and almost always without controls designed for the way these systems actually fail. The result is a large, fast-growing surface that traditional network, endpoint, and application tools cannot fully see, because the attacks are written in natural language and target model behavior rather than infrastructure.

This article gives security leaders a durable mental model. First it explains why AI expands the attack surface in ways older architectures did not. Then it provides a systematic map of the AI layer, component by component. It uses the OWASP Top 10 for LLM Applications as an organizing frame to connect each part of the surface to a well-understood risk category. Finally, it lays out how to inventory the surface you already have, reduce what you can, and monitor the rest with defense in depth. For the wider context of how these pieces fit together as a discipline, see our overview of what AI security is, and for how Deflected covers the surface operationally, the Deflected platform.

How AI expands the enterprise attack surface

Attack surface is a familiar idea in security: it is the sum of all the points where an unauthorized user could try to enter or extract data from a system. For decades, reducing it meant closing ports, patching software, tightening identity, and shrinking the code that faces the internet. AI does not replace any of that. It adds an entirely new category of exposure on top of it — one with properties that make the old playbook necessary but no longer sufficient.

The payload is language, not code

The defining shift is that an AI system's most dangerous inputs are written in plain language. A web application firewall inspects HTTP requests for the signatures of malicious code. It has no concept of a sentence that politely instructs a model to ignore its own rules, reveal a system prompt, or call a tool it should not. When the exploit is a paragraph of English rather than a crafted script, the controls built to catch scripts simply do not engage. The attacker no longer needs a software vulnerability; they need a persuasive instruction.

The behavior is non-deterministic

Traditional systems are largely deterministic: the same input produces the same output, so you can test exhaustively and reason about coverage. Large language models are probabilistic. The same prompt can yield different responses on different runs, and a defense that blocks an attack ninety-nine times can let it through on the hundredth. This breaks the assumption underneath most testing regimes and means AI systems must be validated continuously rather than certified once.

The data flows in both directions

In a conventional application, sensitive data leaks through a bug — an unauthenticated endpoint, a misconfigured bucket, an injection flaw. In an AI system, data can leak simply because the model generated it. A model with access to customer records, internal documents, or its own system prompt can be led to reveal them in an ordinary-looking response. Data-loss prevention that watches files and network traffic does not read model output for regulated information escaping in prose.

The system can act on its own

The newest and sharpest expansion comes from agents: AI systems granted tools and autonomy to take real actions — querying databases, calling internal APIs, sending messages, executing code. An agent turns a language exploit into a live capability. A single successful injection no longer just produces bad text; it can move money, delete records, or reach into systems the model was connected to for convenience. We cover this class in depth in our guide to AI agent security.

The supply chain is opaque

Finally, AI is built from borrowed parts: foundation models trained by third parties, open-weight models pulled from public hubs, datasets of unknown provenance, embedding models, and a deep stack of libraries. Each is a dependency you did not write and often cannot fully inspect. A poisoned dataset or a backdoored model introduces risk long before the first prompt is ever sent.

Put together, these five properties mean the AI attack surface is not merely bigger than the traditional one — it is different in kind. It requires a map of its own.

A systematic map of the AI layer

To defend the AI attack surface, you first have to see it. The AI layer decomposes into a set of components, each of which is an entry point with its own failure modes. What follows is a component-by-component map. Treat it as a checklist: for every AI system you run, every one of these is either present and secured, present and unsecured, or absent — and you should know which.

Models and model weights

At the center sits the model itself — the trained parameters that generate output. The model is a target in several senses. Its weights, if self-hosted, are high-value intellectual property that can be stolen outright. Its behavior can be extracted through model inversion and membership inference attacks that probe outputs to reconstruct training data or determine whether a specific record was used in training. And the model can be pushed outside its intended behavior with adversarial inputs crafted to force a misclassification or an unsafe response. Whether you run an open-weight model in your own environment or call a hosted one over an API changes who holds which part of this risk, but the model is always a surface.

Prompts and system prompts

Every request to a model is a prompt, usually assembled on the fly from several sources: the application's fixed instructions, the user's input, and often retrieved context. The system prompt — the hidden instruction set that defines the assistant's role, rules, and boundaries — is a particularly sensitive asset. If it leaks, an attacker learns exactly what guardrails exist and how to talk around them; worse, teams frequently embed secrets, internal URLs, or business logic in system prompts that were never meant to be seen. The prompt-assembly step is also where untrusted user input meets trusted instructions, which makes it the primary site of prompt injection. Because so much depends on it, the prompt is the most heavily contested part of the surface.

Retrieval-augmented generation and vector stores

Most enterprise AI systems ground their answers in company data through retrieval-augmented generation (RAG): documents are embedded into vectors, stored in a vector database, and the most relevant chunks are retrieved and injected into the prompt at query time. This makes the retrieval pipeline part of the attack surface twice over. First, the vector store is a data store, subject to the same access-control and tenancy questions as any database — poor isolation can let one user's query surface another tenant's documents. Second, and more subtly, any document that lands in the corpus becomes a potential injection vector: a malicious instruction hidden in a PDF, a support ticket, or a web page can be retrieved and executed as if it were trusted context. This is indirect prompt injection, and it is one of the hardest problems on the surface. There are also embedding weaknesses — attacks that manipulate how content is embedded or retrieved to poison results or extract data from the store.

Agents and tools

When a model is given tools — functions it can call to search, calculate, query, or act — it becomes an agent, and the surface expands dramatically. Each tool is a new capability that an attacker who controls the model's input may be able to invoke. The risk here is excessive agency: an agent granted more permissions, more tools, or more autonomy than its task requires. If the agent can delete records and an injection tells it to, it will try. Multi-step and multi-agent systems compound this, because one manipulated step can cascade through the others. The tools an agent can reach, the scope of each tool's permissions, and the trust boundary between the model's reasoning and its actions are all part of the surface.

Training data and fine-tuning

Models learn from data, and whoever influences that data influences the model. Data poisoning introduces manipulated examples into a training or fine-tuning set to degrade performance, insert a hidden bias, or plant a backdoor — a trigger phrase that makes the model behave maliciously only when it appears. Fine-tuning on enterprise data is especially sensitive: it can bake proprietary or regulated information into the model's weights, where it may later resurface in outputs. The pipelines that collect, label, and feed data into training are therefore a surface, as is any external data source that flows into them.

The model supply chain

Beyond your own training data lies the broader supply chain: foundation models, open-weight checkpoints downloaded from public repositories, pre-trained embedding models, third-party datasets, and the frameworks that run them. Any of these can arrive compromised — a model file that executes code on load, a checkpoint with a backdoor, a dependency with a known vulnerability, or a dataset with poisoned records. Because these components are integrated early and trusted implicitly, supply-chain compromise is a way to attack a system before it is even deployed. Vetting what enters the pipeline is a distinct discipline; Deflected offers it as a dedicated service.

Inference APIs

The interface through which applications call the model — whether a hosted provider's endpoint or your own inference server — is a classic attack surface with AI-specific twists. API keys can be stolen and abused. Weak authentication or authorization can let one caller reach another's data or model. And because inference is computationally expensive, the API is exposed to unbounded consumption: attackers can drive up cost and deny service by flooding it with expensive requests, or attempt model extraction by querying it systematically enough to clone its behavior. Rate limits, quotas, key management, and tenant isolation all live here.

Plugins and integrations

AI systems rarely stand alone. They connect to plugins, third-party services, browser extensions, and increasingly to tool servers over emerging protocols. Each integration is a bridge between the model and another system, and each bridge is a place where trust can be misplaced. A plugin that accepts free-text parameters from the model without validation, or that runs with broad privileges, extends the surface into whatever it touches. Insecure plugin design and over-privileged integrations are a recurring source of real-world incidents.

Logs and telemetry

The last, and most overlooked, component is the exhaust: the logs, traces, and telemetry that record prompts, responses, retrieved documents, and tool calls. This data is invaluable for detection and forensics — and precisely for that reason it is sensitive. Prompt and response logs routinely contain the very PII, secrets, and proprietary content that the rest of the surface is meant to protect. If logging is verbose and access is loose, the observability layer becomes a soft copy of everything, sitting in systems that are often less protected than the primary data stores. Logs are both a defensive asset and a surface to secure.

The OWASP Top 10 for LLM Applications as a frame

A map of components tells you where the surface is. To prioritize, you also need a shared vocabulary for how it gets attacked. The most widely adopted frame is the OWASP Top 10 for Large Language Model Applications, a community-maintained list of the most critical risks in LLM-based systems. Its 2025 edition gives security teams a common language and maps cleanly onto the components above. Walking it in order is a practical way to pressure-test any AI system.

LLM01 — Prompt Injection

Untrusted input manipulates the model's behavior, whether the user types it directly or it arrives hidden inside retrieved content (indirect injection). It is listed first because it is the most common and consequential AI vulnerability — the rough equivalent of injection flaws in traditional applications. It can be used to bypass guardrails, exfiltrate context, or trigger unintended tool calls. We treat it in depth in our writeup on the AI threat landscape.

LLM02 — Sensitive Information Disclosure

The model reveals data it should not: PII, credentials, proprietary content, or details of its own system prompt. This maps directly to the prompt, RAG, and training-data components, where sensitive material enters the model's reach and can escape through its output.

LLM03 — Supply Chain

Vulnerabilities introduced through third-party models, datasets, pre-trained components, and dependencies. This is the model supply chain from the map, elevated to a top-tier risk because a compromised component undermines everything built on it.

LLM04 — Data and Model Poisoning

Manipulation of training or fine-tuning data to corrupt the model, introduce bias, or plant a backdoor. It targets the training-data and fine-tuning components and can be nearly invisible until the trigger condition is met.

LLM05 — Improper Output Handling

Treating model output as trusted before it reaches another system. If a model's response is passed unchecked into a shell, a database query, a browser, or downstream code, the model becomes a conduit for classic injection attacks — cross-site scripting, SQL injection, remote code execution — one step removed. The fix is to treat every model output as untrusted input to whatever consumes it.

LLM06 — Excessive Agency

Granting an AI system more functionality, permissions, or autonomy than it needs, so that a manipulated model can take damaging actions. This is the core risk of the agents-and-tools component and the reason least privilege matters so much in agentic designs.

LLM07 — System Prompt Leakage

Exposure of the system prompt and anything unwisely embedded in it. New as a distinct entry in the 2025 list, it reflects how often teams place secrets, access rules, or business logic in system prompts and assume they are hidden. The lesson is that a system prompt should never be a place to store secrets.

LLM08 — Vector and Embedding Weaknesses

Flaws in how RAG systems embed, store, and retrieve content — enabling data leakage across tenants, poisoning of the retrieval corpus, or manipulation of results. This maps to the RAG and vector-store component and is often overlooked because the vector database feels like plumbing rather than an attack surface.

LLM09 — Misinformation

The model produces confident, plausible, and wrong output — hallucination — that users act on. When an application over-relies on model output for decisions without verification or human oversight, the business consequences of a fabricated answer can be as real as those of a breach.

LLM10 — Unbounded Consumption

Uncontrolled resource use: denial-of-service through expensive queries, runaway cost, or systematic querying aimed at extracting or cloning the model. This maps to the inference-API component and to the economics of running models at scale.

How each surface is attacked — and the business impact

A map and a taxonomy are only useful if they connect to consequences leadership cares about. Below, each major part of the surface is paired with a concrete attack pattern and the business impact it produces. This is the translation layer between a technical finding and a board conversation.

Prompt and system prompt → data exposure and reputational damage

An attacker embeds an instruction in their input, or in a document the system will retrieve, telling the model to reveal its system prompt or to include hidden data in its answer. The immediate impact is disclosure of confidential material — internal rules, embedded secrets, or another user's data. The downstream impact is regulatory exposure if that data is regulated, and reputational damage if the leak becomes public. Because the exploit is a sentence, it can be discovered and shared widely the moment one user finds it.

RAG and vector store → cross-tenant leakage and silent manipulation

Two failure modes dominate. In the first, weak tenant isolation lets a query retrieve documents belonging to another customer, a direct confidentiality breach in any multi-tenant product. In the second, a poisoned document planted in the corpus quietly steers every answer that retrieves it — feeding users wrong information or hidden instructions without any visible sign of compromise. The business impact ranges from a contractual data-protection failure to a slow erosion of trust in a product whose answers can no longer be relied upon.

Agents and tools → unauthorized actions and financial loss

Here the impact turns physical, in the sense that money and records move. An injection that reaches an over-privileged agent can cause it to send an email, modify a record, initiate a transaction, or call an internal API on the attacker's behalf. Excessive agency converts a language attack into an operational one. The business impact is direct financial loss, data destruction, or unauthorized changes to systems of record — and, because the agent acted with legitimate credentials, the activity can be hard to distinguish from normal use after the fact.

Training data and supply chain → latent, hard-to-detect compromise

A poisoned dataset or a backdoored third-party model introduces a defect that may lie dormant for months. The model performs normally until a trigger appears, then behaves as the attacker intended. The business impact is severe precisely because it is latent: by the time the backdoor fires, the compromised model may be deployed across many products, and root-cause analysis has to reach all the way back into the supply chain. This is why vetting components before they enter the pipeline is far cheaper than discovering the problem in production.

Inference API → cost blowout and model theft

Attackers abuse a stolen key or an unprotected endpoint to run large volumes of expensive requests, or query the model methodically to reconstruct its behavior. The impact is twofold: a sudden, sometimes very large, cloud bill, and the slower harm of intellectual-property loss if the model's behavior is successfully extracted. Both are avoidable with quotas, rate limiting, and disciplined key management, yet both are common because the API is treated as internal plumbing rather than an exposed surface.

Output handling → downstream compromise

When model output is piped into another system without validation, the model becomes an injection vector for that system. A response containing a script that a browser then renders, or a string that a database then executes, turns a language model into a delivery mechanism for a classic web vulnerability. The business impact is a compromise of the consuming system — the model is merely the courier — which makes these incidents easy to misattribute and slow to fix.

Logs and telemetry → a second copy of everything sensitive

If prompt and response logs capture regulated data and those logs sit in a broadly accessible store, a breach of the logging system is a breach of everything the AI touched. The business impact mirrors any large data exposure, with the added sting that the exposed data was collected for security purposes. This is the surface that is easiest to forget and cheapest to fix, by redacting sensitive fields and restricting access before an incident, not after.

How to inventory the AI attack surface

You cannot secure what you cannot see, and the AI attack surface is unusually easy to lose track of because AI features are cheap to build and often ship without a security review. The first discipline, then, is inventory. The goal is a living register of every AI system in the organization and, for each, the components from the map above.

Find the AI you know about — and the AI you don't

Start with the sanctioned systems: the copilots, chatbots, and agents your teams have built or bought deliberately. For each, record the model in use, whether it is hosted or self-run, the data it can reach, the tools it can call, and the integrations it depends on. Then go looking for shadow AI — the unsanctioned tools employees adopt on their own, the quiet pasting of sensitive data into public models, the browser extensions and side projects that never went through review. Shadow AI is often the largest and least-governed part of the surface, and surfacing it is a discipline in its own right; see our piece on securing AI agents for how autonomy magnifies the stakes of what you find.

Record the surface per system

For every system on the list, capture a consistent set of attributes so the inventory is comparable and auditable:

• Model and provenance — which model, from where, hosted or self-hosted, and what version.
• Data reach — what the system can read, including RAG corpora and vector stores, and the sensitivity of that data.
• Tools and permissions — every tool an agent can call and the scope of each, mapped against what the task actually requires.
• Integrations — plugins, third-party services, and downstream systems that consume the model's output.
• Interfaces — the inference APIs and endpoints exposed, and how they are authenticated and rate-limited.
• Data handling — where prompts, responses, and logs are stored, for how long, and who can read them.

Classify by exposure and blast radius

Not every system deserves equal attention. Rank each by two dimensions: how exposed it is (public-facing and untrusted-input systems rank highest) and how large its blast radius is (agents that can act, and systems touching regulated data, rank highest). A public chatbot with tool access over customer records is a different risk from an internal summarizer with no tools and no sensitive data, and the inventory should make that difference obvious. This ranking is what turns a flat list into a prioritized program, and it maps naturally onto framework expectations for AI risk identification. For the governance context, our overview of AI security as a discipline connects inventory to the wider control set.

How to reduce the AI attack surface

Once the surface is visible, the next discipline is reduction: removing what does not need to exist and constraining what does. Attack-surface reduction is the highest-leverage work in security because a component that is not present cannot be attacked, and a capability that is tightly scoped cannot be abused far.

Remove what you do not need

The cheapest control is deletion. Retire AI features that are no longer used. Disconnect integrations and plugins that were wired up for a demo and never removed. Trim RAG corpora down to the documents that are actually needed, and drop stale content that only enlarges the pool an injection could hide in. Consolidate redundant models. Every component removed is one you no longer have to defend, monitor, or explain to an auditor.

Apply least privilege to agents and tools

For the components you keep, the governing principle is least privilege — nowhere more important than with agents. An agent should have exactly the tools its task requires and no more, each scoped to the narrowest permission that works. Prefer read-only access where actions are not essential. Put human approval in front of high-impact operations such as payments, deletions, or external communications. Constraining excessive agency (OWASP LLM06) is the single most effective way to cap the blast radius of a successful injection.

Isolate untrusted content from trusted instructions

Much of prompt injection comes from mixing untrusted input with trusted instructions in the same context. Reduce the surface by keeping them apart: clearly delimit user and retrieved content, avoid granting retrieved documents the authority of system instructions, and design prompts so that data is treated as data rather than as commands. In RAG systems, vet what enters the corpus and enforce strict tenant isolation on the vector store so one caller can never retrieve another's documents.

Treat every model output as untrusted

To close improper output handling (OWASP LLM05), never pass a model's response into another system unvalidated. Encode or sanitize output before it reaches a browser, parameterize anything that becomes a query, and validate structure and type before downstream code acts on it. The model is an untrusted source to everything it feeds.

Harden the supply chain and the APIs

Vet third-party models and datasets before they enter the pipeline, prefer components with clear provenance, and scan model files and dependencies for known issues. On the inference side, enforce strong authentication and authorization, rotate and tightly scope API keys, and apply quotas and rate limits to blunt unbounded consumption. And minimize the telemetry surface: redact sensitive fields from logs, shorten retention, and restrict who can read them.

Put an AI gateway in the path

The most efficient way to enforce many of these reductions at once is an AI gateway — an inline control point that sits between your applications and your models, inspecting every prompt and response in real time. A gateway can block prompt injection and jailbreaks, catch PII and data exfiltration before it reaches a model or a user, enforce policy consistently across every AI system, and log every decision for audit. Rather than re-implementing controls in each application, you concentrate them where all traffic converges. Deflected's Prompt Firewall is exactly this: an inline gateway that inspects prompts and responses and blocks the attacks before they land, with every decision logged.

How to monitor and defend the surface over time

A surface that has been mapped, inventoried, and reduced still has to be defended continuously, because it changes constantly and because the attacks against it evolve. Monitoring the AI attack surface rests on four practices that reinforce one another: defense in depth, an AI gateway for real-time enforcement, continuous red teaming for validation, and governance to tie it all to accountability.

Defense in depth

No single control secures the AI layer, because the surface has too many distinct entry points. Defense in depth layers controls so that a failure at one point is caught at another: input inspection at the gateway, least privilege at the tools, isolation at the vector store, output validation before downstream systems, quotas at the API, and encryption underneath it all. The aim is that no one manipulation — a clever prompt, a poisoned document, a stolen key — reaches business impact unchecked, because it must pass through more than one control to do so.

An AI gateway for real-time detection

The gateway is not only a reduction control; it is the primary source of live signal. Because every prompt and response flows through it, it is where attacks are detected as they happen and where an immutable audit log of every decision is written. That log is what lets you answer, after an incident, exactly what was asked, what was retrieved, what the model returned, and what was blocked — and it is the evidence auditors and regulators expect. Real-time inspection turns the AI layer from an opaque black box into an observable, defensible system.

Continuous red teaming

Because model behavior is non-deterministic and the threat landscape shifts weekly, point-in-time testing is not enough. Red teaming attacks your own models the way a real adversary would — probing for injection, jailbreaks, data leakage, and excessive agency — and returns a prioritized, fixable report. Done continuously rather than once a year, it finds the weaknesses on your surface before an attacker does and produces the evidence of resilience that boards increasingly ask for. Deflected's Continuous AI Red Team runs this adversarial testing as an always-on capability rather than a single engagement.

Governance and accountability

Technical controls need an owner and a frame. Governance connects the surface to policy, to framework alignment such as the NIST AI Risk Management Framework and the EU AI Act, and to the reporting that proves the program works. It answers who is accountable for each AI system, what the acceptable-use policy is, how new systems enter the inventory, and how evidence is produced for audits. Without it, monitoring generates data that no one is responsible for acting on. The full operational picture — gateway, red teaming, governance, and encryption together — is laid out on the Deflected platform.

Encryption underneath the whole surface

Finally, every part of the surface handles data, and that data must be protected in transit and at rest — including against a threat that is invisible today. Adversaries are already capturing encrypted data to decrypt later, once quantum computers mature, a strategy known as harvest now, decrypt later. Any AI data that will still be sensitive in five to ten years should therefore be encrypted with post-quantum cryptography now. Deflected uses NIST-standardized algorithms across the platform:

• ML-KEM-1024 (NIST FIPS 203) for key encapsulation at a 256-bit quantum security level.
• Hybrid X25519 + ML-KEM key exchange, which runs a proven classical algorithm alongside the post-quantum one so you are protected even if either scheme is ever weakened.
• AES-256 for symmetric encryption of data at rest and in transit.

Mapped, reduced, monitored, and encrypted, the AI attack surface stops being an unknown and becomes a managed part of the enterprise — one that security can stand behind as confidently as it does the network, the endpoint, and the application.

Frequently asked questions