AI data loss prevention for agents and LLM traffic

A coding agent opens a file to debug a script, and the next request to the model provider carries those lines back as a tool result. Nothing was attached, uploaded or emailed, so a DLP rule that watches files and mail has nothing to fire on. AI data loss prevention (AI DLP) puts the check on that request.

What is AI data loss prevention (AI DLP)?

AI data loss prevention (AI DLP) inspects what leaves an organization through AI traffic: the prompt, the files and tool results an agent reads into its context, the tool calls a model sends back, MCP tool parameters and the connections an agent’s processes open. It redacts, blocks or logs sensitive values before they reach a model provider or another host.

You’ll also see it called DLP for AI, GenAI DLP, generative AI DLP or LLM DLP. The term has a second meaning: AI-driven DLP, where machine learning classifies content inside a classic DLP product for email, endpoints and SaaS. That’s a detection method rather than a new channel, and it isn’t what this page covers.

Formal puts these checks in the proxy that already carries the requests. On a macOS or Linux laptop, Transparent Mode hands an agent’s LLM connections to the Formal Endpoint, and in Kubernetes the Endpoint sits in each Pod as a sidecar. Server-side apps call a Connector in your VPC, part of Formal PAM. Both evaluate the same policies at the session, request and response stages (LLM resources).

Where does LLM data leakage happen?

LLM data leakage happens on six paths for a coding agent, and only the first involves someone typing. Each needs a check at a different point: before the request leaves, before the agent runs a command, or on the connection a child process opens.

• Prompts. Text a person types or pastes: a support ticket, a stack trace with a customer’s email in it.
• Files and tool results. When an agent reads a file, the contents go back to the provider as a tool_result part of the next request.
• Tool-call arguments. The model returns a command, and the command can carry a credential. When an agent writes $GITHUB_TOKEN into a Git URL, the shell expands it, so the real token ends up in the command line while the tool call only shows the variable name.
• MCP tool parameters. A tool call sends whatever the agent put in tool_params to the MCP server, including servers nobody reviewed.
• Child processes. An agent starts git, curl and package managers, and their connections go to hosts other than the model provider.
• The prompt logs. Whatever inspects this traffic also stores it, so the log becomes another copy of every prompt.

A destination allowlist doesn’t close the child-process path, because the allowed hosts still accept data. Latent Health’s lead security engineer put it this way:

“Going from 0.0.0.0/0 to an allowlist felt like progress, until we realized every allowed domain is still an exfiltration channel. Formal lets our production agent do the work safely.”

— Kevin Qiu, Lead Security Engineer @ Latent Health

OWASP files these risks under LLM02:2025 Sensitive Information Disclosure, and its mitigations include pattern matching to detect and redact confidential content before processing.

AI DLP policy example: redact SSNs in prompts and tool results

PII redaction is the control to try first, because the agent keeps working. This request-stage policy from the policy evaluation docs rewrites the outgoing body when a text or tool result part contains a US Social Security number:

package formal.v2

import future.keywords.if
import future.keywords.in

ssn_pattern := `\b\d{3}-\d{2}-\d{4}\b`

request := {
  "action": "rewrite",
  "body": regex.replace(input.http.body, ssn_pattern, "[REDACTED-SSN]"),
  "reason": "Redact SSNs from LLM request messages",
} if {
  input.resource.technology == "llm"
  some msg in input.llm.messages
  some part in msg.content_parts
  part.type in {"text", "tool_result"}
  regex.match(ssn_pattern, part.text)
}

The tool_result check is what covers the file case. An SSN in a CSV the agent opened arrives as a tool result and matches the same way typed text does. Once one part matches, the rewrite action applies regex.replace to the whole HTTP body, earlier turns of the conversation included, so the replacement has to keep the JSON valid. [REDACTED-SSN] does. A replacement containing a quote or a backslash would break the request.

The enforcement docs recommend a body rewrite over blocking the session when the goal is redaction. We’d go further: a blocked session mostly teaches people to paste the same text into a tool you don’t see. Start the policy in Dry-run, which logs what it would do without enforcing it, and compare the bodies in the logs before you set it to Active.

The limit is the pattern: LLM PII detection in this policy is a regular expression. It finds 123-45-6789, not 123 45 6789 or a number split across two lines, and every new data type is one more expression to write and test.

Secrets in coding agent tool calls

Secrets tend to leave in a different place from PII. The model returns a tool call, such as a Bash command, and the agent runs it, so the check belongs on the response, before the command executes. The Git tokens guide blocks any LLM response whose tool-call arguments contain a GitHub token prefix or a URL with credentials:

package formal.v2

import future.keywords.if
import future.keywords.in

credential_patterns := [
  `\b(gh[opsur]_|github_pat_)[A-Za-z0-9_]{20,}`,
  `https?://[^\s/@:]+:[^\s/@]+@`,
]

passes_credential(call) if {
  walk(call.input, [_, value])
  is_string(value)
  some pattern in credential_patterns
  regex.match(pattern, value)
}

response := {
  "action": "block",
  "type": "block_with_formal_message",
  "reason": "Agents may not put credentials in commands",
} if {
  input.resource.technology == "llm"
  some call in input.llm.tool_calls
  passes_credential(call)
}

The first pattern works because GitHub prefixes each token type. The second exists because of shell expansion. An agent that writes x-access-token:$GITHUB_TOKEN@ never puts the token in its tool call, so the prefix check sees nothing, but the URL still has the shape of a URL with credentials. What it misses is stated in the guide: a token passed through a credential helper, a script file or a variable outside a URL.

The command line also lands in a log. For intercepted connections, the Endpoint records the command line of the connecting process and its ancestors, as-is. Formal redacts the Authorization header Git sends, but not the command line, which is why the same guide adds a log schema that encrypts command lines and tool-call inputs on the device.

The stronger fix for keys you control is not having them on the laptop. With credential injection, the Endpoint forwards the request to a Connector, and a Native User adds the real key there. The Stripe guide removes the key from .env files and shell profiles, lets curl reach Stripe without one, and allows only balance, charge and customer reads for a payments-engineers group. Our post on proxying Claude Code does the same for the Anthropic key. The rest of the coding agent controls are on AI coding agent security and the Claude Code page.

Data exfiltration prevention for AI agents

The agent process itself is rarely what moves the data out. A git push, a curl or a package publish started from its shell does, to a host that’s often on the allowlist. Data exfiltration prevention for agents therefore starts with the process that opened the connection.

Network rules on the Endpoint select flows by process. On macOS, process.has_agent_ancestor is true for a known agent and every process it spawns, matched by code signature. On Linux, the agent flags match process names. A policy can then treat the agent’s curl differently from the same command in the engineer’s own terminal. The Codex Desktop recipe is the strict version: Codex Desktop and its children may send classified LLM traffic and reach named MCP resources, while its built-in search, and browsers it launches, are blocked. Block agent git pushes stops agent pushes to sensitive repositories over SSH and HTTPS, and clones and fetches keep working.

MCP is the third route. Request-stage policies can block or rewrite a tool call by input.mcp.tool_name and tool_params before it reaches the server. The response stage supports only allow and encrypt, so Formal doesn’t mask what an MCP server returns (MCP resources). Which tools agents may call is the MCP gateway’s job, and which hosts they may reach is the secure web gateway’s, with background in egress filtering and SSL inspection.

OWASP LLM02 findings for what policies miss

A policy catches only what someone wrote a rule for. Formal’s Insights engine reviews new LLM traffic every 15 minutes against the OWASP Top 10 for LLM Applications (2025), and LLM02 is the category for secrets or personal data sent to a model. Insights never change anything on their own. Each finding names the entity and the log events behind it, and goes to Datadog, Splunk, Amazon S3, Google Cloud Storage or Azure Blob Storage with your other logs.

Assessments give each tool call a risk score and a score for how well it matches what the user asked, and a response policy can block when input.assessment.risk crosses a threshold. Only LLM responses with tool calls get those scores. A recurring LLM02 finding is usually the cue for the next pattern rule. The AI gateway page covers Insights alongside capture and spend.

Prompt logs are sensitive data too

The proxy that redacts a prompt also writes it to the activity log, and that log can hold more than the provider received. For HTTP bodies, Formal stores received and sent, the body before and after policies run, so the value a rewrite removed is still in the received copy. LLM content appears twice more, under log.llm and log.request.http.llm.

Log schemas encrypt, drop or truncate those paths: log.llm.messages, response_messages, tool_calls.input and events.data, plus the copies nested under the HTTP request. A log.source condition limits a schema to Endpoint or Connector logs. To remove prompts from logs with a single drop path on log.llm.messages.

One warning from the docs: log schemas don’t apply to retained policy inputs. With Policy Evaluation Input Retention on, Formal keeps each input in full for the period you set, between 0 and 30 days (logs docs). Retention can be set per resource, so you can keep it short, or at 0, on LLM resources.

AI DLP vs traditional DLP

Traditional DLP watches files, email, devices and SaaS storage. AI DLP watches the requests agents and AI apps make. Neither replaces the other, and the split is clearest channel by channel:

The usual objection to proxy-based DLP for LLM traffic is that it only sees what crosses a proxy. That’s true, which is why the Endpoint lives on the device: on macOS and Linux laptops, a coding agent’s provider calls and the processes it starts are on the path without a base URL change. Anything the network rules don’t select passes straight through, unlogged. Keep your existing DLP for USB, email and SaaS, and put AI DLP where prompts and tool calls leave.