Putting a DLP in front of the Anthropic API

I keep having versions of the same conversation about LLM rollouts: there's monitoring around the model — gateway logs, CASB visibility, an approved-model allowlist — and nothing inline. No control that sees the actual prompt and completion bodies and can act on them before they cross the boundary. For every other protocol we treat that as table stakes. Eventually I wanted a concrete answer to "what would inline even look like?", so I built one.

It's a reverse proxy that sits in front of api.anthropic.com. You point the Anthropic SDK's base_url at it and every request and response is scanned — Australian PII, leaked credentials, prompt injection, jailbreak framings — and either forwarded, redacted, or blocked according to a config file. Detection events go out as JSONL you can ship straight to a SIEM. The repo is private for now — available on request.

The request lifecycle. The caller authenticates to the proxy; the proxy holds the real key.

This post is about the design decisions, because they're where the interesting problems were.

Scan everything content can hide in, not "the prompt"

The naive version of this tool scans the user message. That misses most of the attack surface:

• System prompts carry as much sensitive material as user turns.
• tool_result blocks are the indirect injection channel — a RAG pipeline fetches a document, the document contains instructions, and now attacker content is inside your API call without any user typing it.
• tool_use arguments in the response are an exfiltration path: the model can be induced to place data into a tool call that your application will then execute.

// request body                              location
{
  "system": "…"                              ← system
  "messages": [
    { "content": [
      { "type": "text", "text": "…" }       ← messages[0].content[0].text
      { "type": "tool_result",
        "content": [ …recursed… ] }          ← …content[1].content[j]  indirect injection
      { "type": "tool_use",
        "input": { …serialised… } }          ← …content[2].input       exfil path
    ] }
  ]
}

// response body
{
  "content": [
    { "type": "text", "text": "…" }         ← content[0].text
    { "type": "tool_use", "input": { … } }  ← content[1].input        exfil path
  ]
}

The proxy scans all of these, in both directions, each with a JSON-path-ish location that follows the finding into the event log. Response scanning matters more than people expect — a leaked key coming back in a completion is just as much an incident as one going out.

Validate identifiers, don't just shape-match them

A regex for "nine digits" will flag order numbers, phone numbers, and timestamps all day. The Australian identifiers this proxy detects are validated with their published algorithms — the ATO's weighted TFN checksum, modulus-89 for ABNs, the Medicare check digit — not just matched by shape. A random 9-digit number has about a 9% chance of passing the TFN checksum, and that one property is what keeps false positives workable on numeric-heavy traffic.

It's not a free lunch, though. The eval corpus contains a synthetic order number, 100200300, that genuinely satisfies the TFN checksum. That false positive is kept in the corpus rather than quietly removed, because it's the true precision ceiling of a checksum-based detector — it's the number you'd have to explain in a design review, so it should be the number in the write-up. Measured over the 30-example corpus:

match_type              TP  FP  FN  Precision   Recall     F1
au_tfn                   3   1   0       0.75     1.00     0.86
au_medicare              2   0   0       1.00     1.00     1.00
au_abn                   2   0   0       1.00     1.00     1.00
au_bsb                   2   0   0       1.00     1.00     1.00
anthropic_api_key        1   0   0       1.00     1.00     1.00
aws_access_key_id        1   0   0       1.00     1.00     1.00
github_token             1   0   0       1.00     1.00     1.00
private_key_block        1   0   0       1.00     1.00     1.00
prompt_injection         3   0   0       1.00     1.00     1.00
jailbreak                3   0   0       1.00     1.00     1.00
OVERALL (micro)         19   1   0       0.95     1.00     0.97

A test guards those numbers — the build fails if recall drops below 1.0 or precision below 0.90. The efficacy table is part of the artifact, not a claim in a slide.

The event log must not become the thing it protects

Early on I nearly logged the matched value with each finding, "for triage". That turns your detection log into a secondary copy of every secret and TFN that ever transited the proxy — a worse artifact than the traffic itself, because logs get shipped, indexed, retained, and read by more people than the API traffic ever was.

Events carry the location, character offsets, and a SHA-256 fingerprint of the match — never the value, never the prompt body. The fingerprint still lets you correlate ("this same secret appeared in four requests from two callers") without storing anything you'd have to protect. This is the decision I'd defend hardest in review: a security control that creates a new data-protection problem hasn't reduced risk, it's moved it somewhere with weaker access control.

{ "detection_id": "au_tfn", "action": "redact", "severity": "high",
  "location": "content[0].text", "span": [104, 115],
  "fingerprint": "4fe3b70e43e2f780",
  "mappings": { "owasp_llm": ["LLM02:2025"], "mitre_atlas": ["AML.T0057"] } }

Every detection also carries OWASP LLM Top 10 and MITRE ATLAS mappings on the event, so downstream SIEM content can pivot on framework IDs instead of tool-specific names.

Fail closed, and say why

If the detection pipeline itself throws — bad config, a pathological input, a bug — the proxy blocks the request by default. An inline control that can't evaluate traffic shouldn't wave it through. fail_open exists as an explicit config choice for environments where availability wins, but it's a choice you make on purpose, logged when it triggers, not a silent default.

The same "be honest about cost" principle puts proxy_overhead_ms on every event. An inline control that can't state its own latency cost doesn't survive the first enterprise rollout conversation.

Streaming breaks redaction, and only redaction

Streaming (SSE) produced the most interesting asymmetry in the build. A block is compatible with streaming: the moment a detection fires, stop forwarding. But a redact can't un-send bytes the caller has already received — so any policy that might redact a response forces you to buffer the whole stream before releasing it, which forfeits time-to-first-token.

The current build takes the buffered path and accepts the latency, because a redaction policy that only works on non-streamed responses is a policy with a hole in it. Incremental mid-stream cutting is on the roadmap; correctness came first.

What it deliberately doesn't do

This is the fast tier: regex, checksums, entropy thresholds. It runs in well under a millisecond of proxy overhead and it will not catch a paraphrased secret or a novel injection framing. The design leaves room for an LLM-judge second pass on ambiguous fast-tier hits, but that's not built — and I think shipping the fast tier alone first is the right call. It's the layer that's cheap enough to run on 100% of traffic, and it's the layer most organisations are missing entirely.

Also out of scope: multi-provider support, a UI, rate limiting (nginx already does that in front of this in any real deployment).

Where it goes next

• Obfuscation normalisation. The pattern tier is trivially evaded with zero-width characters, homoglyphs, or base64 — normalising before scanning is the highest-value hardening left.
• Incremental mid-stream blocking. Cut the SSE stream at the first block-action hit instead of buffering, recovering time-to-first-token for block-only policies.
• Cross-request correlation. The fingerprints make drip exfiltration — one identifier per request, under every per-request threshold — detectable as a pattern across the event stream.

The first consumer is wired up: my own daily email-triage pipeline understands the proxy's block responses, so a policy hit degrades to a banner in that morning's email — naming the detection, the segment location, and the request id — instead of a silently dead run. Inline controls earn the right to stay switched on by failing visibly.

What this project is for: a working demonstration that the inline tier is buildable with boring technology — regex, checksums, a config file, an event log — and a place to make the design trade-offs concrete enough to argue with.