PrivAiTe vs Presidio, LLM Guard, and LiteLLM PII masking
PrivAiTe is a drop-in, self-hosted LLM proxy that redacts PII before it reaches the provider and restores it in the reply. Compared with the three tools measured below (Microsoft Presidio, a library you assemble; Protect AI LLM Guard; LiteLLM’s built-in Presidio guardrail), it also redacts PII inside tool-call arguments and multimodal content, reversibly and with zero telemetry: 100% of the PII placed in a tool-call argument survives in both measured guardrails. That is a statement about those tools, not about the field: several gateways built for agent traffic now handle tool calls too, and they are listed further down.
This is local pseudonymization, not anonymization, and detection is best-effort. You remain the data controller. See the threat model.
Feature comparison
| PrivAiTe | Microsoft Presidio | Protect AI LLM Guard | LiteLLM PII guardrail | |
|---|---|---|---|---|
| Shape | Drop-in OpenAI-compatible proxy | Python library | Python library | Gateway feature |
| Setup | Point your client at it | Assemble it yourself | Assemble it yourself | Config in the LiteLLM proxy |
| Reversible (restore on reply) | Yes | Manual | Yes (anonymize/deanonymize) | Limited |
| Redacts PII in tool-call arguments | Yes | No | No | No |
| Redacts PII in multimodal text | Yes | OCR only | No | Yes (text parts) |
| Streaming de-anonymization | Yes | n/a | n/a | n/a |
| Secrets and passwords | Yes (ONNX preset) | No | Yes | Partial |
| Detection engine | Presidio + local ONNX model | Presidio | Own scanners | Presidio |
| Self-hosted | Yes | Yes | Yes | Yes |
Presidio is excellent and PrivAiTe builds on it. The point of this table is not that PrivAiTe detects better than Presidio in isolation; it is that PrivAiTe is the ready-to-run proxy around it that also covers the structured and multimodal cases, and restores the original values on the way back.
The tool-call gap, measured
The reproducible benchmark runs the real competitor integrations and sends the same public record in each part of an OpenAI-style request. On the 40 records of Privy, the one corpus of that suite PrivAiTe’s model never trained on (37 direct identifiers: names, contacts, IDs, secrets, addresses), share of direct identifiers that still reach the provider, lower is better:
| User message | Multimodal part | Tool-call arguments | Tool result | |
|---|---|---|---|---|
PrivAiTe minilm (opt-in) |
0.0% | 0.0% | 8.1% | 0.0% |
PrivAiTe onnx (default) |
5.4% | 5.4% | 13.5% | 5.4% |
| LiteLLM Presidio guardrail | 16.2% | 16.2% | 100% | 16.2% |
| Kiji Privacy Proxy 1.6.4 | 24.3% | 100% | 100% | 24.3% |
The sample is small: read it as an order of magnitude.
LiteLLM’s guardrail scrubs message text and multimodal text parts, but never parses tool-call JSON, so 100% of the identifiers inside a tool-call argument reach the provider, even values it detects in plain text. PrivAiTe removes from the tool call everything it detects in the message. The benchmark scores 13 request integrations this way (8 of them not PrivAiTe) and 25 systems on plain text, with the methodology and per-corpus tables in its README.
Gateways built for agent traffic
These projects sit in front of coding agents or LLM APIs like PrivAiTe does. None of them was run in the benchmark above: the table reports what each project’s own README or docs state, as read on 6 October 2026, not a measurement. “Not stated” means the page read does not say, not that the feature is missing.
| Project | Shape | Detection, as stated | Reply restored | Tool-call arguments |
|---|---|---|---|---|
| PasteGuard | Local proxy, browser extension, coding agents | Checksums and format checks, plus GLiNER for names and places; secrets (API keys, private keys, JWTs, passwords, connection strings) | Yes (“restores supported placeholders in the response”); streaming responses are handled | Not stated |
| maskit | Local gateway for coding tools, browser extension | 21 rule categories (7 on by default); optional local ONNX model for names, organizations and addresses, off by default | Yes, streaming for OpenAI and Anthropic | Restored; signed thinking blocks keep their placeholders |
| CosyRedactGateway | Single-file credential gateway (OpenAI Chat Completions, Responses, Anthropic Messages) | High-entropy credential detection, structured rules, Gitleaks-compatible rules; emails, bank cards, ID numbers | Yes, in responses and supported streams | Restored, including streamed deltas |
| packyme/privacy-filter | Go library with HTTP and gRPC services | Regex for structured PII; gitleaks rules, contextual regex and entropy for secrets; no names, by design | No: placeholders “are irreversible” | Not applicable (a text filter) |
| DontFeedTheAI | Proxy for AI-assisted pentesting (Claude Code, OpenAI-compatible clients) | Local Ollama model for hostnames, organization names and credentials in prose; regex for IPs, hashes, tokens and API keys | Yes | Not stated |
| LangSmith LLM Gateway | Managed gateway policy | PII and secrets policy | Yes, streamed or not | Structured arguments are scanned; arguments that arrive as a single JSON string are skipped, as are system and developer prompts |
So looking inside tool calls and restoring a streamed reply are no longer what sets PrivAiTe apart from every alternative. What it brings, stated about itself only:
- Tool-call arguments are parsed as JSON and scrubbed value by value, including when they arrive as a single JSON string, which is how the OpenAI chat format sends them.
- A contextual model for names and addresses with a published recall on a public corpus, and a benchmark anyone can rerun.
- A wire-level measurement on real Claude Code and Codex sessions, misses included (write-up).
- The same engine three ways: proxy, Open WebUI filter, LiteLLM guardrail. It fails closed: a detection error blocks the request.
When to pick which
- Pick Presidio if you want a detection library to embed in your own pipeline and you will handle the proxying, reversal, and tool-call cases yourself.
- Pick LLM Guard if you want a broader prompt-security toolkit (prompt injection, toxicity) and PII is one part of it.
- Pick LiteLLM’s guardrail if you already run the LiteLLM proxy and only need flat message-text PII handling.
- Pick a guard model (Llama Guard, OpenAI’s gpt-oss-safeguard, Mistral’s Shieldstral) if your question is “is this content acceptable?” rather than “what personal data is in it, and how do I get it back?”: they classify, they do not redact. Why the two don’t substitute for each other.
- Pick PasteGuard if you also want a browser extension for web chat, a dashboard of what was masked, or routing sensitive requests to a local model.
- Pick CosyRedactGateway if credentials are your concern, including random-looking tokens with no known prefix and no
password=label, or if you want a single-file gateway. - Pick maskit if your data is Chinese-language (ID cards, licence plates, USCC codes) and you want a console and a browser extension.
- Pick packyme/privacy-filter if you need millisecond, irreversible redaction embedded in a Go gateway and do not need names.
- Pick DontFeedTheAI for pentest engagements, where hostnames, IPs and hashes are the sensitive data.
- Pick LangSmith’s gateway if you already run LangSmith and want a managed policy.
- Pick PrivAiTe if you want measured detection, names and addresses included, on the whole egress path (message text, tool-call arguments, tool results, multimodal text), reversibly, with zero telemetry, and a benchmark you can rerun to check it.
Guard models answer a different question
The open guard models (Llama Guard, OpenAI’s gpt-oss-safeguard, Mistral’s Shieldstral as of August 2026) take a document and a moderation policy — the recent ones accept the policy as plain language at inference time — and return a verdict: acceptable or not, as a probability. That is content-safety classification, and it differs from what PrivAiTe does in two structural ways, not two incidental ones.
A verdict has no spans. A guard model reports that a document crosses the policy, not where the offending value sits. Without character offsets it cannot replace a value with a placeholder, cannot restore it in the reply, and its only enforcement is to pass or reject the request whole. PrivAiTe’s entire mechanism — replace on the way out, restore on the way back, tool-call arguments included — depends on spans, which is why its detectors are span extractors and a verdict model cannot slot in as one.
A judged policy is probabilistic; a declared one is not. A plain-language policy is flexible, and the model judging it returns a score: the same borderline text can land on either side of the threshold depending on the surrounding context, and the model cards themselves note reduced reliability on long or obfuscated inputs. PrivAiTe’s policy layer is deterministic rules over span detections: the same request always produces the same outcome, the whole policy is dry-runnable before you trust it, and a block rule that can never fire refuses to start instead of silently never firing. When the audience is an auditor rather than a demo, that difference is the product.
Honest in both directions: a guard model expresses semantic policies a regex never will (“anything that describes self-harm”), and covers harmful-content moderation, which PrivAiTe deliberately does not do at all. The two compose rather than compete — a guard model deciding what is acceptable, PrivAiTe deciding what leaves for the provider.
Reproduce it
The commands are in the privaite-bench README: each system runs in its own pinned environment, on corpora downloaded at pinned revisions.