MERLONIX

Redact Before the LLM Sees It: A Deterministic Floor, an AI Ceiling, and Never the Reverse

The instant your code writes messages: [{ role: 'user', content: someUserString }] and posts it to OpenAI, Anthropic, or Perplexity, that string has left your trust boundary. Whatever was in it — a customer's email, a pasted Authorization: Bearer header, an internal IP, a Stripe key someone dropped into a support ticket — is now in a third party's logs, retention window, and possibly a future training set. You cannot un-send it.

Any product with an AI feature has this surface, and monitoring tools have it badly, because the raw material is the sensitive stuff: support tickets, asset nicknames a customer typed, the diff of a competitor's page, DNS and TLS metadata. All of it can carry PII or secrets, and all of it is on its way to a model.

We send data to LLMs in several places — summarizing a page diff, drafting a support reply, running a legal filter when a customer names an asset — so we settled on two rules and, more importantly, an ordering between them. The ordering is the entire point of this post.

Rule 1: redact with a deterministic floor, before the call

Every string that is going to an external model runs through one function first. It is boring, deterministic regex — and boring is exactly what you want on the layer that must never surprise you. It replaces, in place:

  • Credentials, always (both redaction depths): JWTs (eyJ….eyJ….sig), Bearer <token>, common SaaS key prefixes (sk_, pk_, ghp_, github_pat_, …), URL query params named token/key/secret/password/auth/access_token/api_key, and PEM -----BEGIN … PRIVATE KEY----- blocks.
  • Personal data, on the full depth: emails, phone numbers (E.164 international and North-American NANP), IPv4 and IPv6, credit-card-shaped digit runs, SSNs, MAC addresses, and US-format dates (which leak dates of birth).
// depth 'credentials' strips only the always-on credential set;
// depth 'full' also strips email/phone/IP/card/SSN/MAC/date.
const safe = redactForAi(userInput, 'full');
// -> "contact [EMAIL] from [IP], card [CARD], token [API_KEY]"

Two design choices in that function matter more than the pattern list.

It fails closed on anything it cannot scan. A non-string input, or an input larger than a fixed byte ceiling, is not "best-effort scanned" — it is replaced wholesale with a placeholder. The reasoning: partial redaction is worse than no output, because it looks clean while a pattern you didn't reach sails through. If the function cannot be sure, it emits nothing but the placeholder. Same for a regex that throws (catastrophic backtracking on a hostile input) — the catch returns the placeholder, never the raw string.

containsPii is defined in terms of the redactor, not a parallel set of rules. It is literally "does redaction change the string?" — redactForAi(x) !== x. One place to keep correct, and it inherits the same fail-closed bias: oversize or non-string returns true (assume PII) rather than false.

That is the floor. Be honest about what a floor is: regex does not understand meaning. A person's name, a street address without a clean number-and-street shape, "the patient in room 4B" — none of that matches a pattern, and none of it gets redacted. Which is exactly why there is a second layer, and exactly why the second layer is ordered the way it is.

Rule 2: an AI second opinion — ordered so it can only make the verdict stricter

For the highest-sensitivity calls — an escalated support question, the legal filter that runs when a customer creates an asset — we add a model-backed guard on top of the regex. The temptation is to "ask the AI if this is safe" and trust its answer. That is backwards, and it is dangerous, because now an AI hiccup can either block legitimate traffic or wave through the obvious. The guard is built so the AI can only ever tighten the decision:

  1. Deterministic first. Run the regex containsPii check. If it flags PII, the input is unsafe — return immediately. No AI call. Every existing detection is preserved, and you don't pay for a model round-trip on the obvious cases.
  2. AI only on the survivors. If and only if the regex passed, ask a small, fast instruct model one narrow yes/no question: does this text still contain PII the regex missed? Treat the answer as unsafe only on a clear positive.
  3. Fail back to the verdict you already trust. On any AI error, timeout, or ambiguous output, return the regex verdict — which already passed.

Read what step 3 means: the AI can move a "safe" to "unsafe," never the reverse. It cannot rescue an input the regex already condemned, and if the model is down, the guard degrades to exactly the deterministic behavior you had before you added it. An outage in the fancy layer cannot make you less safe than the boring layer alone — and it cannot start blocking real support tickets on its own either.

Notice the two layers fail in opposite directions, on purpose:

  • Redaction fails closed — when unsure, redact everything.
  • The AI guard fails to the already-safe verdict — when the model is unsure or unreachable, defer to the deterministic check that already ran.

That asymmetry is deliberate. The layer that decides what content leaves must never leak on uncertainty. The layer that decides whether a whole high-sensitivity flow proceeds must never take that flow down because a best-effort model timed out. Same phrase — "fail safe" — means opposite behavior at the two layers, and getting that right is the difference between defense-in-depth and two ways to break.

Where this actually runs

This isn't a diagram in a wiki. redactForAi runs on the asset nickname a customer types before it is placed in an SSL or DNS analysis prompt, and on a competitor's page diff before it is summarized. The model-backed guard gates the support-agent path and the asset-creation legal filter — low-volume, high-sensitivity calls where the extra model round-trip costs effectively nothing. The high-traffic monitoring path leans on the deterministic redactor, which has no per-call cost and no network dependency.

If you take three things

  1. Redact before the call, not after. The mistake is logging or persisting the raw input "for debugging" and redacting only the copy you show a human. The copy that matters is the one crossing your boundary to the vendor.
  2. Make the deterministic layer fail closed. Oversize, non-string, or a throwing regex must yield a placeholder, never the original. A redactor that best-effort-scans-and-hopes is a redactor that leaks on the input it couldn't reach.
  3. Let the model only tighten. A regex floor plus an AI ceiling is strong; an AI floor is not a floor at all. Order them so a model outage costs you nothing you had, and a model false-negative can never loosen a decision the deterministic check already made.

Merlonix monitors uptime, SSL/TLS, DNS, and answer-engine presence for agencies and their clients — and yes, the AI features that summarize and triage all sit behind the two layers above. If you want the free tooling, the MCP server health check and domain health scan run in your browser with no signup; the full free-tools index has the rest.