Decide what the agent may log before the first trace.

The first trace sets the habit

You turn the agent on with “full debug” because you want to see what it does. Prompts, tool arguments, retrieved chunks, customer names, card last-fours that snuck into a note, internal URLs, and a stack of intermediate thoughts all land in a log sink that half the company can query. Two weeks later someone asks for a copy of a conversation. You discover you kept more than the product database keeps, with weaker access control, and no story for deletion.

The agent’s job needed observability. Observability is not “keep everything forever in plaintext.” Observability is: enough to explain an action, without creating a shadow dossier you never meant to operate.

That distinction is judgment, not tooling. Two teams can buy the same tracing product and end in different places: one keeps tool names and outcomes; the other keeps the customer’s entire message because nobody said they should not. The difference is a path written before the first span.

This note argues for a TWIMCO Agent Log Redaction Path Spec filled before the first production trace—not after the first audit question. The filled spec is the unreproducible asset. Another team can say “redact PII.” They do not have your field list, your unmask role, or your sink map.

When the work is agents that act in real workflows, the offer home is AI applications & agents. Keep the argument about judgment: what you are willing to remember about a run.

Name the path, not the slogan

The TWIMCO Agent Log Redaction Path Spec is short on purpose. Ambition here creates a policy nobody can implement.

Job and sinks. Which agent job, and every place a trace can land: vendor dashboard, your log store, a ticket mirror, an eval warehouse, a Slack alert that pastes a span. If a sink is unknown, the path is incomplete.

Clear fields. What may be stored readable: job id, tool name, success or failure, latency bucket, non-identifying error class, model route id. Write the list. Everything else is not clear by default.

Masked fields. What is stored with a reversible or irreversible mask: subject id hashed, email local-part stripped, free text replaced with a token that points to a sealed store. Say which mask, and whether it is reversible.

Forbidden raw. What may never be written to a general sink: full prompt with customer content, raw tool payloads that include secrets, payment data, auth tokens, retrieved documents that were already access-controlled. If you need them for a dispute, they live in a sealed store with a different path—or you accept you cannot replay the prompt later.

Retention. How long clear and masked traces live in each sink. Cite your existing policy if you have one. If you do not, write “not decided” and do not enable the sink until you decide. Do not invent “ninety days” because it sounds careful.

Who may unmask. The role, the evidence required (ticket id, legal hold, named incident), the time box, and how the unmask itself is logged. “Engineering” is not a role. “Anyone with the vendor admin seat” is a finding.

Subject request. What you can delete or export when someone asks about agent traces. If the answer is “we cannot find them across sinks,” the path is not done.

Fill this before you argue about which vendor has the prettier trace UI.

Unmask is a privilege, not a convenience

Debugging will always ask for more. The spec exists so more is a decision, not a habit.

A bad tool call needs a reason code and a masked argument shape. It does not need the customer’s full message in the open search index. If you truly need the raw prompt, the unmask path should feel like breaking glass: named person, named reason, expiry. If unmask is one click for anyone on call, you do not have a path—you have a preference.

Eval sets are a second sink. Teams copy production traces into a notebook to build fixtures. That copy inherits none of your production access rules unless you design it. The spec’s sink list must include eval and fine-tune exports, or they become the leak. Prefer synthetic or already-redacted fixtures when you can; when you cannot, treat the eval store as production-class.

Alerts that paste spans into chat are sinks. A page that includes a prompt fragment has already exfiltrated into a channel with its own retention. Either the alert is redacted, or chat is on the sink map with the same rules—usually worse. Prefer pointers: job id and a link that still enforces auth.

Vendors will offer “full session replay” as a virtue. Sometimes it is. Often it is a second product database with a marketing name. Put that replay UI on the sink map and apply the same clear / masked / forbidden rules. If the vendor cannot redact, you do not enable replay for jobs that touch regulated content—or you accept the risk in writing on the spec.

These points are judgment calls from shipping agent-shaped systems. They are not a claim about any one client’s log volume or a vendor’s default.

Refuse “we will clean it later”

A few patterns show up as soon as agents meet production data.

Full-fidelity traces in week one, redaction “on the roadmap.” The roadmap loses. The first incident review uses the full traces, and now they are evidence you are reluctant to delete. Write the path before week one. If you already turned full debug on, stop new jobs until the sinks are classified—even if that feels slow.

Hashing that is just base64. Mask means you cannot search by the raw value in the general sink. If you can grep the customer email in plaintext, you did not mask. Test it: plant a known value, search, fail the path if it appears.

Vendor retention longer than yours. Your policy says thirty days; the vendor keeps ninety in a UI your whole team can open. The sink map must include vendor defaults, or your policy is theater. Negotiate, configure, or forbid the sink.

Unmask via shared admin with no ticket. Convenience becomes the access control model. Who-may-unmask with evidence is the line that separates an agent platform from a gossip channel.

If you recognize these, stop adding sinks. Complete the Agent Log Redaction Path Spec, then turn logging back on to match it.

Put this into practice

1. Pick one agent job about to touch real customer content. Freeze production tracing if the spec is empty.
2. Fill the TWIMCO Agent Log Redaction Path Spec: sinks, clear fields, masked fields, forbidden raw, retention, unmask role, subject request.
3. Open each sink once and search for a known email or phone. If you find it in plaintext, the mask failed.
4. Route debug needs through unmask, not through wider default logging.
5. If you are scoping agents that act in live workflows, bring the spec into AI applications & agents.

The first trace teaches the system what you are willing to remember. Write that down while you still have a choice.

Related work: Sutton|Place.