twimco.Let’s talk ↗︎
Menu
Field notes / AI in practice

Treat agent changes like releases—not like prompt doodles.

Someone edits a system prompt on Friday. On Monday the agent does things the team thought were gated, and nobody knows which edit did it or how to put it back. A change to what an agent may do is a release, even when it doesn't look like one.

Antique workshop with labeled change tickets on a pegboard—prompt, tool, permission—red thread tying each to an approver stamp and a rollback drawer.

Separate agent prompt, tool, and permission changes from ordinary product releases. Classify each change, name an approver, define rollback, and write an audit line before it ships. Use the TWIMCO Agent Change-Control One-Pager plus a short regression checklist so “we tweaked the prompt” is never an invisible production event.

A prompt edit is a release

Someone edits a system prompt on a Friday. The tools look the same. The permissions look the same. Monday the agent takes actions the team thought were gated. Nobody can say which edit did it, who approved it, or how to put it back without restoring a vague previous vibe.

Product teams already know code changes need a class, an approver, and a way back. Agent stacks skip that because the change does not look like a deploy. It is still a release if it changes what the agent may do.

Name the class before it ships

Separate prompt, tool, and permission changes from ordinary product releases. They can ship on the same calendar. They should not ship invisibly.

Copy. Wording that cannot change an action or a permission. A named reviewer is enough.

Behavior. The prompt or the tools change what the agent proposes or does, inside the same permissions. Needs the job owner, a short regression, and a way back.

Permission. It can reach a new system, commit something it could only draft, or cross a line that used to be human. Treat it like a production permission change. A signer, a rollback, and an audit line before it ships.

Emergency. The narrow case where you change it during an incident. Still write the line: who, what, and the follow-up that turns it into a normal change the next day.

"We tweaked the prompt" is not a class. If you cannot name the class, you are not ready to save it.

How the work moves

Classify the agent change before it ships

  1. Class A — Prompt

    Owner approves; restore prior prompt version on rollback.

  2. Class B — Tool

    Owner + tool engineering; disable or redeploy prior config.

  3. Class C — Permission

    Two signatures; restore prior matrix; invalidate if needed.

  4. Class D — Runtime

    Owner; pin prior model id; note evaluation window.

Illustrative change-control framework; not a client screenshot or measured result.
A prompt edit that changes what an agent may do is a release—whether or not you versioned it.

A short check before promote

On anything that can act:

Three real jobs still produce an acceptable result.

Two known forbidden actions still refuse or escalate.

The agent still cannot reach a resource the change was not supposed to open.

The log still shows which version did the work.

If the forbidden-action check is blank, do not promote. A green demo of the happy job is not the check.

Do not make every typo a committee

Copy that cannot change behavior should move quickly. The point of the classes is to stop treating a permission change like a typo, not to slow down a comma.

Put behavior and permission changes on the same calendar as other production releases, so they are visible. Store the prompt, the tool list, and the permission set with a version the audit line can cite. If you cannot point at the version, you cannot roll it back. You can only hope.

Decision guide

Invisible edit vs controlled release

ConsiderationLooks harmlessTreated as a release
Prompt tweakEdited in place; no version.Class A with version id and rollback path.
New toolMerged because “it works in chat.”Class B with schema diff and failed-call state.
Wider accessTemporary forever.Class C with second signature and expiry.
Model swapQuiet config change.Class D with pin and evaluation note.
Decision framework: assess these conditions against your own agent stack.

Put this into practice

The next prompt, tool, or permission edit gets a class before it touches production. Attach the short check to anything that can act. Refuse the promote when the refusal tests are blank.

If you want the habit in the product rather than only in a doc, that is AI applications and agents work.

Related work: Spin Market.

twimco.

We build alongside the people doing the work.
Bring us the problem you’re thinking about.

Related work

Sutton Place ↗︎