twimco.Let’s talk ↗︎
Menu
Field notes / AI & judgment

Do not promote what you cannot pin.

Since last week's convincing demo, the model alias moved, the prompt was edited in a console and a tool gained a parameter. What you're about to promote isn't what was reviewed. Write down exactly what will run, and stop if any of it drifts.

Editorial illustration for Field Note: Do not promote what you cannot pin.

Promote an agent or model-backed workflow only with a TWIMCO Production Pin Card: model id, prompt identity, tool versions, config that changes behavior, and a named rollback owner. If any pinned item has drifted and the card was not re-approved, do not promote. The card is a release fact, not a measure of model quality.

A promotion names what will run

Someone asks to put the agent in front of real work. The demo last week was convincing. Since then the model alias moved, the prompt was edited in a console, a tool gained a parameter, and the temperature was changed to make a stakeholder happier. The thing being promoted is not the thing that was reviewed. It is a neighborhood of things, and the neighborhood is not a release.

Change control says how a release is proposed, reviewed, and shipped. Shadow-mode criteria say when you have seen enough to consider promotion. Neither one is the pin. The pin is the list of identities that must be true in production after you ship, and the rule that drift cancels the permission to promote until someone re-accepts the new identities.

This note is the TWIMCO Production Pin Card. One card per production workload: an agent, a classifier, a generation step that customers will see, a support workflow that writes. It does not score quality. It stops you from shipping an unnamed bundle and calling it the bundle you approved. It is not a live tool. Fill the card. If you are still deciding whether the workflow belongs in an agent at all, use the live AI workflow readiness check first. Readiness is not a pin.

On Sutton|Place, the partnership treats what is true, and who may act, as inspectable. A model-backed release deserves the same habit. The build around that pin is AI applications and agents when the product is the workload, not a one-off experiment in a notebook that happened to gain a cron job.

Write the card at the moment promotion becomes possible, and require it again whenever a pinned item changes. A card that was true for a pilot and then ignored is how pilots become production by inertia.

What the card pins

The TWIMCO Production Pin Card has these lines. Unknown is allowed only as “not pinned,” and not pinned means do not promote.

Workload name. The job, not the vendor. “Customer email draft” or “lot description check” is a workload. “The AI” is not.

Model identity. The provider and the model id you will call, including whether the id is a dated snapshot or a floating alias. A floating alias is a pin only if you explicitly accept that the provider may change behavior without you. That acceptance is a decision, written on the card, with a role. Prefer a snapshot you can name. If you cannot read the id your code calls, you do not have this line.

Prompt identity. A hash or version of the exact prompt template in the release, not a description of its intent. If the prompt is assembled from pieces, identify the pieces and the assembly. A prompt edited in a hosted console, with no export, cannot be pinned. That is a reason to change how you store it, not a reason to skip the line.

Tool and dependency versions. Every tool the workload can call, with a version or a schema hash, and the side effects each tool is allowed. A tool that can write is not the same pin as a tool that can only read, even if the name matches. Include retrieval indexes or knowledge files if changing them changes answers. “Latest” is not a version.

Behavior config. The settings that change outputs or actions: temperature, tool-choice mode, max steps, timeouts, allowed domains, and any safety or policy filter you rely on. You do not need a long dump of irrelevant defaults. You need the ones your review actually depended on. If reviewers assumed a filter you are not pinning, the review did not review this release.

Eval or acceptance reference. A pointer to the review you are promoting on: a scenario set, a shadow period, a signed acceptance. The card does not invent a score. If the acceptance was “we looked at ten examples,” write that, so the pin does not launder a glance into a test suite.

Environment. Which production target this pin applies to. A pin for a sandbox is not a pin for production. Say so, or you will promote the wrong card.

Date and approver. Who accepted this exact set, and when. The approver is a role that is allowed to accept risk for this workload. A developer clicking deploy is not automatically that role, unless you have said they are.

Sequence of work

Pin the set, then promote

  1. Name the workload

    The job customers or operators will actually meet. Not “the AI.”

  2. Write the identities

    Model, prompt, tools, and the config the review depended on.

  3. Point at acceptance

    The review or shadow period you are promoting on, without inventing a score.

  4. Match or stop

    If production would differ from the pin, re-approve or do not promote.

  5. Keep the previous pin

    Rollback is a named prior card, or an explicit disable.

Illustrative release sequence, not a client screenshot or measured result.
If you cannot name the model, the prompt, and the tools, you are not promoting a release.

Drift stops the promotion

The rule on the card is short. If model, prompt, tools, or behavior config differ from the pin, you are not in the release that was approved. Do not promote. Do not “promote and pin tomorrow.” Either re-approve the new pin, or ship the pinned set.

Drift during a pilot is expected. Drift at the promotion gate is a stop. The stop is not a moral judgment about iteration. Iteration is how you got something worth pinning. The stop says the iteration and the approval have to meet.

Some drifts look small. A tool’s description changed, so the model calls it differently. A retrieval file was refreshed with new policy text. An alias moved to a newer model the provider considers compatible. Compatible is the provider’s word. Your card either accepts floating aliases or it does not. If it does not, the move is a new release.

Who detects drift? Name them. A pin nobody compares to production is a diary. The practical version is a check at deploy, and a check when someone edits the prompt store or the tool registry. The check does not need to be elaborate to be real. It needs to be able to say “this is not the pin” and to have a human path that is not “override and go to lunch.”

Overrides exist in real operations. If you allow one, the card says who may override, how long it lasts, and what must be true before the next deploy. An override without an expiry is a new, unapproved pin that is too shy to say its name.

Do not add a numeric drift score. This note does not have one, and you should not invent one to make the gate look scientific. The gate is identity. Either the pinned things match or they do not. Where “match” is fuzzy, as with a retrieved document that changes often, the card defines the unit of identity: the corpus version, the file hash, or an explicit acceptance that the corpus floats. Fuzziness written down is a pin. Fuzziness hoped away is not.

Keep failed promotions visible. A team that hides a stopped promote will route around the card. The record can be a line in the release log: pinned set, result, approver. That is enough.

Rollback has a name before you need it

The pin is also the thing you roll back to. If you cannot say what the previous pin was, you do not have a rollback. You have a desire to undo.

The TWIMCO Production Pin Card names a rollback owner and the previous acceptable pin, or it says “no previous pin; rollback is disable.” Disable is a legitimate rollback for a first release. Pretending you can restore a prompt nobody saved is not.

Rollback includes tools and config, not only the model id. Rolling the model back while leaving a new write-capable tool in place is a different workload. The owner should know that, because they will be asked to “just switch the model” under pressure.

Say what rollback does to in-flight work. Cases already handed to a human stay with the packet they have. Cases mid-tool-call need the same idempotency and recovery you would demand of any job. The pin card does not replace that design. It points at it, so nobody believes a model revert undoes a write that already happened.

Practice the path once before the first production promote, even as a table read: who is called, where the previous pin is, what disable looks like to a user, what happens to the queue. A table read is not a drill with metrics. It is a way to find the blank line while the blank line is cheap.

These failure shapes are composites, not client outcomes. The prompt lives only in a vendor console, so the pin is a screenshot. The alias was accepted without a decision, and the answers changed over a weekend. The tool version floated, and a write fired that the review never included. The rollback owner was a team name, and at the moment of need nobody was sure they were allowed to disable the workflow. Each of those is a reason a line on the card is mandatory. None of them is a rate you should quote.

Decision guide

A floating setup versus a pinned release

ConsiderationReady in spiritReady to pin
ModelAn alias someone trusts the provider to move.A named id, or a written acceptance that the alias may move.
PromptWhatever is currently in the console.A hash or version of the template the review saw.
ToolsLatest, including writes the review did not discuss.Versions, schemas, and which tools may change state.
UndoSwitch the model and hope.A rollback owner and a previous pin, or disable.
Decision framework: assess these conditions against the workload you want to promote.

Put this into practice

  1. Name one workload that is a candidate for production. Open a TWIMCO Production Pin Card for it.
  2. Fill model identity, prompt identity, tool versions, behavior config, acceptance reference, environment, approver, and rollback owner.
  3. If any line is “not pinned,” do not promote. Either pin it or reduce the release until the card covers what will actually run.
  4. Put a check at deploy that compares the running set to the card. Overrides need a role and an expiry.
  5. When the workload itself needs to be built so those identities are even readable, scope it under AI applications and agents. Treat the pin the way Sutton|Place treats verification: a fact someone can inspect after the meeting, not a feeling from the demo.

You can change the model, the prompt, and the tools. You cannot call that change the release you already approved.

twimco.

We build alongside the people doing the work.
Bring us the problem you’re thinking about.

Related work

Sutton Place ↗︎