Specify the human review queue before the agent fills it.
Once an agent starts producing drafts, "we'll review them" turns into a pile with an unspoken deadline and free-text reject reasons. Human review works when you set it up like any other queue you'd staff.
Before agents send work to humans, fill the TWIMCO HITL Review Queue Spec: batch size, review SLA, evidence required on each item, reject taxonomy, and requeue vs close paths. A queue without those fields becomes either a parking lot or a rubber stamp—neither is review.
Human review is a queue
The agent produces drafts, proposals, or flagged actions. People "will review." The pile grows. The batch is whatever arrived overnight. The deadline is unspoken. The evidence on each item is inconsistent. Reject reasons are free text. Nobody knows whether to send it back or close it.
A human in the loop only works when the queue is specified like any other queue you would staff.
Write the queue before it fills
Batch. How many items a person is expected to clear, and what happens to the rest.
Deadline. How fast. If the deadline is so tight that nobody can reject, you do not have review. You have a rubber stamp with a timer. If there is no deadline, you have a parking lot.
Evidence. What must be attached before an item can be claimed. If it is not there, the item is not ready. It is not the reviewer's job to go hunting.
Reject reasons. A short shared list. Free text alone cannot be learned from. People will invent private languages, and you will not see the pattern.
Send back or close. What each one means, and how many times an item may return to the agent before a person has to stop the loop.
Publish the reject list where reviewers work, not in a doc they will not open under pressure.
How the work moves
HITL queue spec spine
- Batch size
Max per sitting; overflow rule written.
- SLA
First review and disposition; breach path.
- Evidence
Minimum links before reviewable.
- Reject taxonomy
Closed reasons; free text only as supplement.
- Requeue / close
When agent may retry vs human-only exit.
A human review queue without a reject taxonomy trains reviewers to invent private languages.
Stand it up on one output
Do not boil the ocean. One kind of agent output. Fill the page. Enforce the evidence rule in the tool, so an item without it cannot be claimed.
Look at missed deadlines in the same meeting where you look at agent changes. A queue that breaches and a prompt that changed are often the same story.
If review load is real, budget for it. Agents sold as a way to remove people, and then staffed with an undesigned pile, are how the pile becomes the product.
What the undesigned queue becomes
An infinite pile. A deadline so short that reject never happens. Reject reasons nobody can aggregate. A requeue storm, where the agent retries without a limit.
None of those are review. They are ways to say a human was involved without giving that human a job.
Put this into practice
Fill the page for one output type this week. Put the reject list where the work happens. Make evidence required before an item is claimable.
The product around that queue is AI applications and agents.
Related work: Spin Market.
Decision guide
Side channel vs specified queue
| Consideration | Informal review | Queue specified |
|---|---|---|
| Volume | Whatever arrived. | Batch size and overflow rule. |
| Speed | Whenever someone has time. | SLA with breach path. |
| Reject | Free-text essays. | Taxonomy codes in the record. |
| After | Unclear if agent retries. | Requeue constraints or close. |
We build alongside the people doing the work.
Bring us the problem you’re thinking about.