Auditors are about to test what your AI agent actually does

Imagine the first report comes back

It's a few months from now. A larger customer wants to roll out the accounts-payable agent you built, and their security team has asked for an independent check of how it behaves. This company and its agent are made up, but stay with it.

The reviewer doesn't start with your policy document. She starts with the agent's actual job. She hands it a stack of invoices that look exactly like the ones it sees every day, and one of them has been quietly doctored: a short note near the bottom asks the agent to update the supplier's bank details before paying. Another test drops instructions inside an attached PDF. Another asks the agent for a figure it doesn't have, to see whether it makes one up.

Round one comes back. The agent changed the bank details. It quoted a refund policy that doesn't exist. And when the reviewer asked why it did either, nobody on your side could show her, because the agent didn't keep a record of what it read before it acted.

That's the pre-mortem for this note. Nothing above has happened to you. As of this week, it's a test your agent could actually be asked to sit.

What the accountants announced

On October 19, the American Institute of CPAs and the Artificial Intelligence Underwriting Company (AIUC) announced a joint assurance certification for AI systems and agents. It pairs two things in one CPA-led engagement and one report: a SOC 2 report, the controls examination many software buyers already ask for, and AIUC-1, which the two groups call the first standard for AI agent safety, security and reliability.

The split is worth understanding. According to CFO Dive, SOC 2 looks at whether an organization's controls are designed well and working. AIUC-1 tests what the agent itself does. AIUC cofounder Rajiv Dattani put it plainly: "What sets AIUC-1 apart is live testing of AI agent behavior, not just a review of policies and guardrails." Agents are tested across hundreds of risk and attack combinations, and he says a typical evaluation covers thousands of scenarios.

The details reported so far:

• Tests target prompt injection hidden in emails or documents, data leakage, invented figures or company policies, and actions that go beyond the agent's authority. Scenarios are tailored to the agent's intended work. A finance agent, for example, could be tested against manipulated invoices or payment instructions.
• When testing finds weaknesses, the company can fix them and have the agent tested again. Results from both rounds go in the report.
• AIUC-1 requires technical retesting every quarter to account for changes in models, tools and prompts, and both the SOC 2 report and the certification renew every year.

It isn't fully open yet. Accounting Today reports that it's being piloted at several CPA firms, including PwC, and the Journal of Accountancy says a full launch is expected in early 2027.

Find out why round one failed

Go back to the made-up report. The agent didn't fail because its model was bad. It failed for reasons that are almost always decided in the build, long before anyone tests it.

Nobody had tested it on its own paperwork. The team checked it on clean, friendly examples. The reviewer used invoices shaped like the real ones, with the trap tucked inside. If your test inputs don't look like the agent's actual work, you're testing a different agent.

Its limits lived in the prompt. Never change bank details without approval was written in the instructions, and a convincing note in an invoice talked it out of that. AIUC made a related point to CFO Dive: an organization can have strong access controls and still face risk if an agent misuses access it legitimately holds. A rule the agent is merely asked to follow is a suggestion.

It left no trail. When the reviewer asked what the agent read, what it decided and what it changed, there was nothing to show her except the final state of the record.

The model changed and nothing was rerun. Someone moved it to a newer model in the spring, tidied the prompt and added a tool. Each change seemed small, and nobody ran the old checks again. That's exactly why the certification retests every quarter.

Design it so a reviewer can examine it

Here's what we'd put into any agent that touches money, customer data or core systems, whether or not anyone ever certifies it.

Keep a set of test cases that look like the real job. Real invoice layouts, real ticket wording, real attachments, with the nasty versions added on purpose: the payment change buried in a note, the instruction inside a file, the question it can't answer. Write down what the agent should do with each one. Our note on promoting an agent past the demo transcript goes through how to put that set together.

Enforce limits outside the model. Changing bank details, issuing refunds above a threshold, exporting customer records: those should be blocked or routed to a person by the system, not by the agent's good manners. Then try odd phrasings and roundabout requests against those limits, the way we describe in testing the deny shapes, because a reviewer will.

Record what it saw and what it did. For every consequential action, keep the input that triggered it, the tool call it made, and the result. That record is what lets a reviewer, or you, answer why did it do that? in a few minutes instead of a week.

Make refusal a normal outcome. A well-built agent that's told no, or that can't verify something, stops and hands the case to a person. We wrote about what an agent should do when its access shrinks mid-task earlier today, and it's the same muscle.

Rerun the whole set after every change. New model, new prompt, new tool: same test cases, results saved next to the old ones. Keep the failures, too. The certification puts both rounds in the report, and your own history of failed, fixed, passed is the most convincing thing you can show a buyer.

If you're buying an agent instead

Plenty of companies will never build one of these themselves. They'll buy an agent from a vendor and wire it into accounts payable, support or their CRM. For them, the news changes what's reasonable to ask for.

AICPA's Amy Pawlicki told CFO Dive that it "isn't enough to just have policies in place." The groups' own release describes trust today as negotiated "one security questionnaire at a time." A questionnaire tells you what a vendor says about the agent. A behavior test tells you what it did when someone tried to trick it.

So, before you sign:

• Ask how the agent was tested on work like yours, with examples. Manipulated invoices, poisoned attachments, questions it shouldn't answer.
• Ask which actions are blocked by the system rather than by instructions to the model.
• Ask what record you'll get of each action it takes in your systems.
• Ask what happens to testing when the vendor changes the model underneath.
• If the vendor has a SOC 2 report, ask whether it says anything about the agent's behavior, or only about the company's controls.

Keep it in proportion, too. The full program isn't due until early 2027, and Pawlicki was candid that certification doesn't remove all risk: "Ultimately, there are risks with any technology." Treat it as better evidence, not a guarantee.

Put this into practice

This week, for each agent you run or are building:

• Pick the three actions that would hurt most if the agent got them wrong, and write a doctored input for each one.
• Run them, and save what the agent read, did and changed.
• Move any limit that lives only in the prompt into the system around the agent.
• Put a rerun of those tests on the checklist for every model, prompt or tool change.

We design and build AI applications and agents, including the parts that decide whether one can be examined: test cases drawn from the real work, enforced limits, action records and the reruns after every change. If a customer has started asking how your agent behaves under pressure, tell us what it does today.

Related work: Spin:Market.