Human in the loop AI: design patterns for approvals and exceptions

Human in the loop AI means a person reviews, approves or corrects an AI system's output at defined points before it affects a customer, a payment or a record. The design question is where to place those points. Put them at every step and you've built an expensive way to do the work by hand. Put them nowhere and you'll find errors after they've cost money.

#Why the loop exists

AI models read documents and write text well. They also return plausible wrong answers without signaling an error. A rule that fails stops loudly. A model that fails can look like a success.

A human checkpoint catches the confident wrong answer before it becomes a posted invoice, a refund or an email to a customer. It also produces the data you need to improve the system, because every correction is a labeled example of what went wrong.

The goal isn't to review everything forever. It's to review the right cases now, and fewer of them as you gather evidence that the system is accurate.

#Where a person belongs

Keep people in the loop wherever at least one of these is true:

• The action is hard to undo. Payments, bank detail changes, contract terms, deletions and filings all qualify.
• The value is high: an invoice above a set amount, a discount outside policy, a refund above a cap.
• The system is unsure. The model reports low confidence, or two independent checks disagree.
• The case is new, such as a first-time supplier, an unfamiliar document layout or a request type that wasn't in the test set.
• The output goes outside the company. Messages to customers or regulators deserve review, especially in the first months.
• A judgment call is the job. The AI can prepare the file, but the decision is human.

Where none of these apply, such as tagging an internal ticket with a category, automation without review is usually fine, with sampling to check it.

#Design patterns

Most production systems combine several. An invoice workflow might use a confidence threshold to send uncertain extractions to an exception queue, a value threshold for the approval step, and sampling on the cases that passed straight through.

#Confidence thresholds without fooling yourself

A confidence score is only useful if it tracks reality. Some extraction tools report scores that look precise but are poorly calibrated. To set a threshold sensibly:

1. Run the system on a few hundred past cases where you know the right answer.
2. Group results by score and look at how often each band was actually correct.
3. Choose the cutoff where the error rate is acceptable for the cost of a mistake. For an invoice field that rules also check, a lower cutoff may be fine. For a bank account number, set it very high or always review.
4. Treat fields separately. A model may read totals reliably and struggle with handwritten notes.
5. Re-check regularly. Model updates and new suppliers change the distribution.

Don't rely on a model saying it's "sure." Use rules as a second opinion: totals that must equal the sum of lines, dates that must be plausible, vendors that must exist in the master record. When the AI and a rule disagree, send the case to a person.

#Building a review screen people will actually use

Reviewers decide whether this works. If the screen is slow or unclear, they'll approve everything to clear the queue, and the loop becomes a rubber stamp. A good review screen shows:

• The source document or message next to the extracted data
• What the AI produced, with the uncertain fields highlighted
• Which check failed and why
• The relevant context: the PO, the customer's history, the policy clause
• Clear options to approve, edit, or reject with a reason
• A keyboard-friendly layout for people who do this all day

Rubber-stamping has a name, automation bias: people trust automated suggestions more as they watch them be right. Counter it by keeping the queue focused on cases that need judgment, rotating reviewers, and occasionally seeding known test cases to check that reviewers still catch errors.

#Audit logs

A human in the loop only means something if you can prove what happened. Every case should leave a record containing:

1. The original input (or a reference to it)
2. What each AI step produced
3. Which rules ran and whether they passed
4. Who reviewed it, what they changed and when
5. What was written to each system, and the identifier of the resulting record
6. The model and instruction versions in use at the time

With that, "Why was this invoice paid?" and "Who approved this refund?" take minutes to answer. It's also how you measure the loop itself: override rate, time in queue, and which fields reviewers correct most.

Key point: Place human review where an error is expensive or the system is uncertain, make the review screen fast and clear, and log every decision. Then use the override data to reduce review on the steps where people rarely change the outcome.

#A worked example

Take a team that receives 2,000 supplier invoices a month. These figures are illustrative.

1. An AI step extracts vendor, invoice number, date, totals and line items.
2. Rules check the vendor against the master list, match to the purchase order and verify that line totals add up.
3. Invoices that pass every check and fall under the team's approval limit proceed to the normal approval flow. A sample is reviewed each week.
4. Invoices that fail a check go to an exception queue with the failed check shown. An AP clerk fixes the field or contacts the supplier.
5. Any change to a vendor's bank details is held for dual approval, whatever the confidence.
6. Every case is logged. Each month, the team looks at which fields clerks corrected most and feeds that back into the instructions and rules.

Over time the exception queue should shrink on fields where the system proves reliable. If it doesn't, the data shows where extraction needs work. See our post on automating invoice processing with AI for the full flow, and finance operations for how we apply this.

#How to reduce review safely

Start with review on everything that touches a system of record, then loosen in steps.

1. Run shadow mode until the system matches your team's decisions at an agreed rate.
2. Go live with approvals on.
3. Track override rates by category. If reviewers change fewer than an agreed share of one category's cases for several weeks, move it to sampling.
4. Keep gates on irreversible actions whatever the numbers say.
5. Pull a category back to full review if accuracy slips after a model, supplier or process change.

This works for workflows and agents alike. Our overview of AI workflow automation shows where these checkpoints sit in a full process, and how a project runs covers how we agree them during design.

If you want to work out where approvals belong in a process of your own, book a strategy call. We'll map the steps and mark the checkpoints with you.

#Frequently asked questions

#What does human in the loop mean in AI?

It means a person takes part in the AI's process, usually by reviewing, approving or correcting its output before it has an effect. In business automation, this appears as approval steps, exception queues and review screens for uncertain cases.

#What is the difference between human in the loop and human on the loop?

In the loop, a person approves or corrects individual cases before they proceed. On the loop, the system acts and a person supervises overall performance, stepping in when metrics drift or an alert fires. Many systems use the first for risky actions and the second for routine ones.

#Does human in the loop make automation pointless?

Not if the loop is well designed. The system handles the reading, matching and drafting that consume most of the time, and people review a fraction of cases. A good design focuses people on exceptions and high-risk actions instead of every record.

#How do I know when to remove a review step?

Use data. Track how often reviewers change the outcome for a category of cases. When changes become rare and the cost of an error is low, move that category from full review to sampling. Keep approvals in place for payments and other irreversible actions.

#What should an AI audit log contain?

The input, the output of each AI step, the rules that ran, the reviewer and their decision, what was written to which system, and the model and instruction versions in use. Without these, you can't explain a past decision or tell whether a change made things better.