How to choose an AI automation agency: a buyer's checklist

To choose an AI automation agency, check five things before price: whether they will map your workflow before quoting, how they measure accuracy, who owns the code and prompts, what support covers after launch, and how they handle your data. Ask for evidence on each, and compare proposals on scope and terms, not the headline fee. The checklist below puts those checks in order. We are an agency ourselves, so run this list on us too.

#Before you talk to anyone

Write one page that an agency can respond to.

1. The workflow, in one sentence about what happens today. For example: "supplier invoices arrive by email, get keyed into the ERP, matched to a purchase order and approved in three steps."
2. Volume and timing: items per month, peaks, and how long each takes by hand.
3. Systems: which ERP, CRM, help desk or inbox is involved, and whether you have API access.
4. The constraint. Compliance rules, data that can't leave your environment, or a hard go-live date.
5. An owner, meaning one person on your side who can answer questions within a day.

If you can't write the page yet, buy a short audit before you buy a build. Learning that a workflow is a poor fit is much cheaper before anyone has built it.

#Questions to ask, grouped by risk

#Fit and scoping

• Will you map my workflow and test on my real samples before quoting a fixed price?
• What would make you say this is not worth automating?
• Which parts will stay manual, and why?
• What have you built that resembles this, and can I speak to someone who used it?

#Accuracy and control

• How do you measure accuracy, on which data, and what number do you commit to?
• Where does a person approve before the system acts?
• What happens when the system is unsure or fails? Show me the exception queue.
• Can I see an audit trail for one processed item, from input to action?

#Delivery

• Who will actually build it, and will they be on my calls?
• What is the plan for the first 30, 60 and 90 days?
• How do you handle changes in scope?
• What do you need from my team, and by when?

#Ownership and exit

• Who owns the code, prompts, workflows and evaluation set?
• Does it run in my accounts or yours?
• If we part ways, what do I receive and in what format?

#Run and support

• Who monitors it after launch, and how do I find out something broke?
• What does the support period include, and what does it cost afterwards?
• Who pays the model provider, and how will usage be reported?

#Security

• Where will my data be processed and stored, and which providers see it?
• Do you use my data to train models, and is that written in the contract?
• What access does the system get to my accounts, and is it least-privilege?
• Which security certifications do you hold, and can you share the report?

#Red flags

Some answers should slow you down.

• A fixed price before anyone has seen the workflow. Either the scope is padded or the problem is guessed.
• No evaluation method. "It works great in demos" is not a measurement.
• Promises of full automation. Real workflows have exceptions, and a credible plan names them.
• Vague ownership. If the agency keeps the prompts, workflows or accounts, you are renting.
• Months of strategy slides before any prototype. Discovery matters, but it should produce something you can run.
• Case studies with no detail. Look for the workflow, the systems, the volume and what was measured. Results quoted without a method are marketing.
• No post-launch plan. Models and source systems change, so an unmonitored automation drifts.
• Broad permissions. An agent that can send payments or delete records without approval should worry you.
• Pressure to sign this week.

#How to run the reference call and the demo

A reference only helps if you ask about the unglamorous parts. What broke in the first month? How long did fixes take? What did the agency charge for changes? Would they hire it again for a different workflow? If confidentiality rules out a reference, ask for a detailed walkthrough of a comparable build instead: the systems, the volume, the accuracy method and what went wrong.

For the demo, bring your own material. Hand the agency five real examples, two of them ugly, such as a scanned invoice or an email that mixes two requests, and watch what happens. Ask to see the failure path. A good demo shows the exception queue, the approval step and the log for one item. A weak one only shows the happy path.

#Compare proposals side by side

Ask each agency to answer the same rows. Fill the table yourself if they do not.

Score each row yourself as clear, partial or missing. Three missing rows means more questions before you sign anything.

#Ownership and IP terms to settle

Put these in the contract, not a call.

1. Code and configuration: you receive the workflows, scripts and infrastructure definitions.
2. Prompts and evaluation data: you own the prompts, the instructions and the labeled test cases built from your data. A new team can't take over without them.
3. Accounts: model provider keys, cloud resources and workflow-tool accounts sit in your name where possible.
4. Reusable components: the agency may keep general tooling it brings. Agree in writing what that covers.
5. Documentation: an operations guide that says how to run, monitor and restart the system.

#Run and support terms

Launch is where the work starts. Ask for a support period with named coverage: monitoring, fixes for breakages, re-testing after a model or source-system change, and a defined response time. Agree who handles small changes and how they're priced. The cost lines in AI automation cost will help you compare run proposals as well as build quotes.

#Security basics

An agency doesn't need every certification, but you should know which ones it holds and which it doesn't. Ask where data is processed, how long it is kept, who at the agency can see it, and whether it goes to third-party model providers under terms that bar training on your data. For sensitive data, ask about deployment in your own environment. Give the system only the access it needs, and require approval for payments, deletions and external messages.

Key point: Judge an agency on how it handles the unglamorous parts: scoping, measurement, exceptions, ownership and support. Anyone can demo a happy path.

#A simple way to decide

1. Shortlist two or three agencies and send the same one-page brief.
2. Hold one call each, using the questions above.
3. Request written proposals with the comparison rows filled in.
4. Ask for a reference or a walkthrough of a comparable build.
5. Start with a paid audit or a small pilot, with agreed success metrics.
6. Expand only after the pilot meets them.

You can see how we run a project on our how we work page, and what a scoping output looks like in our sample audit. If you would like to put our process against this checklist, book a strategy call.

#Frequently asked questions

#How do I choose an AI automation agency?

Start with a one-page brief, then compare agencies on scoping method, accuracy measurement, ownership terms, post-launch support and security. Price matters, and it is the last thing to compare, because two quotes rarely cover the same scope.

#What should an AI automation agency ask me?

They should ask about volumes, systems, exceptions, approvals and data sensitivity, and they should ask for real samples. An agency that asks few questions before quoting has probably not understood the work.

#What are the biggest red flags?

A fixed price before seeing the workflow, no accuracy method, promises of full automation, unclear ownership of code and prompts, and no plan for monitoring after launch.

#Should I hire an agency or build in-house?

Build in-house if you have engineers who can own evaluation, security and upkeep. Hire an agency when you need to move faster than you can hire, or want a team that has built similar systems. Many teams use an agency to build and then take over running it.

#How long should a pilot last?

Long enough to run on live volume against agreed metrics, often several weeks after build. A pilot without success criteria tells you little.