Skip to content
New: a sample workflow audit report
Scenox Studios

AI customer service: what to automate first, and how to design escalation

AI customer service works best on repeatable ticket types. See what to automate first, how to measure containment honestly and how to design escalation.

By the Scenox Studios engineering team

7 min read

How we research and update articles
On this page
  1. What an AI customer service agent actually does
  2. What to automate first, by ticket type
  3. Containment versus deflection
  4. Escalation design
  5. How to roll it out
  6. Guardrails worth setting
  7. When it is not worth it
  8. Frequently asked questions

AI customer service uses language models to answer, classify and resolve support requests across chat, email and voice. It works best on repeatable ticket types where the answer lives in your systems: order status, password resets, policy questions, address changes. Start there, measure real resolution rather than chats that merely ended, and give every conversation a clear path to a person.

What an AI customer service agent actually does

An AI customer service agent is software that reads a customer message, works out the intent, looks up the data it needs, and either resolves the request or passes it on with a summary. It does that with three capabilities.

First, it understands free text. Typos, mixed topics and missing details get classified into your ticket types.

Second, it answers from your sources. It retrieves answers from your help center, policies and product docs and cites the article it used. The pattern is called retrieval-augmented generation, and our knowledge assistants page explains how it works.

Third, it takes actions in your systems: looking up an order in Shopify, checking a subscription in Stripe, updating a ticket in Zendesk, issuing a return label. An agent that only talks can only deflect. One that acts can resolve.

"Conversational AI for customer service" usually means the same thing with more emphasis on the dialogue layer. The questions that decide whether it works stay the same: what can it see, what can it do, and when does it stop?

What to automate first, by ticket type

Don't start with the whole inbox. Pull a few months of tickets, group them by type, and rank each type by volume and by how predictable the fix is. This table shows a typical order. Your own data may reshuffle it.

Ticket typeAutomate?What the agent doesWhere a person stays involved
Order status and trackingYes, firstLooks up the order, reads the carrier status, repliesLost or delayed parcels past a set threshold
Password reset and login helpYes, firstTriggers the reset flow, confirms by a verified channelSuspected account takeover
Policy and how-to questionsYesAnswers from the help center, cites the articleQuestions the sources do not cover
Address or detail changesYes, with checksConfirms identity, updates the record if the order has not shippedOrders already in fulfillment
Returns and exchangesPartlyChecks eligibility against policy, creates the return, sends the labelOut-of-window or high-value returns
Billing questionsPartlyExplains charges and invoice lines, resends invoicesRefunds, disputes, credits
Technical troubleshootingPartlyWalks through documented steps, collects logs and screenshotsAnything that needs engineering
Complaints and cancellationsTriage onlyDetects sentiment, summarizes, routes with contextThe conversation itself
Legal, safety and medical issuesNoRoutes immediatelyEverything
Vulnerable or distressed customersNoDetects and routesEverything

The "Triage only" rows are a deliberate choice. We usually don't automate cancellations or complaints end to end, because the interaction is where retention is won or lost, and an agent that blocks a cancellation can create bigger problems than the ticket it saves.

Containment versus deflection

Vendors report numbers such as "automation rate" or "deflection rate". Know what each one counts before you compare them.

  • Deflection counts customers who didn't create a ticket after seeing an automated answer. It can't tell you whether they got what they needed. Some of them gave up.
  • Containment counts conversations that finished without a human joining. A customer who left in frustration is contained.
  • Resolution counts conversations where the customer's problem got solved. It's the figure that matters, and the hardest to measure.

Measure resolution with several signals together. The customer confirms the answer worked, no new ticket on the same issue shows up within a set window, and a person reviews a sample of conversations. Set a target you can defend after reading transcripts, not one you copied off a dashboard.

For example, suppose an agent contains 60 percent of 1,000 monthly conversations. A review of 100 contained conversations finds 80 solved, 12 abandoned and 8 with a wrong answer. Your contained rate is 60 percent and your resolved rate is nearer 48. Both numbers are useful. Only one is the outcome you wanted.

Key point: Report containment and resolution as separate numbers, and sample real transcripts every week. If containment climbs while resolution stays flat, the agent is just getting better at ending conversations.

Escalation design

Escalation decides whether customers trust an AI customer service build. Design it before you design the answers.

When to escalate

Set triggers in rules, not in the model's mood:

  • The customer asks for a person, in any wording.
  • Two failed attempts to resolve the same issue.
  • Detected anger, distress or an urgent safety word.
  • Topics on your never-automate list (legal threats, injury, fraud, regulatory complaints).
  • Account value, plan or customer segment that you want handled by a person.
  • Low confidence in the retrieved answer, or no source found.

What the handoff includes

A bad handoff makes the customer repeat everything. A good one passes:

  1. The full conversation, with the agent's actions listed.
  2. A two-line summary of the issue.
  3. Customer identity and relevant account data already looked up.
  4. The suggested next step and why.
  5. The priority and the promised response time.

Where the handoff goes

Route to the queue that owns that ticket type, not a general inbox. Billing issues go to billing, technical issues to tier 2. Outside staffed hours, the agent should say when a person will respond and create the ticket with that time on it. Our support operations blueprint shows how routing, SLAs and macros fit around the agent.

How to roll it out

  1. Audit ticket history. Group three to six months of tickets by type, volume and handle time.
  2. Fix the knowledge base first. An agent that answers from out-of-date articles will repeat their mistakes at scale.
  3. Pick two or three ticket types with high volume and clear answers.
  4. Connect read-only systems first. Order lookup, subscription status, ticket history.
  5. Add actions one at a time. Each new action (refund, address change) gets its own limits and approval rules.
  6. Run in shadow mode. The agent drafts replies for staff to approve before it talks to customers.
  7. Go live on one channel. Start with chat or email, then add voice.
  8. Review weekly. Read transcripts, update sources, adjust triggers.

Shadow mode is worth the extra weeks. Staff see exactly where the agent is wrong, and you collect real examples for testing.

Guardrails worth setting

  • Limit refunds, credits and account changes by amount and type. Above the limit, a person approves.
  • Verify identity before sharing account details or making changes.
  • Never let the agent promise things outside policy, such as delivery dates it cannot see or compensation it cannot grant.
  • Log every action with the data it used, so you can answer "why did the agent do that?"
  • Keep a kill switch per ticket type.

When it is not worth it

If you handle fewer than a few hundred tickets a month, a good help center and a small team may beat a custom agent. If your answers differ by customer in ways nobody has written down, the first job is documenting policy. If your main support problem is a product defect, an AI agent will answer the same angry ticket faster and leave the cause alone.

For teams with the volume, our customer support page describes how we build and run these agents. To talk through your ticket mix, book a strategy call. We will tell you which types are worth automating and which are not.

Frequently asked questions

What can AI customer service handle on its own?

Repeatable requests where the answer sits in your systems or documented policy: order status, password resets, how-to questions, simple returns and detail changes. Anything involving judgment, money above a limit, legal risk or an upset customer should go to a person with the context attached.

What is the difference between a chatbot and an AI customer service agent?

A traditional chatbot follows scripted decision trees and fails when a customer words things unexpectedly. An AI agent reads free text, retrieves answers from your sources and can take actions in connected systems. The agent still needs limits and review, because it can be wrong in fluent language.

How do you measure whether AI customer support is working?

Track resolution, not just containment. Combine customer confirmation, repeat contacts within a time window and weekly review of sampled transcripts. Add customer satisfaction scores for automated and human conversations, and compare them.

Will AI customer service replace support agents?

For routine tickets, it can reduce the load. Many teams redirect staff toward complex cases, quality review and improving the knowledge base. Outcomes depend on your ticket mix, so model it on your own data.

How long does it take to launch?

A narrow first release covering a few ticket types, with shadow mode, often takes several weeks. Timing depends on how clean your knowledge base is and how many systems the agent needs to reach.

Get the next article by email

One practical piece a month on automating real operations. No promotions.

One practical article a month. Unsubscribe any time.

All articles

Tell us about the workflow you want to hand over.

A 30-minute call with an engineer. You will leave knowing what a system could do for that workflow and what it would take to build.

What happens next

  1. 1

    Strategy call

    We learn how the workflow runs today and what a good outcome would be.

  2. 2

    Workflow audit

    We map the process and measure where a system would save the most time.

  3. 3

    Proposal

    A written scope, timeline and fixed price for your first system. No obligation to build.