Back to all posts

Amazon's Ticket Labeller No Longer Writes a Word. That's Why It Costs 98% Less

2026-11-12
luodaInt
Amazon's Ticket Labeller No Longer Writes a Word. That's Why It Costs 98% Less

Amazon's Ticket Labeller No Longer Writes a Word. That's Why It Costs 98% Less

On 9 November 2026 the Strands Agents team published a short post by two Amazon engineers, Carr Onstott and Mehul Jaiswal, about a job most companies have somewhere in their stack: putting incoming tickets into the right bucket. Within two days it was near the top of Hacker News with more than 700 points and well over 200 comments.

The story is simple. An internal ticketing system had been using Claude Sonnet 4.6 to label tickets into 20 categories. They swapped it for a small open source decision model, Strands Decider, and report that it did the job as well for about 98% less money.

The interesting part is not that a small model beat a big one. It is why this particular job never needed a model that can write.

What the job actually was

According to the write-up, the tickets could be "customer escalations, capacity requests, automated alarms, security findings, or anything else across the company that needed to fit within 20 category types." The previous setup ran a long system prompt on every ticket, with Sonnet generating JSON and reasoning against a fixed set of labels.

That is a common pattern. A team needs triage, an LLM is the fastest thing to wire up, and it works. Nobody revisits it because nothing is broken. The bill just grows with volume.

A model that can only choose

A decision model, sometimes called a system one model, does not generate text. In Strands' own introduction, it takes some state and a question with a set of options, and returns a probability for each option plus a reliability score. The 2B version is built by taking a pre-trained LLM body (Qwen3.5-2B), removing the part that generates text, and attaching a small head that scores the options.

For ticket labelling, the fit is almost literal. The engineers put it this way: "the state is the ticket, the options are the label catalog."

They started with the 2B model, found that a Gemma 4 based E4B build did clearly better on a test set, tuned the confidence threshold before a label is applied, and report that E4B then beat production Sonnet on a separate set of their tickets. The post does not publish accuracy figures, so that claim rests on their word.

The numbers they published

All of these are Amazon's own measurements for one internal workload, at published us-east-1 list prices:

| | Claude Sonnet 4.6 | Strands Decider E4B | | --- | --- | --- | | Average time per call | 4,215 ms | 897 ms | | Tokens per ticket (input + output) | 6,724 | 2,010 | | Output tokens per ticket | 170 | 1 | | Cost per million tickets | about $22,200 | about $500 |

The token drop came from two places. The decision model outputs one thing, the choice, instead of a JSON answer. And the prompt got much shorter, partly by force: the production label descriptions were longer than the model's 4,096-token window, so each of the 20 labels was rewritten as a one- or two-sentence criterion, plus a "none" option.

Rewriting 20 labels as tight one-line criteria is real work, and it is probably part of why the small model performed well.

A clear label catalog helps any classifier, including the LLM you already have.

The part worth copying

The cost table gets the attention. The engineers say the piece they would reuse regardless of model is the switch.

The labelling function reads a three-way setting on every message:

  • BEDROCK: Sonnet labels every ticket and the decision model is not called.
  • DUAL: both run in parallel. Sonnet's label is applied, and the two answers are compared.
  • DECIDER: the decision model labels every ticket and Sonnet is not called.

In their words: "Running both on real traffic, then judging every disagreement, told us more than any offline eval would have. It also meant going back was a 45-second config change."

That is the pattern any business can adopt this week, whatever models it uses. Before you replace one AI step with another, run both on live traffic, keep the old answer in charge, and look at every case where they disagree. You get your evaluation set from your own data, and you keep a one-step way back.

When this does not apply

The authors are clear about the limits. It worked because the task is a closed choice from a known catalog, the volume is high, and latency mattered. If your labels need explanation text, if the catalog changes weekly, or if the useful context does not fit in a few thousand tokens, they say a decision model "probably can't replace the LLM's job."

Two more caveats from the post itself:

  • On long tickets the decision model drops the oldest comments to fit its window. For some ticket types the oldest comment is the one that matters.
  • JSON key order changed the answer. The same content with keys in a different order moved output probabilities by up to 0.19, so they now send sorted keys. If you compare runs, check this first.

The Hacker News discussion also made a fair point: the same team could perhaps have tried a cheaper LLM tier, and a classic classifier trained on labelled tickets is an older answer to the same problem. Several commenters described the practical path as starting with an expensive general model to get moving, then using its outputs to justify and train something smaller once the workflow is stable.

What to take from it

For most companies running AI automations, the lesson is not "switch to decision models." It is a short audit:

A short audit

  1. List every step where an LLM is asked to pick from a fixed set of answers: routing, tagging, yes/no checks, priority levels.
  2. For each one, note the volume and the cost per call.
  3. Tighten the options into short, unambiguous criteria.
  4. For the high-volume ones, run a cheaper option in parallel, keep the current answer in charge, and review every disagreement before switching.

A lot of AI spend sits in steps where the model is asked to write a paragraph so that a program can read one word out of it. Those are the first places to look.

If you want help finding those steps in your own workflows, get in touch.

Sources