The LLM cost control playbook

LLM spend behaves differently from cloud spend. It grows with usage rather than capacity, it varies tenfold between models doing similar work, and the invoice can't tell you which feature or customer caused it. This playbook sets out how engineering and finance teams can attribute, price, budget and reduce it.

1. Where the money goes

Every provider bills by token, but not every token costs the same:

Two consequences follow. First, input usually dominates volume: retrieval and long conversation histories make prompts far longer than replies. Second, output dominates price per token, and reasoning models produce output you never see. A cost model that ignores either will be wrong.

Model choice matters more than anything else. On our price list, output for a small model can cost a twentieth of a flagship model's from the same provider. Most savings come from using the smallest model that does the job.

2. Attribute every call

Provider dashboards group spend by organization, project and API key. Your questions are about features and customers. Close the gap by recording, for every call:

• Service or feature: which part of your product made it.
• Model and provider, including the deployment or region it ran in.
• Customer: the account it was made for, if you sell to businesses.
• Tokens by type: input, cached input, cache writes and output, as returned in the response.
• Outcome: success, error type, retries and latency.
• Trace: which user request the call belonged to, so multi-step agents add up correctly.

3. Price each call correctly

Multiplying tokens by a model's headline price gets close, but three things move the real number:

Regions and data residency

Azure OpenAI Data Zone deployments and Bedrock geographic inference profiles cost about 10% more than global ones; GovCloud about 20% more. If some of your traffic stays in a region, price those calls at that rate. See why your Azure OpenAI and Bedrock bills run 10% above list.

Caching

Cached input is billed at a fraction of the input price, and on Anthropic cache writes cost more. Price each token type separately. See prompt caching: when it saves money.

Price changes

Providers change prices and release cheaper versions often. Keep a versioned price list so that last month's calls stay priced at last month's rates and reports don't shift when a price changes.

Batch processing, priority tiers and surcharges for very long prompts also change the price. If you use them, either record the tier per call or treat estimated costs for those workloads as approximate.

4. Know your unit economics

Once each call is attributed to a customer, you can compare AI cost with revenue per account. A simple view for a B2B product with a $99-a-month plan:

Illustrative figures, not customer data.

The average hides the outliers, and outliers are where margin disappears. Look at the distribution of cost per customer every month, and decide on purpose what happens at the top: a higher plan, a usage limit, a cheaper model for bulk work, or acceptance.

5. Budgets, forecasts and alerts

Invoices arrive after the money is spent. Set up signals that arrive before:

• Monthly forecast: projected spend for the month from the run rate so far, compared with budget. Alert when the forecast exceeds it.
• Spend limits per service and per customer, for runaway loops and abuse.
• Spikes: spend or call volume far above the same hour last week.
• Reliability: error rate, p95 latency, and services that stop reporting. Failures cost money too, through retries and lost users.

Send alerts where the owning team already works (Slack, Microsoft Teams or email) and give each one an owner. An alert nobody owns is noise.

6. Reduce spend without hurting quality

In rough order of savings per hour of work:

1. Use a smaller model where it's good enough. Classification, extraction, routing and summarizing short text rarely need a flagship model. Test on real examples, then switch the service.
2. Cache stable prefixes. Put instructions and tool definitions first and keep them identical between requests.
3. Trim input. Retrieve fewer, better chunks; summarize long histories; drop unused tool definitions.
4. Cap output. Set maximum output tokens and ask for concise formats.
5. Batch work that can wait. Overnight and bulk jobs often qualify for batch discounts.
6. Fix retries. A call that fails twice and succeeds the third time costs three times as much. Watch retry rates per service.

Measure each change on the same service and hours before and after, in cost per request and in a quality check you trust.

7. Governance: models, keys and data

• Model retirements. Providers publish retirement dates. Track which services still call a model due to retire, and move them before the date.
• Keys. Store provider keys as secrets, rotate them, and know which services use which key.
• Data. Decide whether prompts and responses may be stored for debugging. If they are, mask personal data before it leaves your service and set a retention period.
• Ownership. Every AI service has a named owner who receives its alerts and reviews its cost monthly.

8. The quarterly checklist

• Every LLM call is recorded with service, model, region, customer, tokens by type and outcome.
• Calls are priced from a versioned price list, including regional prices and caching.
• Each service has an owner, a monthly budget and a forecast alert.
• Cost per customer is reviewed monthly, with a decision for the top 5% of accounts.
• The three most expensive services have been tested on a smaller model.
• Cache hit rates are checked for services with long, stable prompts.
• Retry and error rates are alerted per service.
• No service calls a model that retires within 90 days.
• Prompt storage, masking and retention are written down and enforced.

Alye covers the recording, pricing, budgets, alerts and retirement warnings in this playbook with two lines of code per service. See how LLM cost tracking works or start free.