How to forecast LLM cost per user before you launch

Your LLM cost per user is tokens per turn, times turns per session, times sessions per user per month, times the blended token rate, minus whatever your cache actually absorbs. Everything else is commentary. For a mid-weight assistant feature with retrieval, that arithmetic usually lands between one and four dollars per active user per month. Your p95 user costs five to ten times that, and that’s the number deciding whether the feature has a margin.

The finance posts on this topic are correct and useless. They tell you gross margin matters, that token costs are variable rather than fixed, and that you should track cost per active user. None of them show you the multiplication. This one does.

How to calculate LLM cost per user, in one expression

Write it out with every input exposed. If a number hides inside another number, you can’t sensitivity-test it, and sensitivity is the entire value of the exercise.

cost_per_user_month =
    sessions_per_user_month
  x sum_over_turns( input_tokens(turn) x effective_input_rate
                  + output_tokens(turn) x output_rate )

effective_input_rate =
    (1 - cache_hit_rate) x input_rate
  + cache_hit_rate x cached_read_rate

Five inputs. Two of them get published by your provider and change without your permission. Two of them are architecture decisions you control. The fifth, sessions per user per month, you can’t know before launch, and it carries the widest variance of the set. Hold that thought.

A worked request path, framed as an illustration

Take a support-reply assistant inside a B2B product. An agent opens a ticket, the assistant drafts a reply against the company’s own help centre, the agent pushes back a few times, then sends. It’s an invented feature used to show the method, not a system we built for anyone.

Count what actually goes over the wire on a single turn:

The third row is where forecasts break. Conversation history is replayed in full on every turn, so a six-turn session doesn’t cost six times the first turn. It costs the sum of a growing prefix. First turn: 4,520 input tokens. Sixth turn: 6,870. Total for the session, roughly 34,200 input tokens and 2,100 output tokens.

The multiplication

Twenty sessions per user per month gives 684,000 input tokens and 42,000 output tokens per user. At an illustrative mid-tier rate of $2.50 per million input and $10 per million output, that’s about $1.71 of input and $0.42 of output. Call it $2.13 per active user per month before caching, and check live provider pricing before you quote anyone, because these numbers move.

Now apply a cache. The 4,400-token prefix of system prompt plus retrieved context is identical on every turn after the first. If cached reads price at roughly a tenth of fresh input and you land a 60% hit rate across all input tokens, the input line drops from $1.71 to about $0.79. The feature now costs roughly $1.21 per active user per month.

That’s the whole method. Six numbers, one multiplication, an afternoon. On its own it’s also wrong, in a specific and predictable direction.

Why the average user is the wrong unit

The mean is a fine description of a normal distribution and a terrible description of a power law. AI feature usage is a power law. A small group of users runs the feature constantly, in longer sessions, on harder tickets, and they don’t appear in your pilot cohort because your pilot cohort was picked for enthusiasm rather than volume.

The clearest published worked example of this comes from Cycles’ write-up of AI agent unit economics, where an anonymised B2B SaaS company priced a support copilot at $15 per user per month and modelled $3 per user in LLM cost, targeting an 80% gross margin. By month three the actual figure was $11.50 per user and the gross margin on the feature was 23%. Their distribution ran 70% of users under $3 a month, 20% between $8 and $25, and 10% between $40 and $120. The top decile consumed 72% of total spend. One user, running automated tests against the copilot endpoint, cost $310 on their own.

Treat that as an anonymised illustration, because that’s how its author presents it. The shape is the point, and the shape isn’t controversial. When the top decile carries most of the bill, a mean-based forecast runs structurally optimistic: the mean you measured in the pilot is not the mean of the population you’re about to onboard.

Model the p95, price against the mean

The practical fix is two forecasts. Compute the mean-user cost for your P&L. Then compute a p95 user (in our illustration, five times the sessions and ten turns instead of six, which lands near $18 to $21 a month against a $2 average) and check that number against your price. If the p95 user destroys the margin on their own, you’ve got a pricing and a design problem, and no amount of prompt trimming fixes it.

Do this before you build. Retrofitting economics into a shipped feature means changing what users already have, and users notice removals.

The four numbers people get wrong in the forecast

These are the errors we see repeatedly, in roughly descending order of damage.

History replay. As above. Multiplying a first-turn cost by turn count under-reads a long session by 30% to 50%. Anything with tool calls is worse, because every tool result also joins the transcript and gets replayed.

Retries and agent loops. A schema violation that triggers a retry doubles that turn. An agent that plans, calls, reads, and re-plans may issue four or five model calls per visible response. Forecast per model call, never per user-visible message.

Output token rates. Output is typically priced three to five times higher than input, and reasoning models bill their thinking tokens as output. A feature that looks input-dominated on a token count can be output-dominated on the invoice. Multiply before you conclude.

Non-user traffic. Evaluation runs, CI suites, and load tests bill to the same account. Untagged, your cost-per-user metric quietly includes your own engineering team.

The four levers that move the number before launch

Ordered by how much they move, with a note on which are cheap to retrofit and which are structural.

Context budget

Retrieved context is usually the largest single line, and most systems over-retrieve because k was set to twenty during a demo and never revisited. Six well-reranked chunks generally beat twenty mediocre ones on answer quality and cost a third as much. Set an explicit token budget per request, enforce it in code, and log every request that hits the ceiling.

Cheap to retrofit. It’s a config change plus a reranker, and it’s the first thing we’d touch on an existing feature. What makes it work is having something to measure against, which is the retrieval evaluation harness we argue for in the RAG piece. That article answers which technique to use, this one answers what it costs per head, and they need each other.

Model routing by task, not by tier

The common version of this lever is wrong. Teams route by a difficulty score, misclassify the hard cases, and ship worse answers to save pennies. Route by task instead, because tasks have knowable requirements. Query rewriting, intent classification, title generation, and summarising a retrieved chunk are small-model work with deterministic quality bars. The customer-facing draft is not.

Done that way, routing frequently removes half the calls from the expensive model without touching the output users read. Moderately cheap to retrofit if your call sites already go through one wrapper. Expensive if every feature calls the SDK directly, which is the usual state of a codebase six months in.

Cache design as an architecture decision

Prompt caching is not a flag you turn on. It is a constraint on how you order a prompt, and it has to be designed in. Caches key on an exact prefix, so anything that varies must sit after everything that doesn’t. A timestamp, a user name, or a randomised few-shot example placed at the top of a system prompt destroys the hit rate for every request.

Put the stable system prompt and tool definitions first, then retrieved context ordered deterministically, then the volatile turn. Hold retrieval output stable across a session rather than re-retrieving on every message. Watch your provider’s cache TTL too. A five-minute window rewards session bursts and gives you nothing on a user who returns after lunch.

This is the lever that’s genuinely expensive to retrofit. Reordering a prompt changes model behaviour, which means re-running your evaluation set, which means you needed an evaluation set. Design it in on day one or accept that you will pay to redo it.

A hard per-user ceiling, enforced in code

Every one of the runaway-cost stories has the same root cause: nothing in the system was allowed to say no. Meter tokens per user per billing period in your own database, not in the provider dashboard, and put three thresholds on it. A soft alert at maybe 3x your modelled average. A degradation step at 5x, meaning a smaller model, tighter context and no speculative pre-fetch. A hard stop at 10x, with a message that says what happened.

Cheap to retrofit and the highest-value hour of work on this list. The ceiling is also what makes seat pricing survivable, because it converts an unbounded liability into a known one.

Per seat, per usage, or credits

Seat pricing is the default because buyers understand it. It is also a bet that your cost distribution is tight, and for AI features it usually is not. Seats work once you’ve enforced a ceiling and the feature is a bounded add-on rather than the whole product.

Usage pricing aligns cost and revenue perfectly and creates a worse product. Users billed per action ration their usage, and a rationed AI feature never becomes habitual enough to defend a renewal. We’d rather absorb variance than teach people to hesitate.

Credits are the compromise, and the compromise mostly works: an allowance inside the seat price sized at roughly your p75 user, with overage that’s visible without being punitive. We’d refuse to build a credit system where the exchange rate between a credit and a token is undocumented. That is obfuscation rather than pricing, and it corrodes trust the first time a customer does the arithmetic.

Where this method breaks, including for us

The limitation to name first: the input with the largest effect on the answer is the one you know least about. Sessions per user per month is behavioural. It depends on where the feature sits in the interface, whether it’s opt-in, and whether it’s fast enough to feel free. A pre-launch forecast can be wrong on that variable by 5x, and being wrong by 5x on the dominant term means the output isn’t a prediction.

So do not treat the number as a forecast. Treat it as a sensitivity model with one question to answer: is there any plausible usage level at which this feature runs margin-negative at your intended price? If the answer is no across a 10x range of session counts, ship and instrument. If it’s yes at 3x, you’ve got a design problem to solve before launch rather than a monitoring problem to solve after it.

The second limitation is ours specifically. We can build the request path and the metering, and tell you what the arithmetic says. We cannot tell you what your users will do with a capability they’ve never had. Anyone quoting a confident per-user number for a feature that hasn’t launched is quoting you their assumptions.

When not to build this at all

If your AI feature is a chat box over your own documentation, serving under a few thousand sessions a month, do not commission a custom build and do not commission a cost model for it. The assistant features already bundled into the major SaaS platforms, and the retrieval built directly into the model APIs, cover that case for less than the cost model would take to write, and somebody else upgrades the model underneath you.

The arithmetic here starts earning its keep at roughly the point where the monthly LLM bill exceeds the cost of an engineer thinking about it. Call that a few thousand dollars a month, or a few thousand daily active users, whichever arrives first. Below that line, ship something and find out what people do with it.

Above that line, the decisions compound. Cache-friendly prompt ordering, a routing layer, and per-user metering are all cheaper to design in than to add, which is why we’d rather have this conversation before the build than after. That’s most of what a short AI architecture and readiness engagement is for, and the build that follows is ordinary retrieval and workflow automation engineering.

What to instrument on day one

You cannot manage a distribution you cannot see, and the provider dashboard shows you an account total. Log one row per model call, in your own database, with these fields: user or tenant id, feature name, model, input tokens, cached-read tokens, output tokens, latency, whether it was a retry, and a traffic-source tag separating real users from evaluation and CI runs.

Then report percentiles rather than averages: p50, p90, p99 cost per active user, refreshed weekly. A dashboard showing a single mean looks calm right up until the month the invoice isn’t. Langfuse and Helicone do this off the shelf, and the schema above is a day of work if you’d rather own it.

What a build like this costs to commission, as opposed to run, is a different question, taken apart in what building an MVP actually costs. The five builds we publish, two of them our own products and three shipped for clients, show the shape of the systems this method is written for.

The compressed version

Count the tokens on one real request path, including the system prompt, the retrieved context, and the history that gets replayed. Multiply by turns, then by sessions, then by rate. Divide the input line by your realistic cache hit rate rather than your hoped-for one. Then throw the mean away and run the same arithmetic for a user at the 95th percentile, because that user decides whether you’ve got a margin.

Fix the context budget, route by task, order the prompt for the cache, and put a ceiling in the code. Instrument percentiles from the first request. If that sounds like a conversation you want to have about a feature you haven’t shipped yet, send us the request pathand we’ll do the multiplication with you.