LLM API Pricing Compared: What 1M Tokens Really Costs

Published 2026-09-19 · by Tool Station · 100% private tools, no upload

Model pricing pages quote a number per million tokens, which is almost useless on its own. What you actually pay depends on the split between input and output tokens, how chatty your prompts are, and whether you can use caching or batch mode. Here is how to work it out — and a calculator that does it for you.

Why "price per million tokens" misleads everyone

Almost every provider advertises a single headline rate. In practice you buy two different things at two different prices: input tokens (what you send — system prompt, user message, retrieved documents, chat history) and output tokens (what the model writes back). Output is consistently priced at three to five times input, because generation costs far more compute than reading.

That asymmetry flips the usual intuition. A customer-support bot that replies in two sentences but is fed a 4,000-token product manual and the full conversation history is overwhelmingly an input workload. A code generator that receives a 50-line file and writes a 600-line module is an output workload. Two apps with identical request counts can differ by an order of magnitude in cost.

There is a second trap: chat applications resend the entire conversation on every turn unless you deliberately trim it. By turn twenty you may be paying to re-read 20,000 tokens of history to generate 200 new ones. Most "unexpected invoice" stories trace back to this single behaviour.

How the major providers structure pricing

Without quoting numbers that will be stale by the time you read this, the structural differences matter more than the rates:

• Tiered model families. Most vendors ship a cheap fast model, a mid model and a flagship reasoning model. The gap between the cheapest and the most capable is routinely 20–50x. Route by task, not by habit.
• Context-length tiers. Several providers charge more per token once a request exceeds a context threshold (commonly around 128K–200K tokens). Long-context requests are not simply "more of the same".
• Prompt caching. If you repeatedly send the same large prefix — a system prompt, a manual, a codebase — cached input tokens are typically billed at a small fraction of the standard rate. This is the single biggest lever for RAG and agent workloads.
• Batch mode. Non-interactive jobs submitted to a batch endpoint usually receive a substantial discount (commonly around half price) in exchange for completion within a day instead of seconds. Nightly classification, summarisation and embedding jobs belong here.
• Free or heavily discounted tiers. Some providers run experimental models at zero or near-zero cost, which is genuinely useful for prototyping but comes with rate limits and no reliability guarantees.

Worked example: a support chatbot

Take a realistic support bot. Each conversation runs eight turns. The system prompt plus product documentation is 3,000 tokens and is re-sent every turn because the conversation is stateless. The user writes about 40 tokens per turn, and the assistant replies in roughly 120 tokens.

Two conclusions fall out immediately. First, the system prompt is roughly 86% of the bill — before a single user word. Second, if you enable prompt caching on that 3,000-token prefix, the dominant cost collapses to a fraction of its current size. Trimming history is worth far less than caching the prefix, yet teams usually do the opposite because trimming is easier to implement.

At 10,000 conversations a month that is about 278M input tokens and 9.6M output tokens. Multiply by your provider's current rates and you have a real budget number — and that is exactly the arithmetic the calculator below performs, including the model-comparison step.

Worked example: nightly document summarisation

Now a batch workload: 20,000 support tickets per night, each about 900 tokens, each producing a 120-token summary. That is 18M input and 2.4M output tokens per night, roughly 540M input per month. Two things change the answer dramatically: submit it to the batch endpoint if one is available, and use the cheapest model that produces an acceptable summary. Summarisation is precisely the task where a small model matches a large one. Teams routinely overspend here by routing everything to a flagship model out of caution.

How to cut the bill without changing models

1. Cache the stable prefix. Put everything that never changes at the front of the prompt so it can be cached, and keep volatile content at the end.
2. Stop resending history verbatim. Summarise older turns instead of replaying them. A rolling 200-token summary usually preserves enough context.
3. Trim retrieved context. Ten retrieved chunks are rarely better than the four best-ranked ones, and reranking is cheaper than stuffing.
4. Cap output. Set a sensible max_tokens. An unbounded generation occasionally rambles for thousands of tokens and you pay for all of it.
5. Route by difficulty. Classify the request first with a cheap model, escalate only the hard cases. Most production traffic does not need the flagship.
6. Use batch endpoints for anything asynchronous. If nobody is waiting at a screen, there is no reason to pay interactive rates.
7. Log tokens per request. You cannot optimise what you do not measure — and per-request token logging is what turns "the API bill is high" into "this one endpoint is 60% of it".

What prices do over time

The durable trend is downward: each generation of models has delivered similar capability at a lower price per token, and aggressive entrants periodically force across-the-board cuts. The practical implication is architectural rather than financial — do not build a system whose economics depend on today's rates. Keep the model behind an interface, log token usage per feature, and re-evaluate routing every quarter. Sites that publish price tables go stale within weeks, which is why the comparison tool on this site takes your numbers and your model choice rather than pretending to hold a live price feed.

A sanity check on "free" and "unlimited" offers

Openrouter-style aggregators and free tiers are excellent for prototyping, and worth watching for rate limits, data-retention terms and the possibility that a model disappears. If a free tier stores your prompts for training, that is a real cost even when the invoice says zero — particularly for anything touching customer data, contracts or source code.

Frequently asked questions