Question 1

How do I calculate the cost of an LLM API call?

Accepted Answer

Cost per request is input tokens divided by one million times the input price, plus output tokens divided by one million times the output price. Output tokens are usually priced two to four times higher than input tokens, so a request that reads a long prompt but writes a short answer costs less than the token count alone suggests, and a request that generates a lot of text costs more. Multiply cost per request by requests per day and by about 30.4 for a monthly figure. Prompt caching reduces the billed input tokens on repeated context. The calculator does all of this live: enter your token counts, prices, and volume, and it returns cost per request, per day, per month, and the blended dollars-per-million-tokens rate.

Question 2

Why are output tokens more expensive than input tokens?

Accepted Answer

Output tokens are generated one at a time in the autoregressive decode phase, where the model reads all of its weights and the entire KV cache from memory for every single token produced. That makes decoding memory-bandwidth-bound and slow relative to the input (prefill) phase, which processes the whole prompt in parallel and is far more hardware-efficient. Providers price this asymmetry in, typically charging two to four times more per output token than per input token. The practical consequence is that generation-heavy workloads — long completions, chain-of-thought reasoning, agentic loops that emit many tokens — are where cost concentrates, and where limiting output length or using a cheaper model for the generation step pays off most.

Question 3

When is it cheaper to self-host an LLM than to use an API?

Accepted Answer

Self-hosting on rented GPUs is a fixed monthly cost — you pay for the GPUs 24 hours a day whether or not requests arrive — while API pricing is purely per-token and scales with usage. So self-hosting wins only above a break-even volume, where the fixed GPU rent divided across your tokens drops below the per-token API price. Below that volume the API is cheaper because you are not paying for idle capacity. The calculator computes this break-even in requests per day from your GPU count, hourly GPU price, and throughput. Crucially, the honest self-host cost is higher than the raw GPU rent: you must add engineering and on-call time, the reliability and utilization risk of running the fleet, and the fact that real utilization is rarely 100 percent. Treat the break-even as a floor that reality pushes upward, not a precise crossover.

Question 4

How much does prompt caching reduce LLM cost?

Accepted Answer

Prompt caching reduces the input-token cost on the portion of the prompt that repeats across requests — a shared system prompt, a fixed set of instructions, a document reused across many questions. Cached input tokens are billed at a large discount or not reprocessed at all, so a workload with a long, stable prefix and a short variable suffix can see input costs fall substantially at high cache-hit rates. It does not reduce output-token cost, which is generated fresh every time, so its impact depends on how input-heavy your workload is. The calculator applies a cache-hit slider to the input side; set it to your realistic hit rate to see the effect. Caching is most valuable for high-volume, high-overlap workloads and negligible for one-off, all-unique prompts.

Question 5

What is the blended dollars-per-million-tokens rate?

Accepted Answer

The blended rate is the total cost of a request divided by its total tokens (input plus output), expressed per million tokens. It is useful because published prices split input and output, but your actual cost depends on your input-to-output ratio, so a single blended number lets you compare workloads and providers on a like-for-like basis. A retrieval-augmented request with a 3000-token context and a 200-token answer has a very different blended rate than a short-prompt, long-generation creative task, even on the same price sheet. Tracking your blended rate over time is a good FinOps signal: if it climbs, either your prompts are growing, your outputs are lengthening, or your cache-hit rate has fallen.

Question 6

How do I estimate the monthly cost of an AI feature before launch?

Accepted Answer

Estimate four inputs and let the arithmetic follow: expected requests per day, average input tokens per request, average output tokens per request, and the price of the model you plan to use. Requests per day comes from your product forecast; token counts you can measure from a prototype against real prompts rather than guessing. Multiply out to a monthly figure and then stress-test it: model a growth multiple on volume, a longer-context version of your prompt, and a worst-case output length, because AI-feature cost overruns almost always come from underestimating one of those three. Add a margin for retries and failed generations. The result is the gross cost input to the unit-economics question of whether the feature can be priced profitably — flat pricing on a token-metered backend is the classic margin trap.

Question 7

Does this calculator use current OpenAI or Anthropic prices?

Accepted Answer

No, and that is deliberate. Model prices change frequently and vary by model, region, and contract, so hardcoding a specific vendor’s current price into a calculator would make it wrong within weeks. Instead, the price fields are your editable inputs, and the tier presets are round, illustrative ballpark figures — not any product’s official price — meant only as a starting point you replace with your own quote from the vendor’s pricing page. This keeps the tool accurate indefinitely: the math is evergreen, and you supply the current number. Always confirm pricing on the provider’s own page before committing to a budget.

Question 8

What costs does this calculator leave out?

Accepted Answer

It models the direct token or GPU cost of inference, which is the dominant and most controllable line, but a full cost picture includes more. On the API side, it omits embeddings, fine-tuning, image or audio modalities, and rate-limit-driven retries. On the self-host side, it counts rented GPU time but not the engineering and on-call labor to operate the fleet, storage and networking, the cost of low utilization, or the reliability risk of running your own serving stack. It also does not model throughput or latency, which are separate calculations, or the memory sizing that decides how many GPUs you need in the first place — for that, use the VRAM sizing calculator linked below. Treat the output as the inference-cost baseline for planning, then layer the operational costs specific to your deployment.

LLM Inference Cost Calculator