Self-Hosted LLM Cost Calculator

Self-hosted LLM cost calculator: when buying a GPU beats paying per token

This tool answers one financial question. Given a monthly token volume, a commercial API you would otherwise call, and a specific piece of hardware you would otherwise buy or rent, how many months until the hardware has paid for itself — and can that hardware physically serve the volume at all? It puts the API bill and the self-hosting bill side by side, plots both cumulatively over 24 months, and delivers a verdict.

The honest headline, before any of the arithmetic: self-hosting wins on cost in two situations — very high sustained volume, or hardware you already own and are not paying to acquire. At hobby volumes against a cheap hosted open model, buying a GPU does not pay back within the useful life of the card. The tool is built to show you that rather than to sell you a build.

What goes into each side of the comparison

Volume is entered either as one of four presets or as raw monthly input and output token counts. The presets are roughly: a hobbyist at 30 million input and 9 million output tokens a month; a small app at 300 million and 90 million; a growing app at 3 billion and 900 million; and a high-volume service at 30 billion and 9 billion. The three-to-one input-to-output ratio in those presets reflects typical chat and RAG traffic, where prompts are long and answers are short. If your workload is a summariser or a code generator, override it — output tokens are the expensive ones on both sides of the comparison.

The capacity check is the part people skip

Cost per month is meaningless if the box cannot keep up. Before comparing money, the calculator estimates single-stream generation speed for the open model you have chosen on the hardware you have chosen, then works out how many compute-hours per month your volume needs. There are 730 hours in an average month; if the requirement exceeds that, you get an explicit warning that the hardware cannot serve the volume, no matter how good the break-even looks.

This bites earlier than expected. An RTX 4090 running Llama 3.3 70B generates around 14 tokens/sec single-stream. The small-app preset's 90 million output tokens a month would take roughly 2,300 compute-hours — more than three months of continuous running crammed into one month. Two used RTX 3090s in tensor-parallel roughly double the rate to around 25 tok/s and still need about 1,300 hours. The same preset against a mixture-of-experts model changes the picture completely: Qwen3 30B-A3B on one 4090 projects to about 232 tok/s and needs roughly 143 hours, comfortably inside the month.

That is the single most useful output of the tool. If your volume needs a serious dense model, single-stream self-hosting runs out of hours long before it runs out of money, and the answer is batched serving, more GPUs, or a smaller or sparser model.

Compare like with like, or the verdict is meaningless

The dataset includes frontier models, mid-tier and budget models, and — crucially — open models served by hosted inference providers. Choosing a frontier API as your baseline makes self-hosting look extraordinary, because you are comparing a top-end proprietary model against electricity. At the small-app preset a frontier model at $3 per million input and $15 per million output tokens costs about $2,250 a month; running Llama 3.1 8B on a 4090 you own costs roughly $17 a month in power. Of course the GPU "pays for itself in under a month". You have also swapped models, and an 8B is not a frontier model.

The apples-to-apples comparison is against a hosted version of the same open weights. Run those and the numbers become sober:

Read those rows together and the shape of the answer appears. Against a cheap hosted open model, a GPU bought specifically to save money is a bad trade at hobby and small-app volumes — the hosted rate for an 8B model is so low that electricity alone can exceed it. Break-even only becomes credible once volume is high enough that the monthly API bill runs into the hundreds or thousands, and precisely at that point the capacity warning starts firing and the true cost is several GPUs, not one. The one unambiguous win is the last row: if the hardware is already on your desk and already paid for, the marginal cost is a few dollars of electricity and self-hosting is cheaper immediately.

The hardware and models on offer

The self-hosting side offers three categories, and which one you pick changes the shape of the answer more than any other input:

• Buy consumer hardware — a used RTX 3090, an RTX 4090, an RTX 5090, a pair of used 3090s for 48 GB of combined VRAM, or a large-memory Mac Studio. These carry an up-front cost and a wattage, so they generate a genuine payback period.
• Rent cloud GPUs — hourly RTX 4090, A100 80 GB or H100 80 GB. No up-front cost, so break-even is either immediate or never; renting is a pure monthly comparison against the API rate, and at low volumes the hourly rate usually loses.
• Hardware you already own — zero purchase price, electricity only. This is the configuration that most often beats an API, and it is worth checking first before considering a purchase.

Alongside the hardware you choose which open model would be running — the list spans Llama 3.1 8B, Qwen3 14B, Qwen3 30B-A3B, Qwen3 32B, a Gemma mixture-of-experts entry, Llama 3.3 70B and GPT-OSS 120B. The choice feeds the speed and capacity estimate at 4-bit quantisation, so a sparse mixture-of-experts model and a dense model of similar size produce completely different hour counts on identical hardware. The Mac Studio option is instructive here: at roughly 90 W it has by far the lowest running cost of the purchase options and enough unified memory for large models, but its bandwidth caps generation speed, so it uses more of the month's hours to deliver the same tokens.

Reading the 24-month chart

The chart plots cumulative spend rather than monthly spend, which is the right view for a purchase decision. The API line starts at zero and climbs at a constant slope. The self-hosting line starts at the purchase price and climbs slowly. Break-even is where they cross. If they have not crossed by the right-hand edge, the hardware has not paid for itself in two years, and the tool states plainly that a payback beyond about 36 months is longer than the realistic life of the card — at which point the API is the practical choice regardless of what the arithmetic says about month 40.

How the electricity figure is built

For owned hardware the calculator does not simply multiply the card's rated draw by 730 hours. It charges the full load wattage only for the hours the model is actually computing, and adds a much smaller idle draw scaled from the card's power rating for the remaining hours, then multiplies the total by the rate per kilowatt-hour you enter. The default rate is $0.15/kWh; change it, because power prices vary by more than a factor of three between regions and it moves the answer materially at high utilisation. Cloud rentals are billed only for the hours needed, which assumes you are disciplined about shutting instances down — a rented GPU left running idle is charged like one under full load, and this model does not simulate that mistake for you.

What the model deliberately leaves out

• Batched serving. The capacity estimate is single-stream. Production engines such as vLLM amortise the weight reads across concurrent requests and can multiply throughput several-fold, which makes self-hosting substantially more attractive at exactly the high volumes where capacity was the blocker. Treat the hours figure as a conservative floor.
• Resale value. GPUs retain value; ignoring that understates the case for buying.
• Your time. No line item covers driver updates, engine upgrades, model evaluation, monitoring, or the night the box falls over. For a one-person project this is usually the largest real cost and it does not appear on either side of the ledger.
• The rest of the machine. Purchase prices here are for the accelerator; a host system, power supply and cooling are extra.
• Capability. A self-hosted open model is not a frontier model. The tool says so on screen, and it is the assumption most likely to invalidate a favourable break-even.

The reasons to self-host that are not about money

Cost is the question this calculator answers, but it is often not the deciding one. Data residency — prompts that legally or contractually cannot leave your premises — makes the break-even irrelevant, because the API is not an option at any price. So does the need for predictable spend rather than a bill that scales with usage, freedom from rate limits and provider deprecations, offline operation, or running a fine-tune that no provider hosts. If one of those applies, use the tool to size the hardware and understand the running cost rather than to justify the decision.

Conversely, if none of them applies and your volume sits at hobby or small-app scale, the arithmetic here will usually tell you to keep paying per token. That is a legitimate result, and it is cheaper than finding it out after the card arrives.

Prices go stale

API rates and cloud GPU rates move frequently, and the dataset carries the date it was last checked on screen. Used-GPU prices in particular are a moving target. Before committing to a purchase on the strength of a break-even in months, re-check the current price of the specific card and the current per-million rate of the specific API — a 20% move in either can turn a fourteen-month payback into a thirty-month one.