LLM Token Counter

Estimate tokens and API cost for GPT, Claude, Gemini, Llama and open models. Check context window fit and compare prices across models instantly.

Advertisement

LLM Token Counter and API Cost Estimator

Paste your prompt and this tool estimates how many tokens it will consume, what an API call will cost, and how much of the model’s context window it fills. It covers the major commercial models — GPT-4o, GPT-4 Turbo, GPT-4, GPT-3.5 Turbo, o1, Claude Opus 4, Claude Sonnet 4, Claude 3.5 Sonnet, Claude 3 Haiku, Gemini 2.0 Flash, Gemini 1.5 Pro, Amazon Titan — plus open models such as Llama, Qwen, Gemma, Mistral, and DeepSeek, with the ability to search Hugging Face for any other model by name.

One thing to be upfront about: this is an estimator, not a tokenizer. It does not run tiktoken or any model’s actual vocabulary. It applies a per-family heuristic and typically lands within 5–15% of the true count, with the error depending on your text’s language, code density, and punctuation. For exact counts — billing reconciliation, a hard context-limit check — run the official tokenizer for your model. For the thing people actually use a token counter for, which is deciding whether a prompt is affordable and whether a document will fit, an estimate in that range is enough to make the decision.

What a Token Actually Is

Language models do not read characters or words. They read tokens: sub-word fragments produced by a tokenizer that was trained to split text efficiently over the model’s training corpus. Common English words are usually one token each. Rare words split into pieces. Whitespace is typically attached to the following word, so “ the” and “the” can be different tokens.

The rule of thumb you will see everywhere — roughly 4 characters per token for English prose, or about 0.75 words per token — is a decent starting point, and it is where this tool starts too. But it is an average over one kind of text, and the moment your input stops being English prose the ratio moves:

  • Code tokenizes worse than prose. Indentation, brackets, operators, and camelCase identifiers all fragment. A file of Python can run closer to 3 characters per token.
  • Numbers are often split into small digit groups, so a table of figures is far more expensive per character than it looks.
  • Non-English text is dramatically worse on tokenizers with smaller vocabularies. Languages written in non-Latin scripts can cost two to four times as many tokens per character as English saying the same thing.
  • JSON and markup spend tokens on braces, quotes, and repeated key names — a real cost when you are stuffing structured context into a prompt.

Why Token Counts Differ Between Models

There is no universal token. Each model family ships its own tokenizer with its own vocabulary, and vocabulary size is the dominant factor: a bigger vocabulary packs more characters into each token, so the same text produces fewer tokens.

That is exactly how this tool’s estimator is calibrated. Rather than one global ratio, it uses a characters-per-token figure per model family, derived from that family’s vocabulary size:

FamilyApprox. vocabChars per token used
Gemma256K4.2
OpenAI (o200k)200K4.2
Moonshot160K4.0
Qwen152K3.8
Mistral (Nemo and later)131K3.7
DeepSeek129K3.8
Llama128K3.8

On top of the base character division, the estimator adds small adjustments for word count, punctuation, and numeric runs, since punctuation and digits tend to become tokens of their own. That is why the number here is closer than a flat “divide by 4” but still not exact.

The practical consequence: never carry a token count from one model to another. The same prompt priced against GPT-4o and against Llama 3.1 gives different token totals, and comparing providers on price-per-million-tokens alone without accounting for tokenizer efficiency will mislead you.

How to Use the Tool

  1. Paste your text into the counter tab, or load the sample to see the behaviour. Character, word, line, and sentence counts appear alongside the token estimate.
  2. Pick a model. Switch between the commercial API models and the open/local models; for open models you can also search Hugging Face directly if the one you want is not in the list.
  3. Set the output ratio. This multiplies your input tokens to estimate response length, because output tokens are billed separately and usually cost two to five times more than input tokens. A ratio of 1 assumes a reply as long as the prompt; a summarisation task might be 0.2, a code-generation task 3 or more.
  4. Read the cost. Input cost, output cost, and total are shown for the selected model. Open models show no per-token cost, because self-hosting has no per-token price.
  5. Check the context bar. The tool shows what percentage of the model’s context window your input occupies and warns explicitly when it does not fit.
  6. Compare across models in the comparison tab, which prices the same text against every model in the table at once and flags which ones cannot hold it.
  7. Copy or share. Copy a formatted summary of the analysis, or share a link that reopens the tool with your text and model selected.

Pricing figures are public list prices as of June 2026 and are labelled as such in the interface. Providers change prices frequently, sometimes with little notice, so confirm current rates with the provider before committing to a budget.

Worked Example: Why the Output Ratio Matters

Suppose you are building a document-summarisation feature. Each call sends a 12,000-token document and receives a roughly 600-token summary — an output ratio of 0.05. If input costs $2.50 per million tokens and output $10.00 per million, one call is $0.030 of input plus $0.006 of output, about $0.036. At 50,000 documents a month, that is roughly $1,800.

Now flip it to a code-generation feature: a 1,000-token prompt producing 4,000 tokens of code, a ratio of 4. Input is $0.0025, output is $0.040 — the output dominates completely, at more than 90% of the cost. Estimating with input tokens alone would have understated the bill by a factor of sixteen. This is the single most common budgeting mistake with LLM APIs, and it is why the ratio control exists rather than being hidden behind an assumption.

Context Windows and What They Cost

The context window is the hard limit on input plus output for a single request. The models here range from 8,192 tokens (GPT-4, Amazon Titan) through 128,000 (GPT-4o, GPT-4 Turbo, Llama 3.x), 200,000 (o1 and the Claude models), up to 1–2 million for Gemini.

A large context window is a capability, not a free one. You pay for every token you put in it, on every call. Filling a 200,000-token window on each request when 8,000 tokens of retrieved context would answer the question is a twenty-five-fold cost increase for no benefit — and often a quality loss too, since models are measurably worse at using information buried in the middle of a very long context. Use the context bar here to see how much headroom you actually have, then aim to use less of it.

Related Tools

If the answer to your cost problem is running a model yourself, size the hardware first with the LLM VRAM calculator, check throughput with the inference speed calculator, and compare the total against API pricing with the self-hosted LLM cost calculator.

Frequently Asked Questions

How accurate is this token counter?

It is a heuristic estimate, typically within 5–15% of the true count. It does not run the model’s real tokenizer. Use it for budgeting and context-fit decisions; use the official tokenizer library (tiktoken for OpenAI, and each provider’s equivalent) when you need an exact number.

How many tokens is 1,000 words?

Roughly 1,300 tokens for ordinary English prose, using about 0.75 words per token. Code, numeric data, and non-English text will be higher — sometimes much higher.

Why does the same text give different token counts for different models?

Because each model family uses a different tokenizer with a different vocabulary. Larger vocabularies encode more characters per token. This tool accounts for that with a per-family characters-per-token figure rather than one global ratio.

Are the prices current?

They are public list prices verified as of June 2026 and labelled with that date in the tool. Provider pricing changes often — confirm with the provider before basing a budget on them.

Why do output tokens cost more than input tokens?

Input tokens are processed in parallel in a single forward pass, while output tokens are generated one at a time, each requiring a full pass through the model. Generation is the expensive part, and pricing reflects it — typically two to five times the input rate.

What is the output ratio setting?

It is your estimate of how long the response will be relative to the prompt, used to project output token cost. Set it low for summarisation and classification, high for generation and long-form writing.

Can I count tokens for open source models?

Yes. Switch to the open models source for Llama, Qwen, Gemma, Mistral, DeepSeek, and others, or search Hugging Face for a model by name. Open models show token counts and context fit but no per-token price, since self-hosting is not billed per token.

Is my text sent anywhere?

Token estimation and cost calculation run entirely in your browser. The only network request the tool makes is to the Hugging Face model index if you actively search for a model by name — that sends your search term, not your prompt text.

What Is LLM Token Counting

Tokens are the fundamental units that large language models (LLMs) use to process text. Unlike words or characters, tokens are subword units determined by the model's tokenizer — a word might be a single token, or it might be split into multiple tokens depending on its frequency in the training data. Understanding token counts is essential for managing API costs, staying within context window limits, and optimizing prompt engineering.

Different LLM providers use different tokenizers, meaning the same text produces different token counts depending on the model. This tool counts tokens for popular models so you can estimate costs and ensure your prompts fit within context limits.

Tokenizer Comparison

Model FamilyTokenizerAvg. Tokens per WordContext Window
GPT-4 / GPT-4ocl100k_base (tiktoken)~0.75128K tokens
Claude 3.5Custom BPE~0.75200K tokens
Gemini 1.5SentencePiece~0.81M-2M tokens
Llama 3Custom BPE~0.8128K tokens
MistralSentencePiece~0.832K-128K tokens

How Tokenization Works

The text "Tokenization is important" might tokenize as:

TextGPT-4 TokensCount
"Hello world"["Hello", " world"]2
"Tokenization"["Token", "ization"]2
"🎉"["🎉"]1
"antidisestablishmentarianism"["ant", "idis", "establish", "ment", "arian", "ism"]6

Common English words are typically single tokens. Rare words, technical terms, and non-English text split into more tokens.

Common Use Cases

  • Cost estimation: Calculate API costs before running requests. At $0.01/1K input tokens (GPT-4o), a 10,000-token prompt costs $0.10 per request.
  • Context window management: Ensure your prompt plus expected response fits within the model's context window. Exceeding the limit causes truncation or errors.
  • Prompt optimization: Identify wordy prompts that use excessive tokens and optimize them to reduce costs and latency.
  • RAG pipeline tuning: Determine how many retrieved context chunks fit within the available context window alongside the system prompt and user query.
  • Batch processing estimation: Before processing thousands of documents through an LLM API, estimate total token usage and costs.

Best Practices

  1. Account for both input and output tokens — API costs include both the prompt (input) and the response (output). Budget for the maximum expected response length.
  2. Include system prompts in your count — System prompts consume tokens on every request. A 500-token system prompt across 10,000 requests is 5 million tokens.
  3. Use the correct tokenizer — Token counts vary between models. Always count using the tokenizer for your target model to get accurate estimates.
  4. Monitor token usage in production — Track actual vs estimated token usage. Unexpected increases may indicate prompt injection attacks or inefficient prompts.
  5. Optimize prompts for token efficiency — Remove redundant instructions, use concise language, and structure prompts to minimize token usage without sacrificing quality.

Related tools

This tool is provided for informational and educational purposes only. All processing happens in your browser — no data is sent to or stored on our servers. While we strive for accuracy, we make no warranties about the completeness or reliability of results.