Chat with an LLM running entirely in your browser via WebGPU - Llama 3.2, Qwen 2.5, Phi 3.5 or your own GGUF. No signup, no API key, nothing uploaded.
This is a chat interface for a language model that runs on your own machine, in the browser tab, with no account and no API key. You pick a model, it downloads once, and from then on every prompt you type is processed locally. The conversation is never transmitted — there is no server that could see it, because generation happens in the page. Understanding exactly how that works is the difference between a tool that fits your situation and one that disappoints you, so this page is specific about the mechanism, the hardware it needs, and where it falls short of a hosted assistant.
There are two engines, and which one you get depends on your browser and hardware.
.gguf model file from your computer, which runs through llama.cpp compiled to WebAssembly. It works without a GPU, and it is markedly slower.One thing this tool is not: it does not connect to Ollama, LM Studio, or any local API endpoint on your machine. There is no host or port to configure. Everything runs inside the page. That distinction matters because "local AI chat" is used for both ideas — if what you want is a front end for a model server you are already running, this is the wrong tool and the page will link you to a builder for Ollama commands instead.
| Source | Requires | Use it when |
|---|---|---|
| Curated catalogue | WebGPU | You want something that just works. Hand-picked chat models grouped by hardware requirement, filtered at runtime to those the loaded engine actually supports. |
| Hugging Face repo | WebGPU, MLC-format weights | You want a specific model not in the list. Paste the repository name. It must be MLC-converted — ordinary GGUF or safetensors repositories will not load here. |
Your own .gguf file | Nothing but the file | No WebGPU, or you already have a quantised model downloaded. Pick the file directly. |
The catalogue is organised into three tiers, and the tool detects your GPU where it can and marks which models fit:
Alongside general chat models the catalogue includes a code-tuned option and a reasoning model that shows its working step by step, so the tier is not the only axis worth considering.
Precision here, because privacy claims deserve it.
That combination is the real appeal: no API key to obtain, no per-token cost, no rate limit, no usage policy applied to your prompts, and nothing sitting in a vendor's logs. For drafting something confidential, working through an internal document, or simply experimenting without a meter running, it is a materially different proposition from a hosted assistant.
A model that fits in a browser tab is small. That is the entire trade. Frontier hosted models are two to three orders of magnitude larger than anything here, and the gap shows up in exactly the places you would expect: factual recall on obscure topics, long multi-step reasoning, careful instruction following on complicated requests, and staying coherent over a long conversation. The tool says as much in the interface, and it is worth taking seriously before you judge the output.
| Constraint | What it means in practice |
|---|---|
| Model size vs memory | A model must fit in GPU memory to run. Choosing one above your hardware's capacity fails at load time or falls back to painful speeds. The picker flags which models fit your detected GPU. |
| Download size | Weights are hundreds of megabytes to several gigabytes. The first load of a large model is a real wait; every subsequent load is instant from cache. |
| Speed | Tokens per second are shown live while generating and on each finished reply, so you can measure your own hardware rather than guess. WebGPU on a decent GPU is conversational; the WebAssembly fallback is not. |
| Context window | Each model has a fixed limit, commonly a few thousand tokens. The status line shows how much you have used. |
| Response length | Replies are capped at roughly a thousand tokens, which is a long answer but not an essay. |
| Quality | Good at short questions, drafts, rewriting, explanation and code snippets. Unreliable on specific facts and dates. Verify anything that matters. |
Small models have small context windows. As a conversation grows, the tool keeps the system prompt plus as much recent history as fits in about 70% of the window, reserving the rest for the reply. When older turns are dropped it tells you so rather than silently losing them. The counter in the status line shows tokens used against the window, and warns when you pass three-quarters full.
The practical consequence: a long conversation gradually forgets its beginning. If you are working on one thing, restate the important constraints periodically, or clear the chat and start again with a clean summary. This is a genuine limitation of small models, not a bug in the interface, and no amount of clever trimming makes a 4,096-token window behave like a 200,000-token one.
The fast path needs WebGPU. That means Chrome or Edge, or Safari 18 and later; Firefox support is still arriving. It also needs hardware acceleration to be switched on in your browser — a surprisingly common cause of failure, because a browser with graphics acceleration disabled reports no GPU at all even on a capable machine. If WebGPU is missing the tool says why and gives the specific setting to change, rather than showing a generic error.
Beyond the browser, the binding constraint is GPU memory. A machine with 8 GB or more of video memory, or an Apple Silicon Mac with 16 GB of unified memory, comfortably runs the top tier. Integrated graphics will run the small tier and struggle above it.
The CPU fallback has its own limits, and they are worth knowing before you go looking for a file. GGUF models are capped at about 2 GB because of the WebAssembly address space, so you want a Q4 quantisation of a 1B to 3B model. The runtime is also single-threaded, which is why it is several times slower than the GPU path. And a browser cannot read the model directories that Ollama or LM Studio keep — those store weights in their own internal layout. You need an actual .gguf file you can point a file picker at.
Type and press Enter to send; Shift+Enter starts a new line. While a reply is generating you can stop it. Afterwards you can regenerate the last answer, which discards it and runs the same prompt again — useful when a small model produces something garbled on the first attempt, which happens more often than it does with a large one.
A personality control sets the system prompt from four presets — a general helpful assistant, a coding assistant, a concise mode that answers in a sentence or two, and a plain-language explainer — or you can write your own. The chosen prompt is shown in full so you know exactly what the model was told, and changes take effect from your next message. On small models the system prompt earns its keep: the concise preset in particular does more to make a 1B model feel usable than any other single setting, because it stops the model padding.
The whole conversation exports as a Markdown file, headed with the model name and date, so a session worth keeping can leave the browser on your terms. There is also a share link that records which model you had selected — it shares the model choice, never the conversation.
Until recently, chatting with an LLM meant either a cloud API (your text goes to a server) or installing software like Ollama. WebGPU changed that: browsers can now run compute on your graphics card directly, which is exactly what LLM inference needs.
This tool uses two engines. WebLLM compiles models to run on your GPU through WebGPU — this is the fast path, used for the recommended models and HuggingFace MLC repos. wllama is llama.cpp compiled to WebAssembly — slower because it runs on the CPU, but it works in any browser and can open standard .gguf model files.
When you pick a model, your browser downloads its weights from a public CDN (this is the only network traffic involved — your messages are never part of it), caches them, and loads them into GPU memory. From then on, every token the AI generates is computed on your hardware. The privacy is structural, not a policy promise: there is no server that could log your conversation even if we wanted one.
Bigger models give better answers but need more GPU memory and run slower. A practical guide:
1B models (Llama 3.2 1B, Qwen 2.5 1.5B) — run on nearly anything with WebGPU, including integrated graphics and phones. Good for quick factual questions, simple drafts, and trying out local AI. Download: under 1GB.
3B models (Llama 3.2 3B, Qwen 2.5 3B, Phi 3.5 Mini) — need roughly 4GB of GPU memory. Noticeably smarter: better reasoning, longer coherent answers, fewer mistakes. The sweet spot for most laptops with a real GPU or Apple Silicon.
7-8B models (Llama 3.1 8B, Qwen 2.5 7B, Mistral 7B) — need 6GB+ VRAM or a 16GB+ Apple Silicon Mac. These are genuinely useful assistants — the same class of model many people run with Ollama.
The tool detects your GPU and marks what fits, but the real test is trying one: if generation feels too slow (under ~5 tokens/second), step down a size. And if your hardware can handle more than the browser allows, you will get better results running models natively — see What LLM Can I Run for the full picture of your machine's capability.
Yes. The AI model runs on your own GPU (or CPU) inside your browser using WebGPU — the same technology games use for graphics. Your messages are processed locally and never sent to us or any AI provider. Conversations are saved only in your browser's local storage, and you can verify the privacy claim yourself: once a model is downloaded, the chat keeps working with your internet disconnected.
Three sources. Recommended: a curated set from Llama 3.2 1B (runs on almost anything) up to Llama 3.1 8B and Qwen 2.5 7B (needs a GPU with 6GB+ memory). HuggingFace: paste any repo with MLC-format weights. Your own file: load a .gguf model file (up to ~2GB) from your computer — it runs on CPU via WebAssembly, slower but works in any browser. The tool detects your GPU and marks which models will fit.
The first time you use a model, your browser downloads its weights (roughly 700MB for a 1B model up to ~4GB for an 8B model) and compiles it for your GPU. That download is cached by your browser, so every later session starts in seconds — and the same cached model is shared by all the local-AI tools on this site (the summarizer, PII redactor, and phishing analyzer).
Yes, after the first download. Model weights are cached in your browser, so once you have chatted with a model while online, you can load the page and keep chatting with no internet connection. This also makes it one of the few AI chat options that works on an air-gapped or restricted network (load the model first, then disconnect).
For the recommended models: a browser with WebGPU (Chrome, Edge, or Safari 18+) and a GPU. Small 1B models run on integrated graphics and Apple Silicon; the 7-8B models want a discrete GPU with 6GB+ VRAM or a Mac with 16GB+ unified memory. For .gguf files, any modern browser works — no GPU needed — but generation is slower since it runs on the CPU. Use our What LLM Can I Run tool to see exactly what your machine handles.
Two big differences. Privacy: ChatGPT sends every message to OpenAI's servers; this tool sends nothing anywhere. Capability: ChatGPT runs models with hundreds of billions of parameters; browser models top out around 8 billion, so answers are noticeably less capable — fine for quick questions, drafts, brainstorming, and learning, but not a replacement for frontier models on hard problems. You are trading some intelligence for complete privacy and zero cost.
Not directly — Ollama and LM Studio store models in their own internal formats and folders that web pages cannot read (a browser security restriction). Instead, download the .gguf file for the same model from HuggingFace and load it with the "Your .gguf file" option. Note the ~2GB browser limit: that covers Q4 quantizations of 1B-3B models. For bigger models, run them natively — our Ollama Command Builder gives you the exact command.
Model weights are cached per-website, not per-tool. When you download Llama 3.2 1B here, the Private AI Summarizer, PII Redactor, and Phishing Email Analyzer can all use it instantly without re-downloading. The "Downloaded" manager in the model picker shows everything cached and lets you delete models to free disk space.
It is the speed at which the model generates text — roughly how many words per second appear (a token is about three-quarters of a word). 10+ tok/s feels smooth; below 5 tok/s feels slow. The number depends almost entirely on your GPU's memory bandwidth, which is why the same model is fast on an RTX 4090 and slow on integrated graphics. Our LLM Inference Speed Calculator predicts this number for any GPU and model combination.
Model size. The models that fit in a browser have 1-8 billion parameters; frontier cloud models have hundreds of billions plus extensive fine-tuning. Small models make more factual mistakes, follow complex instructions less reliably, and write less polished prose. They are still genuinely useful for everyday questions, summaries, code snippets, and drafts — and they are improving fast. For the best local quality, use the biggest model your hardware supports.
Detect your GPU with one click and see which LLMs your computer can actually run — ranked by whether they fit in your VRAM, need CPU offloading, or will not run at all.
Speed test your GPU for AI — measure real memory bandwidth and compute with WebGPU, run an actual LLM in your browser, and see predicted speeds for every popular model on your hardware.
Summarize any text with AI that runs entirely in your browser. Paste an article, contract, or report and get bullet points or a TL;DR — your text is never uploaded.
Build Ollama commands without memorizing syntax: run, pull, and create commands for any model, complete Modelfiles, and server environment configuration — with built-in VRAM checks.