Calculate the VRAM needed to fine-tune an LLM - full, LoRA or QLoRA - counting gradients, optimizer states and activations. Compare methods, check GPU fit.
Training memory and inference memory are not the same problem, and the gap between them is the reason so many fine-tuning attempts die at an out-of-memory error on hardware that runs the same model happily for chat. Running a model requires holding the weights. Training it requires holding the weights, a gradient for every trainable parameter, two optimiser moments for every trainable parameter, and every intermediate activation the backward pass will need. This calculator breaks that down for full fine-tuning, LoRA and QLoRA, and tells you which GPUs the total fits on.
| Component | Full fine-tune | LoRA | QLoRA |
|---|---|---|---|
| Base weights | 2 bytes/param (bf16) | 2 bytes/param, frozen | ~0.55 bytes/param (4-bit NF4) plus dequantisation buffers |
| Gradients | 2 bytes/param | Adapters only | Adapters only |
| Optimiser state | 8 bytes/param (AdamW fp32), or 2 with 8-bit AdamW | Adapters only | Adapters only |
| Activations | Scales with batch size × sequence length × hidden size × layers; the same for all three methods | ||
| Overhead | 9% of the subtotal, with a 2.5 GiB floor, for CUDA context, temporary buffers and fragmentation | ||
The 12 bytes per parameter that full fine-tuning demands — two for weights, two for gradients, eight for the optimiser — is where the pain comes from. Multiply by anything above a few billion parameters and you are past every consumer GPU before activations are even considered.
Batch size 1, sequence length 2,048, gradient checkpointing on, LoRA rank 16:
| Method | Weights | Gradients | Optimiser | Activations | Total | Trainable params |
|---|---|---|---|---|---|---|
| Full fine-tune | 14.96 GiB | 14.96 GiB | 59.83 GiB | 2.25 GiB | ~100.3 GiB | 8.0 B |
| LoRA | 14.96 GiB | 0.03 GiB | 0.13 GiB | 2.25 GiB | ~19.9 GiB | 16.8 M |
| QLoRA | 4.49 GiB | 0.03 GiB | 0.13 GiB | 2.25 GiB | ~9.4 GiB | 16.8 M |
A little over 100 GiB versus a little over 9 GiB, for the same model and the same data. Full fine-tuning an 8B needs a multi-GPU datacentre node; QLoRA fits on a 12 GB consumer card. The gap is almost entirely optimiser state and gradients — the AdamW moments alone are 59.83 GiB, six times the size of the model in 4-bit form.
Notice also that LoRA does not reduce activation memory at all. It is 2.25 GiB in all three columns, because the forward pass still runs through every layer of the full network regardless of which parameters are trainable. That is why batch size and sequence length hurt LoRA runs just as much as full fine-tunes.
LoRA freezes the base model and inserts small low-rank matrices on the attention projections. The calculator models adapters on the q, k, v and o projections in every layer, each decomposed into an A and a B matrix, which for an 8B model at rank 16 comes to roughly 16.8 million trainable parameters — about 0.2% of the model. Gradients and optimiser state are then charged against that 0.2% rather than the whole network, which is how 74 GiB of gradient and optimiser memory collapses to under 0.2 GiB.
QLoRA adds the second lever: it stores the frozen base in 4-bit NF4 rather than bf16, cutting weight memory by roughly 72% and adding a small dequantisation buffer, since the 4-bit weights must be expanded on the fly during the forward pass. Raising the rank increases capacity and memory together, though gently — going from rank 16 to rank 64 on this model takes trainable parameters from 16.8 M to 67.1 M and the total from ~9.4 to ~10.0 GiB. Rank is rarely what breaks a training run.
| Model | Full fine-tune | LoRA | QLoRA |
|---|---|---|---|
| Llama 3.2 3B | ~40.8 GiB | ~10.1 GiB | ~5.9 GiB |
| Llama 3.1 8B | ~100.3 GiB | ~19.9 GiB | ~9.4 GiB |
| Qwen3 14B | ~183.7 GiB | ~34.1 GiB | ~14.6 GiB |
| Mistral Small 3.2 24B | ~296.3 GiB | ~52.9 GiB | ~19.7 GiB |
| Qwen3 32B | ~405.2 GiB | ~73.2 GiB | ~26.9 GiB |
| Llama 3.3 70B | ~871.7 GiB | ~156.5 GiB | ~56.3 GiB |
Two practical readings. QLoRA on a 32B model needs about 27 GiB — a single 32 GB card, or a large-memory Apple Silicon machine. And QLoRA on a 70B needs about 56 GiB, which is one 80 GB datacentre GPU rather than the eight-GPU node full fine-tuning would demand. The step from "impossible without a cluster" to "one rented card for an afternoon" happens entirely in this table.
The memory numbers usually make the decision for you, but it helps to know what you are giving up:
One consequence of the adapter approach that does not appear in the memory table: a LoRA or QLoRA run produces a small adapter file, often tens of megabytes, rather than a full model copy. Several adapters can be kept for one base model and swapped, which matters if you are training variants for different tasks.
The model picker offers a curated set of popular open weights, a live Hugging Face search that reads real architecture — layer count, hidden size, KV heads — from the repository's configuration, and a custom mode where you enter a parameter count directly if the model is unreleased or the repository does not publish its configuration. Since the training-memory arithmetic depends mostly on parameter count, layer count and hidden size, a custom entry with a plausible architecture gets you a usable estimate for anything. The full configuration — model, method, rank — is encoded in the URL, so a link reproduces the same breakdown for a colleague.
Below the breakdown, every card in the hardware dataset is checked against your total, grouped by vendor and class. Each is marked as fitting on one card, needing a specific number of cards for a multi-GPU split, or unable to help at all. Unified-memory machines — Apple Silicon, Strix Halo — are budgeted at 75% of total memory, because the operating system needs the rest, and they are marked as single-device only since there is no way to combine them. Multi-GPU counts include a 5% per-card duplication allowance for sharded training state, so the number is a little higher than a naive division.
Once training is finished you have a different memory question — what the merged or adapter-loaded model needs at inference time, which is a much smaller number and depends on quantisation and context length rather than optimiser state. The tool links straight through to the inference-side calculator with your model already selected.
Fine-tuning memory has four components, and which one dominates depends on the method:
Weights: the base model itself. Full fine-tuning and LoRA keep it in bf16 (2 bytes/param); QLoRA quantizes it to 4-bit NF4 (~0.55 bytes/param).
Gradients: needed for every trainable parameter. Full fine-tuning trains everything (gradients = weight size). LoRA/QLoRA only train tiny adapters, so gradients are negligible.
Optimizer states: AdamW stores two fp32 moments per trainable parameter — 8 bytes each. This is what makes full fine-tuning so expensive: an 8B model needs 64 GB of optimizer states alone. 8-bit optimizers cut this 4x.
Activations: intermediate values from the forward pass, scaling with batch size x sequence length x model depth. Gradient checkpointing cuts these ~85%.
The punchline: for full fine-tuning, optimizer states dominate. For LoRA/QLoRA, the frozen base model dominates — which is why quantizing it (QLoRA) is such a big win.
What you can realistically train on common hardware (batch 1, 2K sequence, gradient checkpointing):
12-16 GB (RTX 3060/4060 Ti): QLoRA up to 8-9B models. This is the entry point — Llama 3.1 8B, Qwen3 8B, Gemma 4 E4B all work.
24 GB (RTX 3090/4090): QLoRA up to ~14B comfortably, LoRA up to 8B, or QLoRA on 27-32B models with short sequences.
48 GB (2x 3090, RTX 6000 Ada, 64 GB Mac): QLoRA on 70B models — the sweet spot for serious open-model fine-tuning.
80 GB+ (A100/H100): LoRA on 70B, full fine-tuning of 7-8B models, QLoRA on the largest MoE models.
Cloud alternative: renting an A100 80GB at ~1.40/hr means a typical QLoRA run (3-12 hours) costs 5-20 dollars — often cheaper than upgrading hardware for occasional training.
Inference only needs the model weights and KV cache. Training adds three big consumers: gradients (same size as the weights), optimizer states (AdamW keeps two momentum values per parameter in fp32 — 8 bytes per parameter, 4x the bf16 weights), and activations (intermediate values saved during the forward pass for backpropagation, which scale with batch size and sequence length). Full fine-tuning of an 8B model needs ~100+ GB; running it needs ~6 GB.
LoRA freezes the base model in bf16 and trains small adapter matrices instead — you only need gradients and optimizer states for the adapters (typically under 1% of parameters). QLoRA goes further: the frozen base model is quantized to 4-bit (NF4), cutting its memory by ~73%. QLoRA is what makes fine-tuning a 70B model possible on a single 48-64 GB GPU. Quality difference between the two is usually negligible.
With QLoRA: about 10-14 GB (fits an RTX 3060 12GB or any 16 GB card). With LoRA: about 20-24 GB (RTX 3090/4090). With full fine-tuning: 100+ GB (multiple A100s or H100s). These numbers assume batch size 1, 2K sequence length, and gradient checkpointing enabled — the calculator lets you adjust all of these.
Gradient checkpointing trades compute for memory: instead of storing all activations from the forward pass, it stores a fraction and recomputes the rest during backpropagation. It cuts activation memory by ~85% at the cost of ~30% slower training. For consumer GPUs the answer is almost always yes — it is the difference between fitting and not fitting.
Rank controls adapter capacity: r=8-16 works for style/format adaptation and most chat fine-tunes. r=32-64 for teaching substantial new knowledge or behaviors. r=128+ approaches full fine-tuning quality but with diminishing returns. Higher ranks need more memory (linearly), but adapter memory is small compared to the base model, so rank rarely determines whether a job fits.
Larger batches give more stable gradients but need proportionally more activation memory. The standard workaround is gradient accumulation: run multiple batch-size-1 steps and accumulate gradients before updating — same effective batch size, fraction of the memory. If memory is tight, use batch size 1 with accumulation steps of 8-32.
Yes, with MLX (Apple's ML framework) — MLX-LM supports LoRA and QLoRA fine-tuning with similar memory characteristics. A 64 GB Mac can QLoRA-tune models up to ~30B. Training is slower than on NVIDIA GPUs (less compute), but for small datasets and adapters it is entirely practical.
Calculate how much VRAM any LLM needs to run locally. Pick a model, quantization, and context size — see download size, total memory required, and which GPUs it fits on.
Detect your GPU with one click and see which LLMs your computer can actually run — ranked by whether they fit in your VRAM, need CPU offloading, or will not run at all.
Estimate LLM tokens per second from memory bandwidth, model size, quantization, and context window. Compare generation speed across GPUs and understand the memory-bandwidth bottleneck.