QLoRA: 4-Bit Weight Quantization with 16-Bit Fidelity
Fine-tune 7B, 14B, and 70B parameter open-source models on consumer workstations. QLoRA compresses base weights to 4-bit NormalFloat while preserving full gradient expressiveness through low-rank adapter matrices.
NormalFloat 4 (NF4)
Information-theoretically optimal quantile quantization designed for normally distributed neural network weights. Eliminates empirical quantization error inherent in standard 4-bit integers.
Double Quantization
Quantizes the quantization constants themselves, saving ~0.37 bits per parameter. On a 65B/70B model, Double Quantization frees over 3 GB of VRAM without perceptible quality loss.
Paged Optimizers
Leverages unified memory paging to page gradient and optimizer spikes to CPU host RAM during long-sequence activation peaks, preventing sudden out-of-memory (OOM) crashes.
Hardware VRAM Requirements: FP16 vs. QLoRA
Real-world accelerator memory consumption comparing uncompressed weights vs. 4-bit QLoRA adaptation.
| Model Size | Full Tuning (FP32) | Standard LoRA (16-bit) | QLoRA (4-bit NF4) | Minimum Recommended Hardware |
|---|---|---|---|---|
| 3B (e.g. Qwen2.5-3B) | ~24 GB | ~8.5 GB | ~3.2 GB | M1/M2/M3 Mac (8GB+) or RTX 3060 (6GB) |
| 7B / 8B (e.g. Llama-3.1-8B) | ~64 GB | ~18.0 GB | ~6.8 GB | Apple Silicon 16GB or RTX 4070 (12GB) |
| 14B (e.g. Qwen2.5-14B) | ~120 GB | ~34.0 GB | ~12.4 GB | Apple Silicon 24GB+ or RTX 3090/4090 (24GB) |
| 70B (e.g. Llama-3-70B) | ~500+ GB | ~160.0 GB | ~48.5 GB | Mac Studio M2/M3 Ultra (64GB-128GB) |
Ready to train your first QLoRA adapter?
FineTuneMyAI includes automated memory headroom verification to guarantee zero OOM crashes.