Keep up with large language models research. Without the noise.

2023 · 4,351 cites · Fine-Tuning, Efficiency, Training Techniques

We present QLoRA, an efficient finetuning approach that reduces memory usage enough to finetune a 65B parameter model on a single 48GB GPU while preserving full 16-bit finetuning task performance. QLoRA backpropagates gr…