What is fine-tuning?

How fine-tuning works

Fine-tuning starts from a foundation model that has already learned general language or vision patterns from massive pre-training. Engineers assemble a labeled dataset of input-output examples that demonstrate the target task, then run additional training passes that nudge the model's weights toward those examples using gradient descent, usually at a much lower learning rate than pre-training to avoid destroying existing knowledge.

Because the model already understands language, fine-tuning needs comparatively little data. Google notes that engineers can sometimes fine-tune a foundation LLM on just a few hundred or a few thousand examples, though it cautions that standard fine-tuning is often computationally expensive. The output is a new set of weights specialized for the task, which can then be deployed like any other model.

Why fine-tuning matters

Fine-tuning unlocks a model's practical side. A raw foundation model is a general next-token predictor; fine-tuning teaches it to follow instructions, adopt a consistent voice, output structured formats like JSON, or perform a narrow domain task such as medical coding or legal summarization more reliably than prompting alone.

It also bakes behavior into the weights, so the model performs the task without long, expensive prompts at inference time. This makes fine-tuned models cheaper to run per query and more consistent than steering a general model with elaborate prompts, which is why it is favored for high-volume, well-defined tasks where behavior must be dependable.

Full fine-tuning vs parameter-efficient fine-tuning

Full fine-tuning updates every weight in the model. It is the most flexible approach but expensive: it requires enough GPU memory to hold and update all parameters, and it produces a full-size copy of the model for every task. For models with billions of parameters this quickly becomes impractical for most teams.

Parameter-efficient fine-tuning (PEFT) solves this by freezing the original weights and training only a small number of new or selected parameters. Hugging Face describes PEFT as fine-tuning a small number of extra parameters while yielding performance comparable to a fully fine-tuned model, making it possible to adapt and store large models on consumer hardware. LoRA is the best-known PEFT method.

LoRA and low-rank adaptation

LoRA, or Low-Rank Adaptation, was introduced by Hu et al. in 2021. Instead of updating a layer's large weight matrix, LoRA freezes it and injects two small trainable rank-decomposition matrices whose product approximates the needed change. Only those small matrices are trained, and they can be merged back into the weights at deployment so there is no extra inference latency.

The efficiency gains are large. For GPT-3 with 175 billion parameters, the LoRA paper reports it can reduce the number of trainable parameters by up to 10,000 times and cut GPU memory requirements by roughly 3 times, while performing on par with or better than full fine-tuning. Because each LoRA adapter is tiny, teams can maintain many task-specific adapters over one shared base model.

Instruction tuning and RLHF

Two specialized forms of fine-tuning turned raw language models into usable assistants. Instruction tuning (supervised fine-tuning) trains a model on curated instruction-and-response pairs so it learns to follow directions rather than merely continue text. Reinforcement learning from human feedback (RLHF) goes further by training a reward model on human rankings of outputs, then optimizing the language model against that reward.

The InstructGPT paper by Ouyang et al. (2022), the method behind ChatGPT, combined supervised fine-tuning with RLHF and reported a striking result: labelers preferred outputs from the 1.3-billion-parameter InstructGPT model over the 175-billion-parameter GPT-3, despite it having 100 times fewer parameters. This showed that alignment fine-tuning can matter more than raw scale for perceived quality.

Limitations and fine-tuning vs RAG

Fine-tuning has real costs. It requires a curated dataset, and poor data teaches poor behavior. Full fine-tuning can be computationally expensive and risks catastrophic forgetting, where the model loses general abilities while learning the new task. Crucially, fine-tuning is a poor way to inject large bodies of factual knowledge: facts learned this way are not citable, are hard to update, and can still be hallucinated.

That is where retrieval-augmented generation (RAG) comes in. The practical division is that fine-tuning changes how a model behaves while RAG changes what it knows at query time by retrieving source-backed facts. For knowledge that changes often or must be cited, RAG is usually the right tool; for durable skills, tone, and formats, fine-tuning wins. Production systems frequently combine both. See our companion explainer on RAG for the retrieval side.

Frequently asked questions

Fine-tuning vs RAG, which should I use?

Use fine-tuning to change how a model behaves, its tone, format, or task skill, and use RAG to change what it knows by retrieving facts at query time. Fine-tuning is a poor way to store large, changing, or citable knowledge; RAG handles that better. Many production systems combine both.

What is LoRA in fine-tuning?

LoRA (Low-Rank Adaptation) is a parameter-efficient method that freezes the model's original weights and trains small injected rank-decomposition matrices instead. The 2021 LoRA paper reports up to 10,000x fewer trainable parameters and about 3x less GPU memory for GPT-3 175B, with performance on par with full fine-tuning and no added inference latency.

What is RLHF?

Reinforcement learning from human feedback (RLHF) fine-tunes a model against a reward model trained on human rankings of outputs, teaching it to produce helpful, preferred responses. It is the alignment method behind InstructGPT and ChatGPT; Ouyang et al. (2022) found labelers preferred the 1.3B InstructGPT over 175B GPT-3.

How much data do you need to fine-tune a model?

Far less than pre-training, because the model already understands language. Google notes that a foundation LLM can sometimes be fine-tuned on just a few hundred or a few thousand examples. Data quality matters more than quantity: a small, clean, representative dataset usually beats a large noisy one.

Does fine-tuning add new knowledge to a model?

It can shift a model toward domain patterns, but it is an unreliable and expensive way to inject specific facts. Facts learned through fine-tuning cannot be cited, are hard to update, and may still be hallucinated. For current, proprietary, or citable knowledge, retrieval-augmented generation (RAG) is the better approach.

What is catastrophic forgetting?

Catastrophic forgetting is when a model loses previously learned general abilities while being fine-tuned on a new task, because the new training overwrites useful weights. Lower learning rates, mixing in general data, and parameter-efficient methods like LoRA that freeze most weights all help reduce it.

Related

Explainers: LLM, RAG

Glossary: Transfer learning, LoRA (low-rank adaptation), RLHF (reinforcement learning from human feedback), Large language model (LLM), PEFT (parameter-efficient fine-tuning)

Sources

• Google for Developers — Machine Learning Glossary: Generative AI (fine-tuning)
• arXiv — LoRA: Low-Rank Adaptation of Large Language Models (Hu et al., 2021)
• arXiv — Training language models to follow instructions with human feedback (Ouyang et al., 2022)
• Hugging Face — PEFT (Parameter-Efficient Fine-Tuning) documentation