How to Fine-Tune an LLM: Methods, Data, Evaluation and Best Practices
A model-agnostic LLM fine-tuning guide: choosing full tuning, LoRA, QLoRA or DPO, curating training data, beating a prompt baseline and avoiding forgetting.
Knowing how to fine-tune an LLM means knowing three things: when fine-tuning is the right tool, which method fits your budget and goal, and how to prove the tuned model is actually better than the base model with a good prompt. Fine-tuning is the post-training adaptation of a pretrained large language model to a specific task, domain, output format or policy, using a comparatively small set of curated examples.
Fine-tuning rarely needs huge datasets or clusters, but it does need discipline. Most failed projects fail on unclear objectives, noisy data, leaked test sets or an evaluation that never compared the result with a cheaper alternative.
This model-agnostic guide covers the main fine-tuning methods, a step-by-step workflow, data preparation, evaluation, common mistakes and the state of fine-tuning as of November 2026.
What Is LLM Fine-Tuning?
LLM fine-tuning continues training a pretrained model on a task-specific dataset so that its behavior shifts toward a target distribution. In the simplest case, supervised fine-tuning, the model learns from input and target pairs by minimizing token-level cross-entropy, the same next-token objective used in pretraining, applied to your examples.
In practice, "fine-tuning" is usually a stack of adaptations, each fixing a different failure mode:
- Supervised fine-tuning (SFT) on instruction and response pairs teaches adherence, structure, tone and task specificity.
- Preference tuning, such as reinforcement learning from human feedback (RLHF) or direct preference optimization (DPO), pushes the model toward responses people prefer.
- Retrieval-augmented generation (RAG) is not fine-tuning at all, because it does not change weights, but it is often paired with it to ground answers in current or proprietary content.
The key mental model: fine-tuning changes how a model behaves far more reliably than what it knows. The LIMA study from Meta and collaborators found that a large model tuned on only 1,000 carefully curated examples performed strongly, and its authors concluded that almost all of a model's knowledge comes from pretraining (Zhou et al., 2023). Fine-tuning mostly teaches the model which of its existing skills to use and how to present them.
When to Fine-Tune an LLM (and When Not To)
The first best practice is to delay fine-tuning until simpler options are exhausted. Google's Vertex AI documentation recommends starting with prompting and tuning only if needed (Google Cloud). OpenAI's model optimization guide describes the same loop: write evals, set a baseline with prompting, fine-tune only if it helps, then re-run the evals (OpenAI).
Fine-tuning pays off when you need consistent formats or a house style prompts cannot hold, more examples than fit in one context window, shorter prompts that cut latency and cost, or a smaller model that matches a larger one on a narrow task. It is a poor fit when the real problem is missing or fast-changing knowledge. In a direct comparison, researchers found RAG consistently outperformed unsupervised fine-tuning for injecting both existing and new facts (Ovadia et al., 2023).
| Approach | What it changes | Best for | Main cost |
|---|---|---|---|
| Prompt engineering | Instructions and examples at request time | Fast iteration, broad tasks | Longer prompts, less consistency |
| RAG | Context retrieved at request time | Fresh, proprietary or citable facts | Retrieval pipeline and index upkeep |
| Fine-tuning | Model weights or adapters | Format, tone, narrow skills, smaller models | Data curation, training and re-evaluation |
| Fine-tuning plus RAG | Behavior and context together | Domain assistants that must cite sources | Both pipelines to maintain |
A common production pattern is "fine-tune for behavior, retrieve for facts." Deciding where that line sits for a specific product is an architecture question as much as a training one, and it is the kind of tradeoff an LLM solutions engagement should settle before any GPU time is spent.
LLM Fine-Tuning Methods
Methods range from updating every weight to training a tiny set of added parameters, trading capacity against compute, memory and the ability to roll back.
Full Fine-Tuning
Full fine-tuning updates every parameter. It offers the most capacity for deep domain shift or large datasets, but it needs memory for weights, gradients and optimizer states, typically with mixed precision, sharding and gradient checkpointing. It also produces a full model copy per variant and carries the highest forgetting risk. An empirical study of continual instruction tuning observed forgetting across models from 1B to 7B parameters (Luo et al.). Reserve full fine-tuning for cases where evidence shows parameter-efficient methods cannot meet the bar.
LoRA (Low-Rank Adaptation)
LoRA freezes the pretrained weights and trains small low-rank matrices added to each layer. The original paper reported that, compared with full fine-tuning of GPT-3 175B with Adam, LoRA cut trainable parameters by roughly 10,000 times and GPU memory by about three times, with no added inference latency once the adapters are merged (Hu et al., 2021). Adapters are small files, so one base model can serve many variants.
QLoRA
QLoRA quantizes the frozen base model to 4-bit NormalFloat (NF4), adds double quantization and paged optimizers, and trains LoRA adapters on top. Its authors fine-tuned a 65B-parameter model on a single 48 GB GPU while preserving 16-bit task performance (Dettmers et al., 2023). QLoRA is the usual starting point when hardware is limited, though you should verify that quantization error does not hurt your specific task.
Instruction Tuning
Instruction tuning is supervised fine-tuning on instruction and response pairs. Dataset composition dominates: task coverage, domain vocabulary, consistent prompts and examples of acceptable refusals.
Preference Tuning and Reinforcement Fine-Tuning
Preference tuning trains on pairs of better and worse responses. DPO reframes the RLHF objective so it can be solved with a simple classification loss, avoiding a separate reward model and reinforcement learning loop (Rafailov et al., 2023). Reinforcement fine-tuning goes further, grading generated responses and reinforcing higher-scoring reasoning, and it suits verifiable tasks such as code or SQL generation.
How to Fine-Tune an LLM Step by Step
The workflow below applies whether you train an open-weight model yourself or use a managed tuning service.
- Define the objective and constraints. Write the task, the success metric, safety rules and deployment limits for latency, context window and cost.
- Build an evaluation set first. Hold out representative, realistic examples and score the base model with your best prompt. This is the baseline the tuned model must beat.
- Choose the base model. Prefer the smallest model that can plausibly reach the bar, check its license, and use the same tokenizer and chat template it was trained with.
- Prepare the training data. Curate, label, deduplicate and format examples, and keep them strictly separated from the evaluation set.
- Pick the method. Start with LoRA or QLoRA; move to full fine-tuning only if adapters plateau below target.
- Train with small ablations. Run short experiments on learning rate, rank and epochs, and watch training and validation loss together.
- Evaluate, compare and iterate. Score against the baseline, inspect failures, fix the data and repeat before deployment.
For teams training open-weight models, the Hugging Face PEFT library provides reference implementations of LoRA and related methods (Hugging Face PEFT), and TRL provides trainers for SFT, DPO, reward modeling and online reinforcement methods such as GRPO (Hugging Face TRL). Turning that workflow into a repeatable pipeline with versioned data, tracked experiments and gated releases is ordinary machine learning development work, not a one-off notebook.
Preparing Fine-Tuning Data
Data is the dominant factor in fine-tuned model quality, and quality beats quantity. Google's tuning documentation suggests hundreds to thousands of labeled examples for supervised tuning and stresses that well-labeled data matters more than volume (Google Cloud).
Design for Coverage Before Size
Start from failure analysis or a task inventory. List the intents, entities and edge cases the model must handle, including inputs it should refuse or escalate. When mixing tasks, document the ratios and evaluate each task separately.
Make Labels Consistent
Write annotation guidelines with inclusion and exclusion rules and borderline examples. Measure inter-annotator agreement, adjudicate disagreements and update the guide. Inconsistent labels teach the model inconsistency.
Validate Before Training
- Check schema, encoding and formatting against the model's chat template.
- Remove exact and near duplicates to reduce memorization and inflated metrics.
- Split by document, customer or time so test examples never leak into training.
- Audit class balance and probe for spurious correlations.
- Redact or tokenize personal data and record provenance and consent.
Synthetic data can fill gaps or balance classes, but review samples by hand and check the source model's terms of use.
Hyperparameter Starting Points
There are no universal settings, only defensible starting points to validate with small ablations. For supervised fine-tuning, one to three epochs is a common range; more epochs raise memorization risk, so early-stop on validation loss or the task metric. Set the maximum sequence length to cover your real inputs, because silent truncation erases the evidence the model needs.
Research on LoRA has refined older advice. A 2025 study from Thinking Machines Lab found LoRA performed about the same as full fine-tuning on small and medium instruction and reasoning datasets, worked best when applied to all layers including the MLP layers rather than attention only, and had an optimal learning rate about ten times higher than full fine-tuning (Schulman et al., 2025). It also found LoRA less tolerant of very large batches and less efficient once data exceeds adapter capacity. Common LoRA ranks fall between 8 and 64, with the scaling factor alpha often set equal to or double the rank.
How to Evaluate a Fine-Tuned Model
Evaluation should cover capability, reliability and safety, and it must mirror deployment: a model strong at generic questions can still fail policy-constrained answers.
- Task metrics: accuracy or F1 for classification, exact match for extraction, pass rates for code, and rubric or pairwise human preference for open-ended generation.
- Baseline comparison: always compare against the base model with your best prompt, ideally in a blind side-by-side review.
- Regression on general skills: keep a small general benchmark to catch catastrophic forgetting.
- Robustness: test phrasing variations, ambiguous instructions and out-of-domain inputs.
- Contamination checks: confirm test examples never appeared in training data.
- Safety: check refusals, harmful outputs and whether the model escalates when it should.
LLM-as-judge scoring is useful for iteration, but calibrate it against human ratings before trusting it for releases.
LLM Fine-Tuning Mistakes to Avoid
- Fine-tuning to teach facts that change, instead of retrieving them.
- Skipping the prompt-only baseline, so no one knows whether tuning helped.
- Training and testing on overlapping data.
- Using a different chat template or tokenizer than the base model expects.
- Over-training a small dataset until the model memorizes it.
- Forgetting that every base-model upgrade means re-tuning and re-evaluating.
The State of LLM Fine-Tuning in 2026
Two trends define fine-tuning as of November 2026. First, frontier API providers are narrowing self-serve tuning. OpenAI's model optimization guide states that it is winding down its fine-tuning platform, which is no longer available to new users, while existing fine-tuned models remain available for inference until their base models are deprecated (OpenAI). Google's Vertex AI still offers supervised tuning, preference tuning and reinforcement learning fine-tuning for selected Gemini models (Google Cloud).
Second, open-weight fine-tuning keeps getting easier: parameter-efficient methods and maturing open-source trainers make it practical to tune capable models on modest hardware and keep training data in your own environment. If a fine-tuned model is part of your product, plan for portability, because the provider or base model will likely change.
Frequently Asked Questions
How much data do you need to fine-tune an LLM?
Often less than expected: many tasks need hundreds to low thousands of high-quality examples. Coverage and label consistency matter more than raw volume.
Is fine-tuning better than RAG?
They solve different problems. Fine-tuning changes behavior, format and tone; RAG supplies current or proprietary facts at request time. For knowledge that changes, RAG usually wins, and many production systems combine both.
What is the difference between LoRA and QLoRA?
LoRA freezes the base model and trains small low-rank adapters. QLoRA does the same on top of a base model quantized to 4-bit, which cuts memory further so larger models fit on a single GPU.
Can you fine-tune an LLM on a single GPU?
Yes, for many model sizes. Parameter-efficient methods such as LoRA and QLoRA were designed to make single-GPU fine-tuning practical. Full fine-tuning of large models still needs multiple GPUs.
How long does it take to fine-tune an LLM?
Adapter training on a modest dataset can take minutes to hours. Most calendar time goes into defining the task, curating data and evaluating results.
Does fine-tuning cause catastrophic forgetting?
It can, especially with full fine-tuning and many epochs. Adapters, fewer epochs, mixed general data and a regression benchmark help limit it.
Conclusion: How to Fine-Tune an LLM Well
Fine-tuning an LLM well is about order of operations: define the objective, build the evaluation set, beat a prompt-only baseline, start with LoRA or QLoRA, invest in clean data and keep retrieval for facts that change.
Base models and provider offers will keep changing; a workflow built on solid data and honest evaluation survives those changes. If you are weighing whether to fine-tune, retrieve or both for your product, talk to Cognativ about your LLM project.