Split View: LLM 파인튜닝 실전 가이드: LoRA·QLoRA·PEFT로 구현하는 효율적 도메인 적응

Practical Guide to LLM Fine-Tuning: Efficient Domain Adaptation with LoRA, QLoRA, and PEFT

• Introduction
• The Evolving Paradigm of Fine-Tuning
  • Full Fine-Tuning
  • Feature Extraction
  • Parameter-Efficient Fine-Tuning (PEFT)
• Deep Dive into LoRA
  • Mathematical Principles of Low-Rank Decomposition
  • LoRA Initialization Strategy
  • Scaling Factor alpha
  • Inference-Time Merging
• QLoRA Architecture
  • 4-bit NormalFloat (NF4)
  • Double Quantization
  • Paged Optimizers
  • Memory Savings with QLoRA
• Practical Use of the PEFT Library
  • Environment Setup
  • LoRA Configuration and Training
  • Saving and Merging Adapters
• Dataset Preparation and Preprocessing Strategies
  • Data Format: Instruction Tuning Format
  • Data Quality Checklist
• Hyperparameter Tuning Guide
  • Rank (r)
  • Alpha
  • Target Modules
  • Learning Rate
• Comparative Analysis
  • LoRA vs QLoRA vs Full Fine-Tuning
  • Latest Research Findings on Performance
• Operational Considerations
  • Catastrophic Forgetting
  • Overfitting
  • Evaluation Metrics
• Failure Case Studies and Recovery Procedures
  • Case 1: CUDA OOM (Out of Memory)
  • Case 2: Loss Does Not Converge
  • Case 3: Repetitive Output After Training
  • Case 4: Shape Mismatch When Loading Adapter
• Production Checklist
• References
• Conclusion

Introduction

While large language models (LLMs) such as GPT-4, Llama 3, and Mistral have achieved impressive performance on general-purpose tasks, fine-tuning remains essential for optimizing them on domain-specific or proprietary enterprise data. However, full fine-tuning of models with billions of parameters demands enormous GPU memory and training time.

Parameter-Efficient Fine-Tuning (PEFT) techniques were developed to address this challenge. Among them, LoRA (Low-Rank Adaptation) and QLoRA have made it possible to fine-tune models with 70B or more parameters on a single GPU, training only 0.1-1% of the total parameters while achieving performance close to full fine-tuning.

This article covers the entire fine-tuning workflow: from the mathematical principles of LoRA to QLoRA quantization techniques, practical use of the Hugging Face PEFT library, dataset preparation, hyperparameter tuning, comparative analysis, operational considerations, failure recovery, and a production checklist.

The Evolving Paradigm of Fine-Tuning

LLM fine-tuning can be broadly categorized into three paradigms.

Full Fine-Tuning

This is the traditional approach of updating all model parameters. While it can achieve the highest performance, a 7B model alone requires approximately 56GB of GPU memory (FP16 + AdamW optimizer), and a 70B model demands hundreds of gigabytes.

Feature Extraction

This approach freezes the pre-trained model and trains only the top classification layer. It is fast and inexpensive but fails to fully leverage the model's representational power.

Parameter-Efficient Fine-Tuning (PEFT)

This approach freezes most of the model's parameters and trains only a small number of additional parameters. LoRA, Prefix Tuning, and Adapter Layers fall into this category. It can reduce the number of trainable parameters by thousands of times while retaining 90-99% of full fine-tuning performance.

# Comparison of parameter counts: Full Fine-tuning vs PEFT
model_params = {
    "Llama-3-8B": {
        "total": 8_000_000_000,
        "full_ft_trainable": 8_000_000_000,
        "lora_r16_trainable": 20_971_520,   # ~0.26%
        "lora_r64_trainable": 83_886_080,   # ~1.05%
    },
    "Llama-3-70B": {
        "total": 70_000_000_000,
        "full_ft_trainable": 70_000_000_000,
        "lora_r16_trainable": 167_772_160,  # ~0.24%
        "lora_r64_trainable": 671_088_640,  # ~0.96%
    }
}

Deep Dive into LoRA

Mathematical Principles of Low-Rank Decomposition

LoRA (Low-Rank Adaptation) was proposed in a 2021 paper by Edward Hu et al. at Microsoft Research. The core idea is to approximate the update to a pre-trained weight matrix W as the product of low-rank matrices.

For a pre-trained weight matrix W (d x k dimensions), the update is decomposed as follows:

• W_new = W + delta_W = W + B x A
• Where B is a (d x r) matrix and A is an (r x k) matrix
• r is the rank, where r is much smaller than d and k (e.g., r=16 when d=4096, k=4096)

Directly learning delta_W would require d x k = 4096 x 4096 = 16,777,216 parameters, but LoRA decomposes it into B x A, requiring only (d x r) + (r x k) = 4096 x 16 + 16 x 4096 = 131,072 parameters. This is approximately 0.78% of the original.

LoRA Initialization Strategy

• Matrix A: Initialized with a normal distribution (Kaiming initialization)
• Matrix B: Initialized as a zero matrix
• At the start of training, delta_W = B x A = 0, so training begins from the same state as the original model

Scaling Factor alpha

In practice, a scaling factor alpha/r is multiplied to delta_W. alpha is a hyperparameter that controls the magnitude of the LoRA update together with the learning rate. It is typically set to alpha = 2 x r or alpha = r.

import torch
import torch.nn as nn

class LoRALayer(nn.Module):
    """Core implementation of a LoRA layer"""
    def __init__(self, in_features, out_features, rank=16, alpha=32):
        super().__init__()
        self.rank = rank
        self.alpha = alpha
        self.scaling = alpha / rank

        # Original weights (frozen)
        self.weight = nn.Parameter(
            torch.randn(out_features, in_features), requires_grad=False
        )

        # LoRA matrices
        self.lora_A = nn.Parameter(torch.randn(rank, in_features))
        self.lora_B = nn.Parameter(torch.zeros(out_features, rank))

        # Kaiming initialization
        nn.init.kaiming_uniform_(self.lora_A, a=5**0.5)

    def forward(self, x):
        # Original output + LoRA update
        base_output = x @ self.weight.T
        lora_output = (x @ self.lora_A.T @ self.lora_B.T) * self.scaling
        return base_output + lora_output

Inference-Time Merging

One of LoRA's major advantages is the ability to merge adapter weights into the original model at inference time. By merging as W_merged = W + (alpha/r) x B x A, you can serve the model with zero latency overhead using the exact same structure as the original model.

QLoRA Architecture

4-bit NormalFloat (NF4)

QLoRA was proposed in a 2023 paper by Tim Dettmers et al. The key idea is to perform LoRA training on a model that has been quantized to 4 bits.

NF4 (4-bit NormalFloat) is a quantization technique that leverages the fact that pre-trained neural network weights follow a normal distribution. It places more quantization levels near the center of the distribution and fewer at the tails, minimizing information loss.

Double Quantization

The quantization constants themselves are quantized a second time to further reduce memory overhead. With a block size of 64, this saves approximately 0.37 bits of memory per parameter.

Paged Optimizers

When GPU memory is insufficient, optimizer states are automatically paged to CPU memory to prevent OOM (Out-of-Memory) errors. This leverages NVIDIA's Unified Memory.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

# 4-bit quantization configuration for QLoRA
quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",           # NormalFloat4 quantization
    bnb_4bit_compute_dtype=torch.bfloat16, # Use bf16 for computation
    bnb_4bit_use_double_quant=True,       # Enable Double Quantization
)

model_name = "meta-llama/Llama-3.1-8B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    quantization_config=quantization_config,
    device_map="auto",
    torch_dtype=torch.bfloat16,
)

print(f"Model memory usage: {model.get_memory_footprint() / 1e9:.2f} GB")
# Full FP16: ~16GB -> QLoRA 4bit: ~5GB

Memory Savings with QLoRA

Practical Use of the PEFT Library

The Hugging Face PEFT (Parameter-Efficient Fine-Tuning) library provides a unified interface for various PEFT techniques including LoRA, QLoRA, Prefix Tuning, and Prompt Tuning.

Environment Setup

pip install peft transformers datasets accelerate bitsandbytes trl

LoRA Configuration and Training

from peft import LoraConfig, get_peft_model, TaskType, prepare_model_for_kbit_training
from transformers import TrainingArguments
from trl import SFTTrainer

# Preprocessing for 4-bit models
model = prepare_model_for_kbit_training(model)

# LoRA configuration
lora_config = LoraConfig(
    task_type=TaskType.CAUSAL_LM,
    r=16,                          # Rank
    lora_alpha=32,                 # Scaling factor
    lora_dropout=0.05,             # Dropout
    target_modules=[               # Modules to apply LoRA to
        "q_proj", "k_proj", "v_proj", "o_proj",  # Attention
        "gate_proj", "up_proj", "down_proj",       # MLP
    ],
    bias="none",
)

# Create PEFT model
peft_model = get_peft_model(model, lora_config)
peft_model.print_trainable_parameters()
# trainable params: 20,971,520 || all params: 8,030,261,248 || trainable%: 0.2612

# Training configuration
training_args = TrainingArguments(
    output_dir="./output",
    num_train_epochs=3,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=4,
    learning_rate=2e-4,
    warmup_ratio=0.03,
    lr_scheduler_type="cosine",
    logging_steps=10,
    save_strategy="steps",
    save_steps=100,
    bf16=True,
    optim="paged_adamw_8bit",      # QLoRA: Paged AdamW 8bit
    gradient_checkpointing=True,
    max_grad_norm=0.3,
)

# Run training with SFTTrainer
trainer = SFTTrainer(
    model=peft_model,
    args=training_args,
    train_dataset=train_dataset,
    tokenizer=tokenizer,
    max_seq_length=2048,
    dataset_text_field="text",
)

trainer.train()

Saving and Merging Adapters

# Save adapter only (a few MB in size)
peft_model.save_pretrained("./lora_adapter")

# At inference time: load adapter
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained(model_name)
inference_model = PeftModel.from_pretrained(base_model, "./lora_adapter")

# Merge adapter into base model (inference optimization)
merged_model = inference_model.merge_and_unload()
merged_model.save_pretrained("./merged_model")
tokenizer.save_pretrained("./merged_model")

Dataset Preparation and Preprocessing Strategies

Over 80% of fine-tuning success depends on data quality. No matter how good the technique is, poor data will yield poor results.

Data Format: Instruction Tuning Format

from datasets import load_dataset, Dataset

def format_instruction(sample):
    """Convert to Alpaca-style instruction format"""
    if sample.get("input"):
        text = (
            f"### Instruction:\n{sample['instruction']}\n\n"
            f"### Input:\n{sample['input']}\n\n"
            f"### Response:\n{sample['output']}"
        )
    else:
        text = (
            f"### Instruction:\n{sample['instruction']}\n\n"
            f"### Response:\n{sample['output']}"
        )
    return {"text": text}

# ChatML format (for modern models like Llama 3)
def format_chatml(sample):
    """Convert to ChatML conversation format"""
    messages = [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": sample["instruction"]},
        {"role": "assistant", "content": sample["output"]},
    ]
    text = tokenizer.apply_chat_template(messages, tokenize=False)
    return {"text": text}

# Load and preprocess dataset
dataset = load_dataset("json", data_files="train_data.jsonl", split="train")
dataset = dataset.map(format_chatml)
dataset = dataset.train_test_split(test_size=0.1)

Data Quality Checklist

1. At least 500-1,000 high-quality examples (quality over quantity)
2. Ensure uniform distribution across domains
3. Remove duplicate data (deduplicate)
4. Check input-output length distributions (remove extreme length discrepancies)
5. Verify label consistency (no contradictory answers for the same question)

Hyperparameter Tuning Guide

Rank (r)

This is the most important hyperparameter in LoRA. Higher rank captures more information but increases the number of trainable parameters.

• r=8: Simple domain adaptation, style transfer
• r=16: General instruction tuning (recommended default)
• r=32-64: Complex tasks, code generation, mathematical reasoning
• r=128+: When expressiveness close to full fine-tuning is needed

Alpha

Typically set to alpha = 2 x r. The alpha/r ratio determines the effective learning rate scaling.

Target Modules

Recent research shows that LoRA must be applied to MLP layers in addition to attention layers to reach full fine-tuning performance levels.

• Minimum: q_proj, v_proj (Query and Value of Attention only)
• Recommended: q_proj, k_proj, v_proj, o_proj (full Attention)
• Maximum: Attention + MLP (gate_proj, up_proj, down_proj)

Learning Rate

LoRA/QLoRA is effective with learning rates approximately 10x higher than full fine-tuning.

• Full Fine-tuning: 1e-5 to 5e-5
• LoRA/QLoRA: 1e-4 to 3e-4

Comparative Analysis

LoRA vs QLoRA vs Full Fine-Tuning

Latest Research Findings on Performance

According to the "LoRA vs Full Fine-tuning: An Illusion of Equivalence" study presented at NeurIPS 2025, LoRA and full fine-tuning access different solution spaces internally even when they achieve the same benchmark performance. For LoRA to match full fine-tuning, the following conditions are necessary:

1. Apply to all layers: LoRA must be applied to MLP layers, not just attention layers
2. Sufficient rank: An appropriate rank must be set for the task complexity
3. Higher learning rate: A learning rate approximately 10x higher than full fine-tuning should be used

Operational Considerations

Catastrophic Forgetting

This is the phenomenon where general knowledge learned during pre-training is forgotten during fine-tuning. LoRA/QLoRA mitigates this compared to full fine-tuning by freezing the original weights, but excessive training can still cause issues.

Mitigation strategies:

• Limit training epochs to 1-3
• Mix 5-10% of general-purpose data into the training data
• Monitor validation loss during training for early stopping

Overfitting

Special care is needed when fine-tuning on small datasets.

Mitigation strategies:

• Set lora_dropout=0.05-0.1
• Use gradient_checkpointing=True to save memory and increase batch size
• Validate regularly with an evaluation dataset

Evaluation Metrics

import evaluate
from transformers import pipeline

def evaluate_model(model, tokenizer, eval_dataset):
    """Evaluate fine-tuned model"""
    # Calculate Perplexity
    perplexity = evaluate.load("perplexity")

    # Task-specific evaluation
    results = {}

    # 1. Loss-based evaluation
    eval_results = trainer.evaluate()
    results["eval_loss"] = eval_results["eval_loss"]
    results["perplexity"] = 2 ** eval_results["eval_loss"]

    # 2. Generation quality evaluation (ROUGE, BLEU)
    rouge = evaluate.load("rouge")
    bleu = evaluate.load("bleu")

    predictions = []
    references = []

    pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
    for sample in eval_dataset:
        output = pipe(sample["input"], max_new_tokens=256)
        predictions.append(output[0]["generated_text"])
        references.append(sample["expected_output"])

    results["rouge"] = rouge.compute(
        predictions=predictions, references=references
    )
    results["bleu"] = bleu.compute(
        predictions=[p.split() for p in predictions],
        references=[[r.split()] for r in references],
    )

    return results

Failure Case Studies and Recovery Procedures

Case 1: CUDA OOM (Out of Memory)

Symptom: RuntimeError: CUDA out of memory error occurs

Recovery procedure:

1. Halve per_device_train_batch_size and double gradient_accumulation_steps
2. Verify gradient_checkpointing=True is set
3. Reduce max_seq_length (4096 -> 2048)
4. If still insufficient, switch to QLoRA with load_in_4bit=True
5. Last resort: reduce rank (r) or narrow target_modules

Case 2: Loss Does Not Converge

Symptom: Training loss oscillates or diverges

Recovery procedure:

1. Check the learning rate -- LoRA works best in the 1e-4 to 3e-4 range
2. Verify that warmup_ratio is set to 0.03-0.1
3. Check for dataset formatting errors (incorrect tokenization, missing special tokens)
4. Apply gradient clipping with max_grad_norm=0.3-1.0

Case 3: Repetitive Output After Training

Symptom: The model generates the same sentence in an infinite loop

Recovery procedure:

1. Reduce the number of training epochs (suspect overfitting)
2. Review training data for duplicate patterns
3. Set repetition_penalty=1.1-1.3 and temperature=0.7-0.9 at inference time
4. Increase the lora_dropout value (0.05 -> 0.1)

Case 4: Shape Mismatch When Loading Adapter

Symptom: RuntimeError: Error(s) in loading state_dict ... size mismatch

Recovery procedure:

1. Verify that the base model and adapter model versions match
2. Confirm that target_modules in adapter_config.json is compatible with the base model architecture
3. Specify the exact model version with the revision parameter

Production Checklist

These are items that must be verified before deploying a fine-tuned model to production.

Pre-training checks:

• Dataset quality validation complete (deduplication, format verification, label consistency)
• Base model license verified (commercial use eligibility)
• Evaluation dataset separated (no overlap with training data)
• GPU memory budget confirmed and QLoRA necessity determined

During training checks:

• Monitor training loss and validation loss with Wandb/TensorBoard
• Early stopping conditions configured
• Regular checkpoint saving enabled (save_steps configured)
• Gradient norm monitoring (early divergence detection)

Post-training checks:

• Measure performance on domain-specific evaluation datasets
• Verify general capability degradation on general benchmarks (MMLU, HellaSwag, etc.)
• Safety testing (check for harmful output generation)
• Decide between adapter merging vs. separate serving
• Verify compatibility with serving frameworks such as vLLM and TGI

Deployment checks:

• A/B test design (existing model vs. fine-tuned model)
• Rollback procedure documented
• Monitoring dashboard configured (response quality, latency, error rate)
• Model version management (adapter checkpoint + base model version mapping)

References

• LoRA: Low-Rank Adaptation of Large Language Models (Hu et al., 2021) -- arxiv.org/abs/2106.09685
• QLoRA: Efficient Finetuning of Quantized LLMs (Dettmers et al., 2023) -- arxiv.org/abs/2305.14314
• LoRA vs Full Fine-tuning: An Illusion of Equivalence (NeurIPS 2025) -- arxiv.org/abs/2410.21228
• Hugging Face PEFT Documentation -- huggingface.co/docs/peft
• LoRA+: Efficient Low Rank Adaptation of Large Models (Hayou et al., 2024) -- arxiv.org/abs/2402.12354
• Hugging Face TRL Library -- huggingface.co/docs/trl

Conclusion

LoRA and QLoRA are technologies that have dramatically lowered the barrier to entry for LLM fine-tuning. It is now possible to adapt models with billions of parameters to specific domains even on a single consumer GPU, and the PEFT library has significantly reduced implementation complexity.

The key lies not in the techniques themselves but in data quality and appropriate hyperparameter selection. In most cases, 500 high-quality training examples are more effective than 50,000 low-quality ones, and the choice of rank and target modules significantly impacts performance.

In production environments, the entire training-evaluation-deployment pipeline must be systematically managed. In particular, monitoring for catastrophic forgetting and overfitting, rollback procedures, and A/B testing are essential for stable service operation.

Follow-up research such as LoRA+, ALoRA, and DoRA continues to be published, and the combination of quantization and PEFT techniques will continue to evolve. While keeping pace with rapid technological change, establishing a data-centric approach and a culture of systematic evaluation first will form the foundation for successful fine-tuning.