A Map of the LLM Training Stack in 2026 — What Each Layer Does For You, and What It Hides

• Introduction — Choosing a Framework Means Choosing What to Hide
• The Reference — Versions and Check Date
• Layer 1 — The Distributed Execution Engine
• Layer 2 — Training Loop Frameworks
• Layer 3 — Config-Driven Post-Training Tools
• What a Vanished Lineage Tells You — torchtune and torchforge
• Fine-Tuning Paths and Alignment Stages — Today's Names
• The Moment a Higher-Level Framework Gets in Your Way
• Closing — Choose a Layer, But Don't Get Trapped In It

Introduction — Choosing a Framework Means Choosing What to Hide

There is no single answer to "which framework should I train with." That's because the layers are different. A loop you wrote yourself with torchrun and a fine-tune that runs off a single YAML file aren't competitors — one sits on top of the other.

So instead of listing frameworks, this post draws a lineage across three layers. For each layer we look at exactly three things: what this layer does for you, what it hides, and whether there's an escape hatch when you need to reopen what it hid.

This field changes on a monthly cadence. Every version below was checked directly against PyPI release history and GitHub release tags on August 2, 2026, and anything unverified is flagged as such in the text. Some of these numbers will already be out of date by the time you're reading this, so it's the layer structure, more than the numbers themselves, that's worth taking away for the long run.

The Reference — Versions and Check Date

Layer 1 — The Distributed Execution Engine

The bottom layer handles "which GPU does this tensor sit on, and when do we issue which collective operation." There are three lineages.

The PyTorch distributed family. torch.distributed provides DDP, FSDP2, DTensor, DeviceMesh, and torch.distributed.pipelining, all in one package. The defining shift as of 2026 is that parallelism is now a core PyTorch API rather than a separate library. FSDP2's fully_shard doesn't wrap the module — it turns the parameters into DTensors in place, so parameter names survive and checkpoint compatibility issues shrink. The official tutorial marks FSDP1 as deprecated.

The DeepSpeed family. ZeRO stages 1 through 3, offloading, sequence parallelism (Ulysses), and inference kernels all live in one package. A common question is "is this still alive" — 0.19.3 shipped on July 23, 2026, so yes, actively. Ownership has shifted: the GitHub org moved from microsoft to deepspeedai, and it's now an LF AI & Data incubation project and a PyTorch Foundation hosted project. If old links and org names are still sitting in a doc, that doc is stale.

The Megatron family. Megatron-Core is the reusable subset pulled out of NVIDIA's Megatron-LM repository, and it's the de facto reference implementation for tensor parallelism, pipeline parallelism, sequence parallelism, context parallelism, and expert parallelism. Transformer Engine, which handles low-precision training, attaches here. FP8 training has been supported for a long time, and NVFP4 4-bit pretraining is supported in Transformer Engine too — NVIDIA's paper reports training a 12B hybrid Mamba-Transformer to 10T tokens with results on par with an FP8 baseline. Whether that result transfers to your own model is a separate question, though — if you're considering 4-bit pretraining, run a short comparison run and check the loss curves directly.

The interface you touch when working with this layer directly is usually a config file. For DeepSpeed, it looks like this.

{
  "bf16": { "enabled": true },
  "zero_optimization": {
    "stage": 3,
    "overlap_comm": true,
    "contiguous_gradients": true,
    "reduce_bucket_size": 5e8,
    "stage3_prefetch_bucket_size": 5e8,
    "stage3_param_persistence_threshold": 1e6,
    "stage3_gather_16bit_weights_on_model_save": true
  },
  "gradient_accumulation_steps": "auto",
  "train_micro_batch_size_per_gpu": "auto",
  "gradient_clipping": 1.0,
  "steps_per_print": 50,
  "wall_clock_breakdown": false
}

The line that causes the most trouble in practice here is stage3_gather_16bit_weights_on_model_save. At stage 3, parameters are sharded across GPUs, so saving without turning this on leaves you with shard fragments that serving can't read. And turning on overlap_comm speeds things up by overlapping communication with compute, but it works against your activation memory headroom — if you're borderline on OOM, suspect this value first.

This layer hides almost nothing. In exchange, it does almost nothing for you. You have to write the data loader, checkpoint format, resume logic, and logging entirely yourself.

Layer 2 — Training Loop Frameworks

The layer stacked on top of layer 1 that provides "a loop that runs training end to end."

torchtitan is the PyTorch team's reference implementation, letting you compose FSDP2, tensor parallelism (including an async variant), pipeline parallelism, context parallelism, and HSDP, with support for Float8 and MXFP8 training. The repo describes itself as "a PyTorch native platform for rapid experimentation and large-scale training," and recommends the PyTorch nightly to get the latest features. In other words, it isn't built to be a production stack running on a stable release. As a reference to read and copy from, it's outstanding.

Megatron-Bridge is a library NVIDIA built to handle bidirectional conversion between Hugging Face checkpoints and Megatron-Core, with a training loop that uses Megatron-Core stacked on top. This bridge matters because of how the practical workflow goes: you receive the model in Hugging Face format, train with Megatron-Core's parallelism, and then have to export back to Hugging Face format for serving. Write that conversion yourself and you will inevitably get the tensor-name mapping and merge/split rules wrong. Documentation states that both NeMo-RL and SkyRL adopted this as their Megatron-Core connector.

NVIDIA NeMo is the comprehensive framework sitting on top of that. Pretraining through alignment through deployment are bundled as recipes, and the reinforcement-learning piece is split out into NeMo-RL. If you can just use NVIDIA's container environment as-is, it's the fastest path; if not, the dependency weight becomes a burden.

What this layer does for you is parallelism configuration, distributed checkpointing, resumption, and throughput logging. What it hides is the detail of the optimizer step and the communication schedule, which you'll need to reopen when debugging a loss spike.

Layer 3 — Config-Driven Post-Training Tools

The topmost layer bundles dataset format, prompt templates, LoRA config, and eval hooks together and runs off one command.

TRL is the post-training standard for the Hugging Face ecosystem. As of the 1.9.2 release notes it includes trainers for SFT, DPO, GRPO, RLOO, KTO, and reward modeling, and the KTO trainer dropped its experimental label in 1.8.0. The defaults themselves shift within minor releases — for instance, 1.7.0 switched the default loss to chunked NLL and added an MoE auxiliary loss. I saw a note on the release page about a few more trainers sitting in an experimental namespace, but I didn't confirm the individual algorithms, so check the list directly in your version's docs. TRL is a library where you absolutely must pin the version.

The minimal shape of using TRL directly looks like this. I've included version pinning and a loss-masking check together.

# pip install "trl==1.9.2" "transformers==5.14.1" "peft==0.20.0" "accelerate==1.14.0"
from datasets import load_dataset
from peft import LoraConfig
from trl import SFTConfig, SFTTrainer

ds = load_dataset("json", data_files="data/sft-train.jsonl", split="train")

cfg = SFTConfig(
    output_dir="out/sft-7b",
    per_device_train_batch_size=4,
    gradient_accumulation_steps=8,
    learning_rate=1e-5,
    num_train_epochs=2,
    bf16=True,
    gradient_checkpointing=True,
    max_length=4096,
    packing=False,              # leave off until you've verified the masking yourself
    logging_steps=10,
    save_steps=200,
    report_to="mlflow",
)

trainer = SFTTrainer(
    model="meta-llama/Llama-3.1-8B",
    args=cfg,
    train_dataset=ds,
    peft_config=LoraConfig(r=32, lora_alpha=64, lora_dropout=0.05,
                           target_modules="all-linear", task_type="CAUSAL_LM"),
)

# Eyeball one batch before you start training
batch = next(iter(trainer.get_train_dataloader()))
ids, labels = batch["input_ids"][0], batch["labels"][0]
tok = trainer.processing_class
print("full:", tok.decode(ids[:200]))
print("loss target:", tok.decode([i for i, l in zip(ids, labels) if l != -100][:200]))

trainer.train()

The last five lines are the most important part of this script. Positions masked with -100 are excluded from the loss, and even a slightly-off template can put loss over the entire prompt, or the opposite — leave the entire response out of the loss. The loss curve looks plausible and goes down in either case, so this usually only gets caught during evaluation after training finishes.

Axolotl stuffs the model, dataset, parallelism, and LoRA config into a single YAML file. Its biggest value is auto-tokenizing several data formats for you, and underneath it picks between Accelerate and DeepSpeed/FSDP.

# axolotl==0.18.0. accelerate launch -m axolotl.cli.train config.yaml
base_model: meta-llama/Llama-3.1-8B
load_in_4bit: false
strict: false

datasets:
  - path: data/sft-train.jsonl
    type: chat_template
    field_messages: messages

sequence_len: 4096
sample_packing: true
gradient_checkpointing: true
bf16: auto

adapter: lora
lora_r: 32
lora_alpha: 64
lora_dropout: 0.05
lora_target_linear: true

micro_batch_size: 4
gradient_accumulation_steps: 8
num_epochs: 2
learning_rate: 1e-5
warmup_ratio: 0.03
lr_scheduler: cosine

deepspeed: deepspeed_configs/zero3_bf16.json
output_dir: ./out/axolotl-8b

LLaMA-Factory sits in a similar category with a web UI and broad model support added on top, and Unsloth is on the side of improving single-GPU fine-tuning speed and memory through kernel optimization. Unsloth's multi-GPU support scope has shifted depending on the deployment form, and I couldn't confirm the exact policy as of this writing — check the repo's docs directly before adopting it if you need multi-GPU.

Large-scale reinforcement learning is a separate lineage. verl and OpenRLHF use a structure where a rollout engine (usually vLLM or SGLang) and a training engine (FSDP or Megatron-Core) are decoupled and attached to each other. If you're planning to run GRPO-family training at thousand-GPU scale, look at this lineage, not TRL.

What a Vanished Lineage Tells You — torchtune and torchforge

There's a case that shows exactly why picking a framework in this space is risky.

torchtune started as a PyTorch-native fine-tuning library and got well received. Then in July 2025, repository issue #2883 announced that active development had stopped, and the last PyPI release sits at 0.6.1, frozen since April 7, 2025. Its successor, torchforge, was announced. But as of August 2, 2026, the top of the torchforge repository carries this banner.

Development paused: Development in Forge has paused. LLM training at PyTorch is being consolidated in torchtitan.

In other words, the successor's successor is now being absorbed too. The lesson to take from this history isn't a knock on any particular library — it's the fact that the expected lifespan of a higher-level framework is shorter than the layer beneath it. The PyTorch distributed API and Megatron-Core have survived for years, but the convenience layer on top of them has been torn up on a roughly two-year cycle.

So the practical rule becomes: keep your project's core assets not in framework config files but in the data pipeline, the eval suite, and the checkpoint format. Keep these three outside the framework, and when the layer on top disappears, you can just swap it out.

Fine-Tuning Paths and Alignment Stages — Today's Names

There are two axes for deciding what to train. One is which weights you touch, the other is what signal you train on.

The LoRA-family variants (DoRA, rsLoRA, LoRA+, PiSSA, etc.) have all been absorbed into PEFT. Which variant lives in which PEFT version varies by release, so check the docs; and in practice, the knob to reach for first isn't the variant choice, it's rank, target modules, and learning rate. Taking QLoRA to a real production service is covered separately in QLoRA Production Fine-Tuning, and it's worth first checking whether fine-tuning is even the right choice at all in RAG vs. Fine-Tuning vs. Prompt Engineering.

The current names for the alignment stages are these.

1. Supervised fine-tuning. Sets format and style using instruction-response pairs. This is where most of the perceived quality gets decided.
2. Preference optimization. Splits into an offline family that trains directly on preference pairs (DPO and its variants), and an online family that samples from the policy and computes advantage within a group (GRPO, RLOO). TRL has trainers for both.
3. Reinforcement learning with verifiable rewards. Uses rewards scored by rule, like matching a math answer or passing a test. Since it doesn't use a learned reward model, the surface for reward hacking is narrow, and it has become the de facto standard for training reasoning ability.

The names keep multiplying. Every time a new acronym shows up, there are only three questions worth asking: where does the reward come from, does it need rollouts, and does a reference model need to sit in memory. Once those three are answered, the GPU count you need falls out.

The Moment a Higher-Level Framework Gets in Your Way

The convenience layer is a net win most of the time. But in the following situations, the cost outweighs the benefit.

• When you need to change the loss function or the masking. Which tokens carry loss is a central lever for post-training quality, and a higher-level framework hides this behind the data format. To confirm where the loss mask actually landed, you have to decode one batch and look at it with your own eyes yourself. Burning several days without doing this check is the most common accident.
• When your parallelism combination falls outside the standard set. A combination like turning on context parallelism and expert parallelism together is often something the higher-level framework's config schema simply can't express.
• When you need to debug. To find the cause of a loss spike, you need to look at gradient norms, activation statistics for a specific layer, and optimizer state, step by step. If a callback gets you there, great; if it doesn't, you end up forking the framework.
• When you can't pin versions. It genuinely happens that several higher-level tools each demand a different version of the same lower-level library. There's no way around it except building your own container image and committing a lockfile.

# At minimum, log this much before you start training
python - <<'PY'
import importlib.metadata as md
for p in ("torch", "transformers", "trl", "peft", "accelerate",
          "deepspeed", "flash-attn", "datasets"):
    try:
        print(f"{p:14} {md.version(p)}")
    except md.PackageNotFoundError:
        print(f"{p:14} (not installed)")
PY
nvidia-smi --query-gpu=name,driver_version,memory.total --format=csv
python -c "import torch; print('nccl', torch.cuda.nccl.version())"

Closing — Choose a Layer, But Don't Get Trapped In It

The criterion for choosing a stack isn't "which is best," it's "which layer are we willing to be directly responsible for." If you're pretraining, you have to go down to Megatron-Core or the PyTorch distributed API; if you're only fine-tuning on in-house data, it's reasonable to stop at TRL or Axolotl.

Whichever layer you pick, though, keep the data pipeline, eval suite, and checkpoint format outside the framework. That's exactly what the two-year history from torchtune to torchforge and back into torchtitan is telling you. A framework is a part you swap out, and designing it so that swap is possible is the actual design work. The principles of parallelism itself are laid out in The Four Kinds of Multi-GPU Parallelism.