Multi-gpu/multi-node Ddp Training

Minimal, readable ~300-line PyTorch repo for training and finetuning GPT-2-scale models; reproduces GPT-2 124M on OpenWebText. Author marks it deprecated in favor of the newer nanochat, but it remains a compact hacking/education base.

AI Frameworks | Python · gpt · llm-training