annotated_deep_learning_paper_implementations

View on GitHub

🧑‍🏫 60+ Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠

★ 67K6,759 forksPythonMITlabmlai

LabML's annotated deep learning library: 60+ readable PyTorch implementations of papers with side-by-side notes, covering transformers (GPT, ViT, XL), LoRA, diffusion/Stable Diffusion, GANs, optimizers, and RL. Best as a learning and reference resource rather than a production framework.

Use Cases

Learning how transformer and attention variants are implementedReferencing readable PyTorch code for LLM architectures like GPT-NeoXStudying LoRA and LLM.int8 quantizationUnderstanding diffusion and Stable Diffusion componentsComparing optimizers (Adam, RAdam, AdaBelief, Sophia)Reimplementing GAN variants (CycleGAN, StyleGAN2)Reinforcement learning algorithm reference (PPO, DQN)Studying RL and CFR game solversReading side-by-side annotated paper implementationsCopying building blocks for custom model researchUnderstanding normalization and activation layersStudying language model sampling techniquesExploring scaling/memory optimization (ZeRO-3)

Built With

Language: Python
Frameworks: PyTorch · JAX · Triton · einops · faiss · gym · torchvision · labml

annotated_deep_learning_paper_implementations

Use Cases

Built With

Tags