Fareed Khan · AI Infrastructure

train-llm-from-scratch

Educational PyTorch repo implementing a decoder-only transformer from scratch (token+position embeddings, multi-head attention, stacked blocks), scaling 13M → 2.1B params (64 blocks) on a subset of The Pile with tiktoken + AdamW. Includes a full post-training suite: SFT, reward modeling, PPO, DPO, GRPO.