ML from Scratch

I'm Harshith Varma, a Data Scientist at Confluent. I'm not someone who knows everything about machine learning. I'm someone who wants to understand everything about machine learning. Most ML resources teach you what to do — import this, call this function — but not why it works. That gap in my own understanding is what pushed me to start deriving every algorithm from scratch instead of treating ML as a toolbox. My goal isn't for you to finish this book. It's for you to reach the point where you can look at any machine learning model and think: "I understand why this works." You don't need a PhD to read this book. You only need a willingness to ask "why?" one more time than most people do. Welcome to the journey. Let's build machine learning from scratch.

Twenty chapters, four parts: Math Foundations, Build a Neural Net, PyTorch, and Training. Read them in order. Every chapter builds something the next one uses. Every chapter carries the same three code layers, in the same order: pure Python (every multiplication shown), NumPy (vectorized), then PyTorch (what you'd write at work). Four recurring callouts do the rest of the teaching — Insight for the key idea, Warn for practical failure modes, Challenge for a do-it-now exercise, and Personify, where a concept explains itself in first person. Every interactive widget on the companion website (ml.harshithvarma.in) becomes a static figure in the book, plus a pointer back to the live version. The figures are honest — you can read this book cover to cover offline. But the widgets let you move the sliders yourself, and moving the sliders is how the intuition sticks.

1. Vol. 1

   Foundations

   Available · Chapters 1–20

   The first twenty chapters. Gradient descent through a trained digit classifier — every derivation shown, every line of code earned.

   • Math Foundations
   • Build a Neural Net
   • PyTorch
   • Training
2. Vol. 2

   Deep Learning: From CNNs to GPT

   Drafted · Chapters 21–48

   Convolution through a working GPT you train and talk to. Fully drafted; not yet through the full review gate or published.

   • CNNs & Vision
   • RNN & LSTM
   • NLP
   • Attention & Transformers
   • Build GPT
3. Vol. 3

   Sections 10–14 (planned)

   Planned · Chapters 49–75

   Scope is set; chapters have not been drafted yet.

   • Fine-Tuning & RLHF
   • Mixture of Experts
   • Diffusion Models
   • Reinforcement Learning
   • Inference & Serving

Every derivation and figure in this book started life as an interactive lesson. The live site covers all 14 sections and 75 lessons with runnable widgets — read the book, then go move the sliders.

N Sai Harshith Varma is a Data Scientist at Confluent, where he builds AI-powered systems professionally. His fascination with machine learning has always been driven by a simple question: why does this actually work? That curiosity has led him to derive algorithms from first principles, implement them in pure Python before reaching for libraries, and share what he learns through articles, open-source projects, educational content, and hands-on workshops at universities. Beyond his work at Confluent, he enjoys building machine learning systems from scratch, exploring large language models, participating in data science competitions, and helping students and developers develop a deeper understanding of AI. He believes that understanding comes not from memorizing APIs, but from knowing what happens underneath them. ML from Scratch is the first book in a series dedicated to making machine learning transparent, intuitive, and deeply understandable, one algorithm at a time.