Building a Transformer From Scratch with NumPy (No PyTorch, No TensorFlow)

Building the Attention Is All You Need Transformer from scratch in NumPy, with a focus on deriving the mathematics, tracking tensor shapes, and understanding why each component works.

Attention Is All You Need

I want this section to hold things that are useful to reread: reading notes, small technical explanations, project writeups, and occasional essays when a topic deserves more space.

The standard is not polish for its own sake. The standard is whether a post captures an idea clearly enough that it helps future work.

Over time, this should become a record of how projects evolve, what research questions seem interesting, and which implementation details were worth writing down.