Writing
Blog
Notes on machine learning, mathematics, and research.
Grokking Beyond the Euclidean Norm of Model Parameters
Why grokking is governed by the property favored late in training—and why that property need not be the Euclidean norm.
What Is Grokking? A Formal Definition
A formal definition of grokking based on the delay between memorization and generalization.
Smoothness, Descent, and Cocoercivity
Definitions, lemmas, and proofs connecting smoothness, descent guarantees, Hessian bounds, and the Baillon–Haddad theorem.
Epoch-wise Bias–Variance Decomposition
A bias–variance decomposition for tracking how bias, variance, and irreducible noise evolve throughout training.
Visualization of the loss landscape and optimization path of a neural network
How low-dimensional subspaces can preserve an optimization trajectory and make a neural network’s loss landscape interpretable.
Word embeddings
The genesis of my word embeddings tutorial, and what led me to machine learning research.