Writing
Blog
Notes on machine learning, mathematics, and research.
Eigenpairs from Matrix–Vector Products: Power Iteration, Deflation, Lanczos, and Arnoldi
What power iteration, deflation, and the Lanczos method recover about a symmetric matrix from matrix–vector products alone: convergence, failure modes, exact termination, and residual certificates, with complete proofs, and what the Arnoldi method recovers about a nonsymmetric one.
Grokking Beyond the Euclidean Norm of Model Parameters
Why grokking is governed by the property favored late in training—and why that property need not be the Euclidean norm.
What Is Grokking? A Formal Definition
A formal definition of grokking based on the delay between memorization and generalization.
Smoothness, Descent, and Cocoercivity
Definitions, lemmas, and proofs connecting smoothness, descent guarantees, Hessian bounds, and the Baillon–Haddad theorem.
Epoch-wise Bias–Variance Decomposition
A bias–variance decomposition for tracking how bias, variance, and irreducible noise evolve throughout training.
Visualization of the loss landscape and optimization path of a neural network
How low-dimensional subspaces can preserve an optimization trajectory and make a neural network’s loss landscape interpretable.
Word embeddings
The genesis of my word embeddings tutorial, and what led me to machine learning research.