Page Not Found
Page not found. Your pixels are in another canvas.
Sitemap
A list of all the posts and pages found on the site. For you robots out there is an XML version available for digesting as well.
Pages
This website is still under development, please refer to my CV for more information about me
Page not in menu
This is a page not in th emain menu
Posts
Epoch-wise bias-variance decomposition
Published:
Let’s suppose we’re training a model parameterized by $\theta$, and let’s denote by $\theta_t$ the parameter $\theta$ at step $t$ given by the optimization algorithm of our choice. In machine learning, it is often helpful to be able to decompose the error $E(\theta)$ as $B^2(\theta)+V(\theta)+N(\theta)$, where $B$ represents the bias, $V$ the variance, and $N$ the noise (irreducible error). In most cases, the decomposition is performed on an optimal solution $\theta^*$ (for instance, $\lim_{t \rightarrow \infty} \theta_t$, or its early stopping version), for example, in order to understand how the bias and variance change with the complexity of the function implementing $\theta$, the size of this function, etc. This has helped explain phenomena such as model-wise double descent. On the other hand, it can also be interesting to visualize how $B(\theta_t)$ and $V(\theta_t)$ evolve with $t$ (which can help explain phenomena like epoch-wise double descent): that’s what we’ll be doing in this blog post.
Generating Random Variables and Stochastic Processes, Generative Flow Networks (GFlowNets)
Published:
Word embeddings
Published:
portfolio
Word embeddings
Published:
Intrinsic Dimension Estimation
Published:
publications
On the use of linguistic similarities to improve Neural Machine Translation for African Languages
Pascal Jr. Tikeng Notsawo, Brice Nanda, James Assiene, 5th Black in AI Workshop @ NeurIPS, 2021.
Stochastic Average Gradient : A Simple Empirical Investigation
Pascal Junior Tikeng Notsawo, IFT6512, Stochastic programming, Université de Montréal, 2023.
Adaptive Discrete Communication Bottlenecks with Dynamic Vector Quantization for Heterogeneous Representational Coarseness
Dianbo Liu, Alex Lamb, Xu Ji, Pascal Jr. Tikeng Notsawo, Mike Mozer, Yoshua Bengio, Kenji Kawaguchi, In Thirthy-Seventh AAAI Conference on Artificial Intelligence, 2023.
Predicting Grokking Long Before it Happens: A look into the loss landscape of models which grok
Pascal Jr. Tikeng Notsawo, Hattie Zhou, Mohammad Pezeshki, Irina Rish, Guillaume Dumas, ICLR 2024 Workshop on Mathematical and Empirical Understanding of Foundation Models, 2023.
Lost in Translation: The Algorithmic Gap Between LMs and the Brain
Tommaso Tosato, Pascal Jr. Tikeng Notsawo, Saskia Helbling, Irina Rish, Guillaume Dumas, Workshop on Large Language Models and Cognition, ICML, 2024.
Grokking Beyond the Euclidean Norm of Model Parameters
Pascal Jr. Tikeng Notsawo, Guillaume Dumas, Guillaume Rabusseau, Forty-Second International Conference on Machine Learning (ICML), 2025.
talks
On the use of linguistic similarities to improve Neural Machine Translation for African Languages
Published:
Predicting Grokking Long Before it Happens: A look into the loss landscape of models which grok
Published:
teaching
Group and home rehearsal courses
Yaounde, Cameroon, 2016, 2017
During my engineering training, I gave tutoring in mathematics, physics and chemistry to college students, at home (private) and in group.
Preparatory classes
Yaounde, Cameroon, 2017, 2018
During my training as an engineer, I prepared many students in mathematics and physical sciences (in short MSP, French system) for the entrance exams of the Grandes Ecoles in Cameroon.