I work on the theory of learning systems. I recently discovered a new type of phase transition in learning systems: Just as physical systems can transition between distinct phases of matter, I showed how learning systems transition between distinct phases characterized by their complexity. As deep neural networks transition from memorization of their training data to perfect generalization, there is a corresponding rise-and-fall of complexity in the network, as measured by its compressibility.
In the past, I've worked on projects in knot theory, neutrino physics, and quantum many-body localization. In industry, I've worked in computational geophysics, reinforcement learning for robotics, and natural language processing with LLMs at a startup I founded.
I hold a postdoctoral appointment in the Mathematical Institute at Oxford, affiliated with the Erlangen AI Hub.
We derive a tight generalization bound for deterministic neural networks based on their last hidden layer representations. By evaluating representation quality with a parameter-free k-nearest-neighbor (k-NN) classifier, our certificate avoids traditional parameter-space complexity measures, KL divergences to a posterior, weight norms, and compression. The bound is tight: trained from scratch, it reaches an 8.0 percentage point gap to test error on CIFAR-10 (ResNet-18) and yields the first non-vacuous certificates we are aware of on CIFAR-100; with frozen DINOv2 representations, it certifies at 1.4% on CIFAR-10 and 23.1% on 1,000-class ImageNet. It decomposes additively into three empirical, independently measurable quantities: local class geometry (k-NN decoder error), decoder-network agreement, and training stability. This allows the bound to act as a diagnostic tool: we demonstrate that actively destroying local class geometry eliminates generalization while leaving dataset memorization intact, identifying class geometry as a causally relevant mechanism for generalization. Finally, we show that learned representations are significantly less sensitive to weight noise than linear readouts, explaining why parameter-space certificates often struggle to remain tight.
We study complexity and generalization in learning systems. We begin by framing learning as a dynamical non-equilibrium process, and suggest that a more complete understanding of machine learning requires methods beyond statistical description. We point out that machine learning must be considered both a mathematical and a natural science, since it requires both a theory of the formal learning system and the environment it operates within. Our central technical contribution is a dynamical complexity measure based on the theory of Kolmogorov complexity and lossy compression. We use this measure to demonstrate a complexity phase transition during learning, in neural networks which suddenly generalize after initially overfitting their training data. We also explore generalization in multi-step decision processes, which break common statistical assumptions underlying generalization. Finally, we explore generalization of learning systems trained only in simulation to the real world.
arXiv, Physica D: Nonlinear Phenomena (twitter thread, blog, link to talk)
We observe a complexity phase transition in grokking neural networks, which suddenly transition from memorization to perfect generalization. We find that during this transition, there is a corresponding rise and fall of complexity in the networks. We explain this phase transition using ideas from Kolmogorov complexity and rate-distortion theory, and derive a principled lossy compression framework for neural networks which allows us to track their complexity dynamics.
arXiv, ICRA 2025, project page, twitter thread
We demonstrate robot policies trained purely offline in a world model transfer to the real world zero-shot. LUMOS is an upgrade to DITTO, adding planning in the learned world model latent space for improved long-horizon performance. LUMOS conditions on natural language commands to perform multi-task manipulation with a single network.
Normally when we train policies offline, it also means they learn off-policy. We use world models to enable safe offline training on-policy. We use reinforcement learning inside the latent space of a learned world model to induce imitation learning, using a robust reward defined in the learned latent space. On-policy RL in the world model transfers to robust imitation learning in the real environment.
First International Meeting for Applied Geoscience & Energy, 2021
A case study using a computer vision system I developed in conjunction with physical models which was led by a researcher at a customer seismic exploration company. The seismic visualizations on this page were produced around this time, as part of some experiments I ran to study generative and latent space modeling with GANs, which I explain here.
American Go E-Journal, Deutsche Go-Zeitung, 2020
An essay on the history of AI in the game of go. Republished by the American Go Association and the German Go Newspaper.
DUNE Collaboration Technical Report, 2016
I ran simulations of proposed changes to the beamline geometry for the DUNE experiment, to understand the effect of the changes on neutrino flavour production statistics. Published as a technical note to the internal DUNE collaboration.