I'm interested in the nature of emergence, which I study using algorithmic information theory and machine learning. Emergence is the observation that systems exhibit new behavior at different degrees of scale and complexity. For example: a cell is alive, but is only made of dead things. Water is wet, yet a single molecule is not.
Previously, I did research in mathematics and physics. I've worked on projects in knot theory, neutrino physics, and quantum many-body localization. In industry, I've worked in computational geophysics, reinforcement learning for robotics, and natural language processing with LLMs at a startup I founded.
A brief statement on my research interests is available here.
Under submission (twitter thread, blog, arXiv, link to talk)
We explain the grokking phenomenon, which is the sudden transition in neural networks from memorization to perfect generalization. To do this, we propose a new measure of intrinsic complexity for networks, based on Kolmogorov complexity and algorithmic rate–distortion theory, which formalizes lossy compression of models. We track the complexity dynamics during the phase transition, and find a characteristic rise and fall of complexity, corresponding to memorization followed by generalization.
Under submission
We extended DITTO (below) to work for long-horizon manipulation tasks on a real robot. Train in dream (world model), transfer zero-shot to real! We also upgraded DITTO to be multi-task, and can condition it on natural language commands.
Preparing for submission (arXiv)
We can't train policies online, on robots, because it's too sample inefficient (and potentially dangerous). Training in simulation can be problematic if the simulator fails to model all relevant dynamics. To address these issues, we propose a method which takes inspiration from DAgger and World Models. We learn a simulator from data, then let a policy practice matching expert demonstrations inside the learned simulator (world model). The policy learns how to correct its own mistakes over multiple time-steps, by learning to match its latent trajectory to an expert demonstrator.
First International Meeting for Applied Geoscience & Energy, 2021
A case study using a computer vision system I developed in conjunction with physical models which was led by a researcher at a customer seismic exploration company. The seismic visualizations on this page were produced around this time, as part of some experiments I ran to study generative and latent space modeling with GANs, which I explain here.
American Go E-Journal, Deutsche Go-Zeitung, 2020
An essay on the history of AI in the game of go. Republished by the American Go Association and the German Go Newspaper.
DUNE Collaboration Technical Report, 2016
I ran simulations of proposed changes to the beamline geometry for the DUNE experiment, to understand the effect of the changes on neutrino flavour production statistics. Published as a technical note to the internal DUNE collaboration.