Research Blog

Ideas behind the papers, explained through pictures, examples, and a little mathematics.

Sampling near a continuum of solutions The solid blue ellipse is the exact set of zero-loss solutions. Sixty-four dots show actual simulated ULA positions after 6,000 steps, at temperature 0.06 and step size 0.01. At positive temperature the particles lie near the set, rather than exactly on it. This finite toy run does not establish equilibrium. Explore a whole solution set x₁ x₂ Zero-loss solutions 64 simulated ULA particles
Toy ULA simulation after 6,000 steps.

· Interactive explainer

Sampling from Manifolds of Minimizers

Explore Gibbs and Langevin sampling, and why guarantees across a family of losses matter.

Explore the simulations
Equal training loss, different validation scores A, B and C lie on the same zero-training-loss curve. Their illustrative validation scores differ. B has the highest score and is selected. Geometry and score proportions are from the article's presentation example; scores are not measured results. Equal training loss Every point on the curve has loss 0. ABC Validation score ↑ ABC
Toy example with illustrative validation scores.

· Visual explainer

Same Training Loss, Different Behavior: Which Solution Should We Choose?

How choosing among training solutions shapes the data mixtures we learn for LLM fine-tuning.

Read the visual story
Top