Notes on a Methodology Transition
Moving from selectionist ML research (run many probes, kill the bad ones) to physicist-mode research (theorem first, two parameters per probe, universality over coverage). A live chronicle.
-
What Experimental Design Actually Means
Theoretical physicists barely need it. Experimental physicists cannot live without it. Life sciences rewrote it for complexity. Pharma made it law. ML borrows the wrong one.
-
Hypothesis Testing from Scratch, and Its Bayesian Analogue
Frequentist hypothesis testing rebuilt from first principles for ML researchers who half-remember p-values. Then: the Bayesian reframe, why it fits the kill-ladder better, and what each one actually buys you.
-
Two Research Modes, and Why the Second One Needs Lean 4
AI makes hypothesis generation cheap. Evaluation stays expensive. Lean 4 proofs are the filter that changes the economics: a proved theorem screens an entire family of candidates before GPU time is allocated.
-
Naming What Fails: The Obstacle Taxonomy
25+ preregistered kills over six weeks of compression research. The tempting story is "compression is hard." The physicist story is better: 25 kills, ~10 structural failure patterns, one Lean theorem per class.
-
Theorem-Screened Experiments
A three-step decision rule for running fewer, better experiments: check your theorem library before you touch a GPU. Calibration checks, falsifier traps, and parameter compression from the physicist mode of ML research.
-
The Five-Minute Daily Drift Check
Solo research programs drift one small exception at a time. Three shell commands, run daily, catch the most common protocol violations before they compound into reproducibility failures.