The Geometry of MoE Compression (Part E)
Expanding on Phase-Collapse: β transfer in FFNs, formal Softmax soundness, and recovering dense performance from quantized layers.
-
The β-lift and FFN Transfer: MoE Compression Part E
Why β transfer in FFNs matters for quantization and the formal 'structure bonus' theorems in MoE compression.
-
Formalizing the Softmax and Hessian: Lean 4 Soundness Proofs
A deep dive into the Lean 4 proofs for Softmax soundness and the BetaLocalHessian skeleton in the lean-mining project.
-
Dense Activation-Fit Recovery: Healing Quantized Layers
How to recover dense performance from quantized layers using activation-fit artifacts and the recovery scripts in lean-mining.