The Pi/8 Instruction Output Audit: Phase 5 Benchmarks
Final results for the Phase 5 benchmarks are in. This post is the “results ledger” entry for the rotational arms—a definitive null result for the fixed placement, and a clean baseline for the Pre-W mechanism.
The SEP Scoreboard: A Negative Delta-of-Deltas
Our decisive metric is the SEP delta-of-deltas: the gain in selectivity from RoPE-Provenance relative to the gain from data alone.
Verdict: The arm failed the pre-registered gate. Its delta-of-deltas was -0.080. It essentially “wasted” the curriculum gain that the vanilla model exploited.
The Instruction Output Audit
To make sure this wasn’t a detection artifact (i.e., the model was following but in a different style), we ran a manual audit of 20 INSTRUCTION-slot outputs (pi8_instruction_output_audit.json).
- Vanilla: Executed the instruction in 3/20 cases.
- Pi/8: Executed in 0/20 cases.
The outputs weren’t empty; they just drifted into repetitive loops or generic summaries. The rotation didn’t just “mask” the instruction; it destroyed the model’s ability to process it as an instruction.
Conclusion: The Rotational Matrix is Closed
The Phase 5 results close the v1/v2 rotational matrix as an Informative Failure.
- Post-projection fixed rotations: Selectively damage the instruction stream (Asymmetric Compliance Damage).
- Learnable post-projection: Optimizer closes the channel to avoid damage.
- Pre-W placement: Restores vanilla-like utility and compliance, but lands in the same SEP band as the zeroed control.
The “Pre-W” placement remains our strongest architectural null. It fixes the damage, even if it hasn’t yet unlocked the separation. The search for a provenance mechanism that the model wants to use continues.
Full results and experiment logs are at the RoPE-Provenance tracker.
This is the final post in this series.