← ALB News · curated headline

Match and Correct: A Repeated Computational Motif in Evoformer Layers

arXiv AI · WORLD · AI · 11/17/2026, 12:00:00 AM

arXiv:2611.19016v1 Announce Type: new Abstract: Triangle attention and triangle multiplicative updates form key components of the Evoformer, and were introduced specifically to encourage consistency with the triangle inequality. Yet whether representational changes over layers reduce triangle inequality violations has not been tested. We probe the pair representation at each of OpenFold's 48 Evoformer layers, decoding final C$_\alpha$-C$_\alpha$ distances with four probe families (a shared linear probe, per-protein linear probes, a sequence-separation-gated linear probe, and XGBoost) and reconstructing intermediate 3D structures from the decoded distances via multidimensional scaling. Across a cohort of 552 CAMEO and ECOD proteins, distance decodability ($R^2$) and reconstruction quality (TM-score) rise in early layers, plateau in the middle of the network, and rise again in late layers. Directly measuring triangle-inequality violations in the decoded distances reveals a similar, more exaggerated pattern: violations rise to a peak, fall to a trough, rise to a second peak, and only then decline, revealing two distinct episodes of decreasing normalized triangle violations rather than one continuous improvement. Cumulative forward and reverse layer-wise ablations of the Pair Transition, MSA Transition, and triangle mechanisms localize the regions in which specific Evoformer components become essential. The two stages of geometric refinement coincide with the use of the Pair Transition MLP and the triangle mechanisms. These results characterize how distance information and decoded geometric consistency evolve across layers, and motivate hypotheses about the computations behind their trajectories.

Continue to arXiv AI →

You are being forwarded in 6s — stay on this page