AI
Ai2's StateBench shows frontier models losing the thread on long assignments
The Allen Institute for AI released StateBench, a four-tier benchmark for tracking how variables change across long documents, from fixed values up to assignment by reference. Gemini 3.8 Flash, Claude Opus 5.5 and both GPT-6 variants handled the easy tiers but fell to 13% accuracy or lower on the longest general-finite problems. Small Gated DeltaNet models trained on the task generalised far beyond their training length; transformers never learned the hardest tier.