ARC Prize Rejects OpenAI's 99.9% Astra Claim: Same-Standard Retest Scores Only 62.7%

OpenAI once claimed GPT-6 Astra scored 99.9% on the ARC-AGI-3 benchmark, and NVIDIA CEO Jensen Huang cited it to declare that "AGI has arrived." But ARC Prize, the benchmark's creator, retested Astra under the same standard procedure used for other models and scored it at only 62.7% — still the benchmark's top result, yet far from 99.9%. The key dispute is the test setup: ARC Prize says the 99.9% came from OpenAI's custom "tools-enabled" harness, which lets the model preserve reasoning state across actions in a way the standard setup does not, at a reported cost of nearly $19,000 per run. ARC Prize therefore ruled the score invalid, since it was not produced under the uniform conditions every lab's model must meet; under the common yardstick the score drops by more than 37 points. More striking is the harness effect itself: ARC Prize's October benchmark revisions show the open-source Prime Agent harness lifted public scores from about 30% to over 95%, and a five-person NVIDIA team took the same Claude Opus 5 from 30% to 100% simply by rebuilding the orchestration code. With weights unchanged, scores can swing by nearly 40 points — suggesting the current bottleneck may lie in orchestration code around the model, not the model itself. The 62.7% is still a huge leap from March, when frontier models scored near 0.3% while humans scored 100%. But a top verified score still under two-thirds is a long way from an "AGI has arrived" headline, and judgments about the AGI timeline are more reliable when based on the benchmark creator's uniformly verified scores.