FrontierChallenge: Evaluating Scientific Workflow Completion

Scientific agents increasingly analyze data, execute code, and produce research artifacts, yet most benchmarks emphasize final answers, isolated programs, or a single domain. We introduce FRONTIERCHALLENGE, a cross-domain benchmark comprising 300 endto-end scientific workflows. In this paper, we release and evaluate 97 of these tasks, spanning quantum chemistry, molecular dynamics, materials characterization, analytical chemistry, life science, and electrochemistry/environment. Each task provides fixed inputs and specifies a bundle of required scientific deliverables. We evaluate twelve frontier models with three agent scaffolds. Pass Rate measures the fraction of tasks satisfying the full-completion criterion, while Avg. Score captures partial progress. Each of the best-performing configurations completed only 20 of the 97 released tasks, yielding a Pass Rate of 20.6%. Partial progress translated especially poorly into complete delivery in analytical chemistry and electrochemistry/environment: Avg. Scores reached 87.6 and 94.9, but the highest Pass Rates were only 4% and 0%. Among non-passing Claude Code trajectories, 75.5% still ended with language claiming completion.

Introduction. Language models are evolving from text generators into agents that can plan, call tools, execute code, and modify persistent files (Liu et al., 2024; Mialon et al., 2024; Xie et al., 2024). Alongside advances in agent scaffolding, continual pre-training has been explored as a way to scale general agent capabilities (Su et al., 2025). Recent systems suggest that agentic support can extend beyond isolated tasks such as literature retrieval or text translation to coordinating multi-stage research workflows with inspectable outputs (Lu et al., 2024). This shift changes what constitutes success on a scientific task. Producing a plausible conclusion is not enough: an agent may need to inspect heterogeneous inputs, select and run an analysis, validate intermediate results, and deliver mutually consistent code, tables, figures, and prose. Existing benchmarks cover expert knowledge, general tool use, software interaction, code repair, paper replication, and scientific data analysis (Phan et al., 2025; Jimenez et al., 2024; Chen et al., 2025; Siegel et al., 2024; Starace et al., 2025).

Discussion / Conclusion. FRONTIERCHALLENGE evaluates whether scientific agents can complete specified, multi-stage workflows and deliver mutually consistent artifacts rather than merely produce plausible answers. The benchmark comprises 300 workflows; this study releases and evaluates 97 across six scientific domains using taskspecific executable Graders. Across the evaluated models and agent scaffolds, Pass Rate ranged from 3.1% to 20.6%, despite Avg. Scores of 67.5 to 87.9. GPT-5.6 Sol with Codex achieved the highest Avg. Score and shared the highest Pass Rate with Grok 4.6 using Claude Code. Partial progress translated especially poorly into complete delivery in analytical chemistry and electrochemistry/environment, and the distinct domain profiles show that aggregate rankings do not capture every scientific setting.