Large Discovery Models: Empirically-grounded Model-Based Open-Ended Search

Scientific discovery often involves optimising expensive-to-evaluate objectives over vast, structured, and open-ended hypothesis spaces, such as molecules, protein sequences, and computer programs. Generative models such as large language models (LLMs) provide expressive priors over such spaces, but their likelihoods and self-assessments are unreliable proxies for the objectives and calibrated epistemic uncertainty, especially for novel candidates outside the observed data distribution. We introduce the Large Discovery Model (LDM), an empirically grounded recurrent architecture that couples a generative model with a Bayesian non-parametric reward surrogate model. The generative model proposes and refines candidate designs, while the surrogate predicts their performance and quantifies uncertainty, yielding an uncertainty-aware value that guides candidate generation, refinement, and selection. The discovery memory and the surrogate model are continually updated as each new experimental observation arrives. We evaluate LDM on three scenarios spanning different design modalities and objectives, including neural-network training, antibody design, and molecular optimisation.

Introduction. The development of large language models (LLMs) has progressed from open-ended generation towards increasingly capable forms of reasoning. Pretraining provides broad linguistic and conceptual knowledge, while inference-time scaling allows a model to generate, compare, and refine multiple candidate solutions. This approach has produced substantial gains in mathematics, coding, theorem proving, and game playing, where solutions may be difficult to find but comparatively easy to evaluate. Formal derivations can be assessed by process or outcome verifiers, programs can be executed against unit tests, and game strategies can be evaluated using exact rules or simulators (Yao et al., 2023a; Madaan et al., 2023; Snell et al., 2025). AlphaZero-like methods make this feedback loop explicit by combining generation, evaluation, and search (Silver et al., 2017; Feng et al., 2024b). OpenR (Wang et al., 2024) provides an open framework for studying such reasoning systems, while AlphaEvolve (Novikov et al., 2025) couples languagemodel proposals with automated evaluators at scale.

Discussion / Conclusion. , Limitations, and Outlook The core challenge of scientific discovery lies in searching for optimal solutions within vast, structured, and open-ended design spaces under a limited budget of costly evaluations. Existing paradigms exhibit complementary limitations: large language models (LLMs) possess strong structured generative capabilities and domain priors, but cannot reliably estimate the true value of external objectives or their associated uncertainty. Bayesian optimisation, by contrast, delivers uncertainty-aware experimental decisions from sparse observations, yet struggles to autonomously propose valid candidates in complex discrete spaces. This work presents the Large Discovery Model (LDM), an experiment-grounded recurrent architecture that deeply couples the generative prior of LLMs with the value signal from a Gaussian process surrogate.