Aspire: Can Models Self-Evolve from Vague Goals?

Many important forms of human learning begin with a vague goal, such as “become a better physicist” or “improve at research.” Learners must interpret the goal, identify capability gaps, decide how to learn, and determine whether they have actually improved. In contrast, existing work on LLM self-evolution typically begins with tasks and evaluation metrics specified by humans, reducing self-evolution to optimizing an explicit objective rather than deciding what and how to learn. We introduce Aspire, a benchmark for vague-goal-driven self-evolution. Aspire provides only a natural-language capability goal while downstream evaluation tasks remain hidden. The agent must operationalize the goal by choosing data and update methods, constructing training and validation signals, and deciding when to evaluate. Aspire supports both model-weight and agent-harness evolution in a unified interactive environment and evaluates the resulting systems on a hidden, expert-authored set of 520 items spanning six goals. Our experiments show that vague goals redirect search effort toward goal interpretation.

Introduction. Many important forms of human learning begin with a broad capability direction rather than a predefined benchmark, training set, or fully computable reward function. A student seeking to become a better physicist, for example, not only chooses textbooks, exercises, and study methods, but also identifies gaps in their knowledge, prioritizes which capabilities to develop, and determines whether learning has produced genuine progress. Autonomous learning therefore involves three coupled decisions: what to improve, how to improve it, and how to verify the improvement. Existing work on LLM self-evolution focuses primarily on the second decision. Given a concrete task, evaluation script, and success metric, LLM agents can already collect data, run training, and revise post-training strategies from feedback. PostTrainBench [20], LaMDAgent [29], Evo-Memory [25], and SEAL [37] show that modern LLMs can autonomously search for effective optimization paths toward a specified objective.

Discussion / Conclusion. Modern LLM agents can search for effective ways to improve a specified objective. The next step is to determine what should be learned when no agent-visible task specification or decomposable reward has already operationalized progress. We formalize this problem as vague-goal-driven self-evolution and introduce Aspire, which pairs vague goals with hidden evaluation and auditable trajectories. Across three studies, the RQ1 process slice associates vague-goal prompting with more goal-definition work; self-directed runs produce evaluated checkpoints far more often than retained improvements; and the highest successor-harness mean remains below a fixed reference under unchanged runtime weights. These results show why progress must be judged against the base model rather than only against the previous checkpoint: closing the training loop is not yet the same as closing the capability loop. Our conclusions are bounded by the six goals and the coverage and scoring quality of their expert-authored evaluation items, one canonical adaptive-feedback run per configuration–goal cell, and content-level trace evidence available only under controlled access.