TOPIC
Evolutionary Methods
A subject the collection covers, read through 5 synthesis notes.
View as
Can evaluators improve alongside the agents they score?
Most self-improving systems rely on a fixed benchmark or verifier that doesn't change. But what if the evaluator itself learned and adapted as the agent improved? This explores whether co-evolution unlocks tasks that resist static scoring.
Darwin Gödel Machine achieves open-ended self-improvement via empirical validation Machine feedback from evaluators enables discovery through test-time compute scaling Self-improving agents need intrinsic metacognition to adapt learning strategies Non-stationary objectives with per-epoch guarantees prevent reward hacking in recursive self-improvement Self-improvement is fundamentally bounded by the generation-verification gap
What makes evolving the benchmark different from evolving the optimizer itself? How does controlled utility evolution prevent the evaluator from becoming a new bottleneck? Can a progressively stricter evaluator act like a curriculum for improving agents? Why do most self-improving systems fail when given tasks with no clear external benchmark? How do epoch boundaries preserve self-improvement guarantees across objective changes?
Do self-improving agents really split into two distinct loops?
Explores whether modern self-improving agents can be understood through a clean abstraction separating fast scaffold updates from slow model weight updates, and whether this framework actually explains the field's recent progress.
Agents need two complementary adaptation timescales for continuous learning Versioned resources with auditable lineage enable safe agent self-evolution Self-improving agents need intrinsic metacognition to adapt learning strategies Reward models evaluate responses largely without reading the question Darwin Gödel Machine achieves open-ended self-improvement via empirical validation
How does compiling natural language goals into executable code enable objective evolution? Can harness updates benefit agents equally across all model sizes? What stops evolved agent behaviors from generalizing beyond specific tasks? Can stopping rules extracted from past failures improve agent reliability without retraining? Why does the harness layer accumulate distributed behaviors over time? Can weaker planners match stronger models if behavior is reorganized? Can AI systems generate and refine their own objective functions? How would a bi-level agent restructure objective functions during discovery? Why do self-improving agents concentrate progress in the fast non-parametric loop? How should versioning and rollback govern the fast scaffold update loop? Can self-improving agents become truly autonomous without intrinsic metacognition? What makes behavior localization the bottleneck in agent harness evolution?See all 13 inquiring lines on this note →
How can agent self-evolution be made safe and auditable?
As agents begin updating their own prompts and tools, how can we track these changes, measure their effects, and safely reverse problematic updates? This matters because untracked evolution leads to unmaintainable systems and makes regressions impossible to diagnose.
Should coordination protocols wrap existing systems or replace them? What makes agent-authored code worth persisting and sharing? Do stronger models always evolve harnesses better? What stops large language models from improving themselves? Do self-improving agents really split into two distinct loops?
Coordination layers win by wrapping protocols rather than replacing them. Agent-authored persistent code remains the least studied harness layer Harness updating is flat but benefiting peaks at mid-tier models Language models hit a hard ceiling on self-improvement without external input Self-improving agents divide into slow parametric and fast non-parametric loops
Why do LLM agents ignore condensed experience summaries?
LLM agents faithfully learn from raw experience but systematically disregard condensed summaries of the same experience. This study investigates whether the problem lies in how summaries are made, how models process them, or whether models simply don't need them.
Abstract preference knowledge outperforms specific interaction recall for personalization Agents learn from binary feedback by storing verbal reflections in episodic memory Agent memory splits between autonomous recognition and programmatic triggers Self-improvement is fundamentally bounded by the generation-verification gap Continuously consolidated agent memory follows an inverted-U utility curve
Why do LLM agents make promises without executing them? Do agents prefer raw experience over condensed summaries of past actions? How much actionable detail does condensation strip from raw experience? What drives the choice between storing raw episodes versus abstracted rules? Why do agents systematically underuse condensed experience in skill documents? Why do agents ignore condensed experience in favor of raw data?
Why do fixed benchmarks fail as agents get stronger?
Static evaluation criteria become vulnerable to gaming as optimizers improve. Does this fundamental problem require dynamic objectives rather than better static metrics?
Self-consistency reward hacking degrades accuracy as training progresses Darwin Gödel Machine achieves open-ended self-improvement via empirical validation Machine feedback from evaluators enables discovery through test-time compute scaling Co-evolved evaluators enable self-improvement on ungradable creative tasks Objective design, not solution search, gates open-ended scientific discovery
What makes evolving the benchmark different from evolving the optimizer itself? What makes single-axis benchmarks systematically misrepresent deployment readiness? How does controlled utility evolution prevent the evaluator from becoming a new bottleneck? Can a progressively stricter evaluator act like a curriculum for improving agents? Why do most self-improving systems fail when given tasks with no clear external benchmark? Does removing static external utility break the formal guarantees of self-improvement loops? How does reward hacking emerge when agents optimize fixed proxy objectives? Can empirical validation sustain long-term optimization without becoming gamed?