Why can an AI agent that acts and learns from mistakes still be boxed in by the examples it was trained on?
What distinguishes bounded classification models from open-ended agentic systems?
This explores what changes when you move from a model that picks an answer from a fixed set of options to a system that has to act, try things, and recover across many open-ended steps. The corpus doesn't compare the two head-on; it gets at the difference through papers on agent learning, agent limits, and what makes a task well-defined.
This explores what changes when you move from a model that picks an answer from a fixed set of options to a system that has to act, try things, and recover across many open-ended steps. The collection has no paper that puts classifiers and agents side by side. It does show where the line between them falls, and that line runs through agents too, not only between agents and everything else.
The first difference is how far the output can go. A classifier's possible answers are fixed before it starts. An agent's are not, but it is still held back by what it was trained on. Agents trained only on expert examples are limited by what the people who built the dataset thought to include. Because they never interacted with an environment during training, they can't learn from their own mistakes (Can agents learn beyond what their training data shows?). A more surprising result: post-training can quietly turn an open-ended system into a narrower one. Base models with only a simple prompt eventually find more correct solutions than their post-trained versions once they get enough attempts. Post-training sharpens performance on easy cases and drops rare solutions that were still reachable (Do base models find more solutions than post-trained ones?). A related effect appears when the training signal barely varies across attempts: models settle into generic template answers that ignore the input (Why do language models collapse into generic templates?).
The second difference is where learning happens. A classifier learns once, during training. Open-ended agents can keep learning while they run, without changing their weights. Reflexion agents write notes about their own failures and reuse them in later attempts. This works because a clear pass/fail signal from the environment stops them from talking themselves out of the failure (Can agents learn from failure without updating their weights?). AgentFly extends this with structured memories of past cases, subtasks, and tool use, and reaches strong results on the GAIA agent benchmark without any retraining (Can agents learn continuously from experience without updating weights?). Another route is to give agents a learned simulator of their environment, so they can practice far beyond what real environments allow (Can language models learn to simulate agent environments?).
The third point is the one most people miss: a well-built agent is mostly made of narrow, well-defined pieces. Most agent work is repetitive, well-defined language tasks that small models handle at 10–30× lower cost. The sensible design uses small models by default and calls large models only when needed (Can small language models handle most agent tasks?). The open-endedness comes from how those pieces are organized, not from each step. That organization has limits of its own. Tasks that need different kinds of expertise, parallel work, and independent checking go beyond what a single agent loop can manage (Do single agents always hit organizational limits?). Self-improvement is also capped by how well a system can check its own outputs (What limits autonomous capability in large language models?).
One lateral thread is worth following if you want a more formal way to think about 'bounded.' Epiplexity is a measure of how much learnable structure a model with limited computing power can actually extract from data. It tracks how well models generalize to unfamiliar data (What can a bounded observer actually learn from data?). Seen this way, the real question is less 'classifier or agent' and more whether the system can keep finding new structure or only reuses structure it already learned.
Sources 10 notes
Agents trained on static expert datasets cannot learn from their own failures or generalize beyond demonstrated scenarios because they never interact with environments during training. Competence is capped by what curators imagined, not by agent capacity.
Across 14 model pairs and three agentic benchmarks, base models equipped with only relaxed system prompts eventually surpass post-trained counterparts in pass@K coverage as rollout budget grows. Post-training bimodalizes task outcomes, sharpening performance on easy cases while eliminating rare-but-reachable solutions.
When within-prompt reward variance is low, task gradients weaken and regularization dominates, pushing policies toward generic outputs. SNR-Aware Filtering—selecting high-variance prompts before updates—recovers performance across tasks and scales.
Reflexion demonstrates that unambiguous environmental feedback (success/failure) enables agents to write useful self-diagnoses and improve across episodes without parameter updates. The binary signal prevents rationalization, and keeping reflections uncompressed preserves their usability.
AgentFly formalizes agent learning as a Memory-augmented MDP with three memory modules (case, subtask, tool) that enable credit assignment and policy improvement entirely through memory operations. The approach achieved 87.88% on GAIA validation without modifying LLM parameters.
Show all 10 sources
Qwen-AgentWorld demonstrates that native language world models trained via next-state prediction on 10M+ trajectories outperform real-environment training on three benchmarks and transfer across seven domains, positioning next-state prediction as a foundation objective for agents.
SLMs handle the repetitive, well-defined language tasks that constitute most agent work at 10–30× lower cost than LLMs, making heterogeneous architectures (SLMs by default, LLMs selective) the economically rational design pattern.
Research shows that real-world tasks requiring heterogeneous expertise, parallel execution, and independent verification exceed what any single agent loop can organize. Graph-based system abstractions are needed to distribute intelligence across specialized agents.
Multi-agent deliberation produces specific failure modes (Degeneration-of-Thought, Silent Agreement), alignment at scale includes problematic self-valuation, and self-improvement is formally bounded by the generation-verification gap. Measurement error and conditional compliance hide the true capability ceiling.
Epiplexity formalizes the structural information a computationally bounded observer can extract from data, separating learnable regularity from time-bounded entropy. This task-free measure correlates with out-of-distribution generalization and explains why some datasets enable broader transfer than others.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Agent Learning via Early Experience
- AgentFly: Fine-tuning LLM Agents without Fine-tuning LLMs
- Useful Memories Become Faulty When Continuously Updated by LLMs
- Small Language Models are the Future of Agentic AI
- Towards a Science of Scaling Agent Systems
- Scaling Behavior of Single LLM-Driven Multi-Agent Systems
- Drop the Hierarchy and Roles: How Self-Organizing LLM Agents Outperform Designed Structures
- Survey on Evaluation of LLM-based Agents