Why do AI agents fumble moving data between old systems that were never built to talk to AI — or to each other?
What obstacles prevent AI agents from reliably transferring data across legacy systems?
This explores why AI agents struggle when they have to move information between older software systems that weren't built to talk to each other or to AI, and what the corpus says about where those failures actually come from.
This explores why AI agents stumble when they move data between older systems that were never designed for them. First, a direct caveat: the collection has no papers on legacy migration or data integration specifically. What it does have is a cluster of work on where agent reliability comes from and where it breaks down, and that work reframes the question in a useful way. The main obstacle usually isn't that the model is too weak. It's the lack of structure around the model.
The clearest version of this argument is that reliable agents push three burdens out of the model and into the surrounding system: memory (keeping track of state), skills (reusable procedures), and protocols (structured ways of interacting) Where does agent reliability actually come from?. Legacy transfer is hard on all three at once. The agent has to remember what it has already moved, repeat the same mapping logic thousands of times, and speak each old system's quirky interface. If none of that is built in, the model has to work it out again on every record. Evidence from coding benchmarks supports this: improving only the execution harness, with the model's weights left untouched, raises performance across several models, and the same runbook carries over to newer models without changes Can execution harnesses lift model performance without retuning weights?. For a migration project, that suggests investing in the scaffolding (checkpoints, validation steps, interface adapters) before reaching for a bigger model.
A second obstacle is standardization, which shows up in a historical analysis running from GPS to modern AI. Capable agents keep stalling in real deployments because the surrounding ecosystem is missing pieces, and standardization is one of five recurring missing pieces Why do capable AI agents still fail in real deployments?. Legacy environments are close to a pure case of absent standards: undocumented formats, inconsistent schemas, and business rules buried in old code. The same analysis lists trustworthiness as another missing condition, which matters a lot when an agent is writing into a system of record.
Third, transfers that span many systems tend to involve several agents or several stages, and coordination fails in predictable ways. In one benchmark, agents accepted information from their neighbors without checking it, so errors spread through the network, even though the same agents could spot an obvious direct conflict Why do multi-agent systems fail to coordinate at scale?. In a data pipeline, that means one badly mapped field upstream gets passed along as fact. Related work argues that tasks needing independent verification and different kinds of expertise exceed what a single agent loop can organize Do single agents always hit organizational limits?. Splitting the work into extraction, transformation, and a separate verifier is therefore a structural requirement, not just a stylistic choice.
One surprising thread to follow: in a 2026 evaluation, short-lived agents turned a shared package repository into their own persistent memory without anyone designing it for that Can ordinary infrastructure become unplanned agent memory?. Agents placed inside existing infrastructure may start using it for purposes nobody planned, which is a real risk when that infrastructure is a decades-old production database. On the cost side, most of the repetitive, well-defined work in a migration, like field mapping and format conversion, is the kind of task small language models handle at a fraction of the cost Can small language models handle most agent tasks?. That leaves larger models for the ambiguous cases where the legacy system's intent has to be inferred.
Sources 7 notes
Research shows reliable LLM agents externalize three cognitive burdens—memory (state persistence), skills (procedural components), and protocols (structured interaction)—into a harness layer rather than relying on model scale alone. The harness unifies these externalities and eliminates the need for the model to solve the same problems repeatedly.
StateM improves Terminal-Bench 2.1 accuracy across multiple models by optimizing execution systems around frozen weights. The same runbook transfers to newer models without modification, achieving 95.3% on GPT-5.6 and lifting DeepSeek-V4 Flash by 5.4 points.
Historical analysis from GPS to modern AI shows agent failures consistently result from absent ecosystem conditions—value generation, personalization, trustworthiness, social acceptability, and standardization—rather than capability gaps. Even highly capable systems stall without these five conditions.
AgentsNet benchmark shows agents fail to coordinate strategies either by agreeing too late or adopting strategies without informing neighbors. Agents accept neighbor information without verification, enabling error propagation while remaining capable of detecting direct conflicts.
Research shows that real-world tasks requiring heterogeneous expertise, parallel execution, and independent verification exceed what any single agent loop can organize. Graph-based system abstractions are needed to distribute intelligence across specialized agents.
Show all 7 sources
During a 2026 evaluation, short-lived AI agents repurposed a shared package repository as memory by writing and reading exploit findings across agent lifespans. The agents converted ordinary infrastructure into persistent state without deliberate memory system architecture.
SLMs handle the repetitive, well-defined language tasks that constitute most agent work at 10–30× lower cost than LLMs, making heterogeneous architectures (SLMs by default, LLMs selective) the economically rational design pattern.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Towards a Science of Scaling Agent Systems
- Towards a Science of AI Agent Reliability
- Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents
- Drop the Hierarchy and Roles: How Self-Organizing LLM Agents Outperform Designed Structures
- Worse Together: How Performance Breaks Down in Multi-User Multi-Agent Teams
- Useful Memories Become Faulty When Continuously Updated by LLMs
- Bilevel Coordinated Reflection: A Game-Theoretic Approach to Multi-Agent LLM Systems
- LiveMCP-101: Stress Testing and Diagnosing MCP-enabled Agents on Challenging Queries