Should 'memory' and 'perception' be required parts of an AI agent, or can they just emerge from how it behaves?

This explores why perception and memory might be better understood as abilities that show up inside an agent's behavior, or in the world around it, than as required modules you have to bolt on. The corpus points to a surprising answer: when researchers look closely, memory and perception keep turning up in places nobody designed them to be. That makes a checklist definition ("an agent must have a memory component and a perception component") look less like a description of how capable systems work and more like a description of how engineers happen to build them.

Start with memory. In one striking result, reinforcement learning agents that were never given a memory objective learned to use their physical environment as memory anyway. The paths and marks they left behind reduced how much history the agent had to keep in its head, and this came purely from ordinary reward optimization Do RL agents accidentally use environments as memory?. If memory can live in the world instead of the agent, then requiring it as an internal trait misses where it actually sits. A related line of work goes further and treats memory as what thinking runs on, not a store that thinking looks things up in: intelligence as reusing past reasoning paths instead of recomputing from scratch Can cognition work by reusing memory instead of recomputing?.

Engineering points the same way. Bolting an external memory module onto a model tends to fail because the module and the model get optimized separately and drift apart. Memory foundation models answer this by putting memory inside the model's backbone as a native state, trained end to end Should agent memory live inside the model backbone?. At the other extreme, Reflexion agents improve across attempts without any weight updates, just by writing plain-language reflections into an episodic log Can agents learn from failure without updating their weights?. Memory can be very different things in different systems, so asking whether an agent "has" it is less useful than asking where it lives and how it gets used.

Perception follows a similar pattern. Language models predict how appropriate people would find social situations better than nearly all individual humans, with no bodies and no direct sensing of the world Can AI systems learn social norms without embodied experience?. Yet all of the models make the same systematic mistakes, which hints at where experience gained through a body might still matter. In multimodal models, perception also turns out to be something you can damage by making it explicit: long chain-of-thought explanations hurt fine-grained visual tasks, because the real bottleneck is where the model directs its visual attention, not what it says out loud Does verbose chain-of-thought actually help multimodal perception tasks?. The explicit-versus-implicit split also shows up at the mechanism level. Models can quietly recognize their own outputs through one internal pathway and report authorship when asked through a completely separate one Do explicit and implicit self-recognition use the same mechanism?.

The common thread is that abilities often exist before anyone names them. Base models already carry reasoning ability that training draws out more than creates Do base models already contain hidden reasoning ability?. Perception and memory look similar: these are capacities to detect and draw out, not boxes to tick. Treating them as implicit keeps you from ruling out systems that have them in unexpected forms, and from crediting systems that have the module but don't really use it. A caveat: the corpus doesn't hold a single paper arguing directly about how agents should be defined. This synthesis is assembled from evidence across memory, perception, and interpretability research.