Could an AI agent figure out on its own how hard to compress its memory, without anyone running test trials first?
Can compression aggressiveness be set automatically for new environments without benchmark testing?
This explores whether an AI agent can work out on its own how much to trim its context (how hard to compress its memory of past steps) when it enters an unfamiliar task, rather than someone tuning that setting by trial runs on a benchmark.
This explores whether an agent can decide for itself how hard to compress its working context in a new environment, without someone first running benchmark trials to tune it. The short answer: the corpus doesn't show anyone doing this fully automatically yet. It does pin down what the setting depends on, and that is the first step toward automating it. The key finding from ACON is that the right level of compression depends on the environment, not on how strong the model is Should compression be aggressive or conservative in different environments?. In noisy, verbose settings like AppWorld, aggressive pruning helps because most of what the agent sees is clutter. In high-fidelity settings like OfficeBench, one small detail can decide the outcome, so cutting hard is risky. So the question becomes: can an agent judge how noisy an environment is before it has failed in it?
Today's answer still involves some testing, just much less of it. ACON doesn't retrain the model. It rewrites the plain-language compression instructions by studying cases where compressed context caused a failure, and it costs under $2 per benchmark Can prompt optimization cut agent token usage without retraining?. That is cheap, but it still learns from failures in a specific environment. It is cheap calibration, not zero calibration.
The more promising path to automation comes from prompt compression. LLMLingua doesn't run trials to decide what to cut. It uses a small model's 'surprise' score (perplexity) to judge how informative each token is, and it gives different parts of the prompt different budgets: instructions and questions are protected, while examples are cut harder Can prompts be compressed without losing model performance?. That is a signal measured on the content itself. In principle, a similar per-environment measurement, such as how much of each observation is predictable boilerplate, could stand in for the noise judgment ACON gets from failure analysis. The deeper theory supports this idea. Language modeling and lossless compression are mathematically the same thing, and text-trained models adapt in context into good compressors even for images and audio Can text-trained models compress images better than specialized tools?. In other words, the model's own predictions already contain a rough measure of how compressible, and therefore how noisy, an environment is.
There is also a reason for hope from harness engineering. The StateM execution runbook carried over to newer models without changes Can execution harnesses lift model performance without retuning weights?. That shows scaffolding settings can sometimes generalize instead of needing a fresh tune each time. What nobody in the collection has shown yet is the same kind of transfer across environments. The surprising takeaway is that the obstacle isn't the model at all. It is how fast you can tell whether an environment rewards keeping details or throwing them away.
Sources 5 notes
ACON's guideline optimization shows that aggressive pruning works best in noisy, verbose environments like AppWorld, while conservative retention matters more in high-fidelity tasks like OfficeBench where subtle facts drive decisions. The choice depends on the task's information structure, not the model's capability.
Microsoft's ACON method reduces agent token consumption and improves task success by iteratively refining compression guidelines through failure analysis, not weight updates. This approach works across proprietary LLM agents and costs under $2 per benchmark.
LLMLingua achieves up to 20x prompt compression with only 1.5-point performance drops by allocating less compression to instructions and questions, more to demonstrations, then pruning tokens using a small model's perplexity scoring aligned to the target LLM.
Chinchilla models trained exclusively on text achieve better compression rates on images and audio than FLAC and PNG by using their context window to adapt as task-specific compressors. This demonstrates that generalization operates through compression, not specialization.
StateM improves Terminal-Bench 2.1 accuracy across multiple models by optimizing execution systems around frozen weights. The same runbook transfers to newer models without modification, achieving 95.3% on GPT-5.6 and lifting DeepSeek-V4 Flash by 5.4 points.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- ACON: Optimizing Context Compression for Long-horizon LLM Agents
- LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models
- Thinking as Compression: Your Reasoning Model is Secretly a Context Compressor
- Requential Coding: Pushing the Limits of Model Compression with Self-Generated Training Data
- StateM: Reaching 95.3% Raw Accuracy, or a \$15 Frontier Run, on Terminal-Bench 2.1 via Harness Scaling
- Language Modeling is Compression
- Do LLMs Encode Functional Importance of Reasoning Tokens?
- Inference-Aware Prompt Optimization for Aligning Black-Box Large Language Models