Should AI adapt on the fly to each situation, or just follow pre-set scripts and trained habits?
How does runtime adaptation differ from scripted response design?
This explores the difference between AI systems that adjust their behavior while they're running, in response to the situation in front of them, and systems whose responses are fixed in advance, whether by hand-written scripts or by habits that training locks in.
This explores the difference between AI that adjusts itself in the moment and AI whose responses are set ahead of time. The corpus suggests the line between them isn't really 'scripts vs. no scripts.' The bigger question is where a system's flexibility lives: in the weights, in what surrounds the model, or in its ability to notice that a situation is new. One reason this matters is that How does AI context differ from conventional software context? argues AI breaks the assumption behind scripted design. In traditional software, the context is stable, so designers can map out responses in advance. With AI, the prompt, the history, the retrieved data and the hidden state keep shifting. That calls for designing the context rather than the interface.
You might expect training to be the cure for scripted behavior. Often it causes it. When every attempt at a prompt gets about the same reward, training signals get weak, and models drift toward generic, one-size-fits-all templates that ignore the input. The model has effectively written its own script (Why do language models collapse into generic templates?). Reinforcement learning also tends to settle on a single output format from pretraining and suppress the others (Does RL training collapse format diversity in pretrained models?). Most strikingly, post-trained models get sharper on easy cases but lose rare solutions. On agentic tasks, base models with loose prompts eventually find more solutions as you give them more attempts (Do base models find more solutions than post-trained ones?). Polish can cost you range.
The runtime side of the corpus shows several ways to adapt without retraining. Some systems adapt *inside* the model at the moment of use. Transformer² mixes task-specific 'expert' adjustments on the fly (Can models dynamically activate expert skills at inference time?). Others steer behavior through small add-ons to a frozen model (Can editing hidden representations beat weight updates for finetuning?, Can we control personality in language models without prompting?). Other systems adapt *around* the model. Agents can write down what went wrong after a failure and read those notes on the next try (Can agents learn from failure without updating their weights?). Agents can refine their own compression instructions by studying failures (Can prompt optimization cut agent token usage without retraining?). And improving the execution 'harness' around a frozen model can raise its scores in ways that carry over to newer models (Can execution harnesses lift model performance without retuning weights?).
The most useful idea may be that the best systems don't choose between the two approaches. They switch between them. A dialogue planner can use a fast learned policy, the closest thing to a script, in familiar situations and switch to slow, deliberate search only when its own uncertainty is high (Can dialogue planning balance fast responses with strategic depth?). Thinkless trains a model to decide for itself when to reason at length and when to just answer (Can models learn when to think versus respond quickly?). In practice, then, runtime adaptation often means knowing when to stop following the script.
Sources 12 notes
AI interactions operate on a substrate of constantly shifting context—prompt, history, retrieved data, hidden state—that users cannot internalize like traditional UIs. This structural mutability demands a new design discipline centered on context engineering rather than interface design.
When within-prompt reward variance is low, task gradients weaken and regularization dominates, pushing policies toward generic outputs. SNR-Aware Filtering—selecting high-variance prompts before updates—recovers performance across tasks and scales.
Controlled experiments show RL consistently amplifies one format distribution from pretraining within the first epoch while collapsing alternatives. The winning format depends on model scale, not necessarily performance, and is largely hidden when starting from proprietary pretrained models.
Across 14 model pairs and three agentic benchmarks, base models equipped with only relaxed system prompts eventually surpass post-trained counterparts in pass@K coverage as rollout budget grows. Post-training bimodalizes task outcomes, sharpening performance on easy cases while eliminating rare-but-reachable solutions.
Transformer2 demonstrates that tuning only singular values within weight matrices produces composable expert vectors that dynamically mix at inference without interference, outperforming LoRA with fewer parameters and enabling continual specialization.
Show all 12 sources
ReFT learns task-specific interventions on frozen model representations rather than updating weights, with LoReFT (low-rank linear subspace variant) dramatically outperforming LoRA across reasoning, instruction-following, and NLU benchmarks while using far fewer parameters.
PsychAdapter modifies every transformer layer with <0.1% additional parameters to achieve 87.3% Big Five accuracy and 96.7% depression/life satisfaction accuracy across GPT-2, Gemma, and Llama 3. This architecture-level approach bypasses prompt resistance entirely.
Reflexion demonstrates that unambiguous environmental feedback (success/failure) enables agents to write useful self-diagnoses and improve across episodes without parameter updates. The binary signal prevents rationalization, and keeping reflections uncompressed preserves their usability.
Microsoft's ACON method reduces agent token consumption and improves task success by iteratively refining compression guidelines through failure analysis, not weight updates. This approach works across proprietary LLM agents and costs under $2 per benchmark.
StateM improves Terminal-Bench 2.1 accuracy across multiple models by optimizing execution systems around frozen weights. The same runbook transfers to newer models without modification, achieving 95.3% on GPT-5.6 and lifting DeepSeek-V4 Flash by 5.4 points.
A framework combining a neural policy model (System 1) for familiar contexts with MCTS planning (System 2) for novel scenarios, switching based on the model's own uncertainty estimates, matches or exceeds pure MCTS performance while reducing computational cost.
Thinkless trains a single model to select between extended reasoning and direct responses using DeGRPO, which decouples mode selection from answer refinement. This prevents mode collapse and enables self-calibrated routing without explicit difficulty labels.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Sharpening Tax in Post-Training
- Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretraining
- Mechanistically Interpreting the Role of Sample Difficulty in RLVR for LLMs
- Planning Like Human: A Dual-process Framework for Dialogue Planning
- Thinkless: LLM Learns When to Think
- PsychAdapter: Adapting LLM Transformers to Reflect Traits, Personality and Mental Health
- ACON: Optimizing Context Compression for Long-horizon LLM Agents
- ReFT: Representation Finetuning for Language Models