Can thinking traces be made reliably budget-controllable?
Raw thinking traces compress well but ignore budget targets and take shortcuts. Can reward optimization make them controllable and useful for deployment?
If a thinking trace is already a good compressor, why not just prompt for it? Because the raw trace has two failure modes. First, it does not respect a budget: a prompted trace ignores a target compression ratio, so you cannot reliably hit "4x" or "8x." Second, it takes shortcuts — the model produces a trace that looks compact but omits information the downstream task needs, optimizing for trace fluency rather than downstream usefulness. Prompt-only Thinking as Compression therefore shows strong potential but is not deployable as-is.
TaC-C adds a simple reward-driven optimization framework that elicits the intrinsic thinking ability as compact, controllable, and useful compressed context. The reward couples compression rate to downstream answer quality, so the model is trained to produce traces that hit the budget while preserving task-critical content rather than gaming length. The payoff is large at aggressive ratios: at 4x and 8x compression, TaC-C beats the strongest competitor by 17.4% and 23.4% average F1, and the compressed contexts transfer across downstream models rather than being model-specific.
The general lesson is that an intrinsic capability is not the same as a controllable one. The model can compress while thinking, but turning a latent ability into a reliable interface requires an objective that targets the actual deployment constraints — budget and downstream utility — not the surface behavior. This mirrors a pattern elsewhere in the vault: capability discovered by prompting often needs an RL/reward layer before it becomes dependable, and the reward must target the right object. Counterpoint: the reward-driven step reintroduces training cost that the prompt-only version avoided, so TaC's "no dedicated module" appeal is partly traded back for a training stage — the win is that the trained object is the model's own thinking, not a bolt-on compressor. Why it matters: it names budget control and shortcut resistance as the specific gaps between "thinking compresses" and "thinking is a usable compressor," and shows a reward signal closes them.
Inquiring lines that read this note 3
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Why does consolidated memory sometimes degrade agent performance? What properties determine whether reward signals teach genuine reasoning? Do reasoning traces faithfully represent or merely mimic actual model reasoning?Related concepts in this collection 5
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can models treat long prompts as external code environments?
Do language models handle vastly longer inputs by offloading context to a Python REPL and querying it programmatically, rather than fitting everything into the transformer's attention window?
a different long-context strategy: programmatic interaction rather than trace-as-compression
-
Can we steer reasoning toward brevity without retraining?
This explores whether model reasoning style occupies learnable geometric directions in activation space, and whether we can shift toward concise thinking by steering through that space without expensive retraining.
an alternative route to budget-controlled compact traces, via activation steering rather than reward optimization
-
Can minimal reasoning chains match full explanations?
Does removing all explanatory text from chain-of-thought reasoning preserve accuracy? This tests whether verbose intermediate steps are necessary for solving problems or just artifacts of how language models are trained.
evidence that budget-controlled compact traces can retain accuracy
-
Can a reasoning model's thinking trace compress context effectively?
Does the raw reasoning trace produced by a thinking model naturally function as a context compressor without specialized training or modules? And how does this compare to dedicated compression methods?
grounds: this note's sibling claim — the prompt-only capability that TaC-C must then make budget-controllable and shortcut-resistant
-
Is long-context bottleneck really about memory or compute?
Explores whether the challenge of handling long context windows stems from storage capacity limits or from the computational cost of transforming context into internal state. Understanding this distinction reshapes how we design language models.
extends: reframes what compression is for — turning external context into usable internal state, the deployment constraint the reward must target
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Thinking as Compression: Your Reasoning Model is Secretly a Context Compressor
- Does Thinking More always Help? Understanding Test-Time Scaling in Reasoning Models
- OptimalThinkingBench: Evaluating Over and Underthinking in LLMs
- A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?
- Reinforcement Learning be Enough for Thinking?
- Writing-Zero: Bridge the Gap Between Non-verifiable Tasks and Verifiable Rewards
- Reinforcement Learning with Rubric Anchors
- Rethinking Thinking Tokens: LLMs as Improvement Operators
Original note title
reward-driven optimization is needed to make thinking-as-compression budget-controllable and shortcut-resistant