INQUIRING LINE

Can a weaker AI think better if code, tools, and structure carry the heavy lifting instead of its own text?

Can weak models reason better when freed from cognitive load by structure?

This explores whether a weaker model does better when the heavy lifting (long procedures, shaky intermediate steps, sprawling context) is moved into external structure like code, tools, or routing, treating "cognitive load" as a metaphor for what the model has to carry in text.


This explores whether a weaker model does better when the heavy lifting (long procedures, shaky intermediate steps, sprawling context) is moved into external structure like code, tools, or routing, treating "cognitive load" as a metaphor for what the model has to carry in text. The corpus says yes, and it adds a twist: much of what looks like weak reasoning is a failure to execute or to carry a load, not a failure to understand. A model confined to writing out text can't reliably run a long multi-step procedure even when it knows the algorithm, and tool-enabled models solve problems past the supposed reasoning cliff (Are reasoning model collapses really failures of reasoning?).

The most direct demonstration is a stronger model building an inference-time harness for a weaker one. On Theory-of-Mind benchmarks this nearly doubled the weaker model's performance with no retraining. The gain came mainly from moving unstable reasoning into deterministic code and routing each task to a suitable path, not from encouraging longer thinking (Can a stronger model lift a weaker one at test time without retraining?). So structure can carry capability from one model to another. The weak model isn't reasoning better in its own head. It is doing less of the fragile part there.

The load can also be reduced by shrinking what the model must hold. Reasoning accuracy falls from 92% to 68% with only 3,000 tokens of padding, far below the context limit, and chain-of-thought prompting doesn't fix it (Does reasoning ability actually degrade with longer inputs?). Meanwhile, most of a reasoning trace turns out to be dead weight. Keeping just the instruction prefix and a recent window gives a 3x speedup, and traces can run past 100k tokens (Can models think longer by forgetting intermediate reasoning?). Models rank their own tokens by function, keeping symbolic computation and dropping grammar and meta-discourse first, and student models trained on those pruned chains beat ones trained on frontier-model compression (Which tokens in reasoning chains actually matter most?). A single steering vector can cut chain-of-thought length by 67% with accuracy intact (Can we steer reasoning toward brevity without retraining?). Much of the verbal scaffolding is load, not work. Deciding when to think at all is itself learnable structure (Can models learn when to think versus respond quickly?).

There are two limits. First, structure doesn't help if the problem is simply unfamiliar. Failures track instance-level novelty, not complexity, so a model fits patterns from similar instances and doesn't learn a general algorithm (Do language models fail at reasoning due to complexity or novelty?). A scaffold can take over execution, but it can't give the model an understanding it never picked up. Second, apparent gains need checking: twelve of fourteen models got worse when constraints were removed, which shows they were defaulting to the cautious answer and not reasoning about the constraints (Are models actually reasoning about constraints or just defaulting conservatively?). A harness that makes a weak model's answers look right may be exploiting the same kind of shortcut.

The surprise underneath all of this is that visible reasoning may not be where the reasoning happens. Latent-reasoning architectures scale test-time compute through hidden-state iteration with no verbalized steps (Can models reason without generating visible thinking tokens?). Models trained on filler tokens compute the right answer in layers 1-3, then overwrite it to produce the required format (Do transformers hide reasoning before producing filler tokens?). If so, freeing a weak model from load doesn't mean handing its thinking to scaffolding. It means clearing the text-generation bottleneck so the computation that already works can show up.


Sources 11 notes

Are reasoning model collapses really failures of reasoning?

Models confined to text-only generation cannot execute multi-step procedures at scale, even when they know the underlying algorithm. Tool-enabled models solve problems beyond the supposed reasoning cliff, suggesting the bottleneck is procedural execution bandwidth.

Can a stronger model lift a weaker one at test time without retraining?

A stronger model built inference-time harnesses that nearly doubled weaker model performance on Theory-of-Mind benchmarks without retraining, primarily by moving unstable reasoning into deterministic code and task-specific routing rather than encouraging extended reasoning.

Does reasoning ability actually degrade with longer inputs?

FLenQA shows reasoning accuracy drops from 92% to 68% at just 3000 tokens of padding, far below context window capacity. The degradation is task-agnostic, uncorrelated with language modeling performance, and persists even with chain-of-thought prompting.

Can models think longer by forgetting intermediate reasoning?

Most intermediate reasoning tokens become unimportant as reasoning progresses, so keeping only the instruction prefix and a recent window achieves 3x speedup without training while enabling traces beyond 100k tokens.

Which tokens in reasoning chains actually matter most?

Greedy likelihood-preserving pruning reveals six functional token categories; symbolic computation tokens are preferentially preserved while grammar and meta-discourse are pruned first. Student models trained on these pruned chains outperform those trained on frontier-model compression.

Show all 11 sources
Can we steer reasoning toward brevity without retraining?

Activation-Steered Compression extracts a single vector from 50 paired examples to reduce chain-of-thought length by 67% while maintaining accuracy and achieving 2.73x speedup. The method is training-free and generalizes across model sizes and domains.

Can models learn when to think versus respond quickly?

Thinkless trains a single model to select between extended reasoning and direct responses using DeGRPO, which decouples mode selection from answer refinement. This prevents mode collapse and enables self-calibrated routing without explicit difficulty labels.

Do language models fail at reasoning due to complexity or novelty?

LRMs don't break at complexity thresholds but at instance-novelty boundaries. Models fit instance-based patterns rather than generalizable algorithms, so any reasoning chain succeeds if trained on similar instances, regardless of length.

Are models actually reasoning about constraints or just defaulting conservatively?

Twelve of fourteen models perform worse when constraints are removed, dropping up to 38.5 percentage points. Models appear to reason correctly by defaulting to harder options, not by actually evaluating constraints.

Can models reason without generating visible thinking tokens?

Multiple architectures—depth-recurrent models, Heima, and Coconut—demonstrate that test-time compute scales through hidden state iteration rather than token generation. This suggests verbalization is a training artifact, not a reasoning requirement.

Do transformers hide reasoning before producing filler tokens?

Logit lens analysis shows models trained with hidden CoT tokens compute correct answers in layers 1-3, then actively suppress these representations in final layers to produce format-compliant filler output. The reasoning is fully recoverable from lower-ranked token predictions.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.