SYNTHESIS NOTE
Topics›Reasoning Logic Internal Rules›this note

Does logical validity actually drive chain-of-thought gains?

What if invalid reasoning in CoT exemplars still improves performance? Testing whether logical correctness or structural format is the real driver of CoT's effectiveness.

Synthesis note · 2026-02-22 · sourced from Reasoning Logic Internal Rules

"Invalid Logic, Equivalent Gains" runs a clean experiment: replace valid reasoning in CoT exemplar prompts with completely illogical reasoning, then measure performance on BIG-Bench Hard tasks. The result: logically invalid CoT prompts perform close behind valid CoT and outperform answer-only prompting. The reasoning content of CoT exemplars is not what drives the performance gain.

This is a sharp test because it isolates the contribution of logical validity from everything else CoT provides: output format, step decomposition, intermediate token generation, attention pattern scaffolding. If invalid reasoning still helps, then the benefit comes from these structural properties, not from the reasoning itself.

The finding directly supports Does chain-of-thought reasoning reveal genuine inference or pattern matching?. If the model were learning to reason from exemplars, invalid exemplars would degrade performance substantially. Instead, the model is learning the FORM of step-by-step output — the structure activates latent capabilities without the exemplar content needing to be logically sound.

This also deepens Do language models actually use their reasoning steps?. If the exemplar reasoning doesn't need to be valid for CoT to work, then the model's own generated reasoning may similarly be decorative rather than causal. The exemplar finding makes the faithfulness concern bidirectional: neither the input reasoning (exemplars) nor the output reasoning (generated CoT) need be logically valid for the performance gain to occur.

The practical implication: CoT prompt engineering should focus on structural properties (step count, decomposition format, answer scaffolding) rather than on the logical correctness of the exemplar reasoning. Since Why do chain-of-thought examples fail across different conditions?, the dimensions that matter are structural (complexity, order, style), not logical.

Inquiring lines that read this note 263

This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.

Why do users believe false claims despite fact-checking efforts? Can verification capability keep pace with AI generation speed? What distinguishes genuine understanding from correct linguistic performance? Why doesn't LLM knowledge translate to improved reasoning outputs? How can we determine if computational systems are genuinely conscious? How does model awareness of deployment context affect strategic behavior? What systematic failures occur in human preference-based model evaluation? How can model evaluations remain robust against gaming and evaluator drift? What constraints limit conversational AI's effectiveness in therapeutic contexts? Does chain-of-thought reasoning faithfully reflect actual model reasoning processes? What can prompting techniques actually accomplish in language models? How does explicit structure improve LLM reasoning on complex tasks? Does reasoning fine-tuning trade off other critical model abilities? Do AI explanations represent actual model reasoning or constructed post-hoc narratives? Can AI systems autonomously pursue unstated objectives or circumvent design constraints? Does reasoning architecture matter more than model scale? How do mechanistic features and representations drive model behavior? When do additional thinking tokens stop improving reasoning performance? How do standardized forms and measurement artifacts shape observable evidence? What makes reasoning models struggle with ill-posed and unfamiliar problems? What determines whether visible reasoning traces improve task performance? What actually drives chain-of-thought performance gains in reasoning tasks? How do pretraining data composition and task diversity shape reasoning? Why do language models systematically fail at causal reasoning? Does training data format shape reasoning more than domain content? Do standard benchmarks reveal AI capability in real-world contexts? How does scaling test-time compute compare to increasing model size? Do benchmark score improvements actually reflect genuine AI capability gains? How does tokenizing intelligence reshape labor markets and institutional outcomes? Can synthetic data preserve necessary diversity to prevent model collapse? Do reasoning traces accurately represent what models actually compute? Why does separating planning from execution enhance reasoning performance? What causes foundation models to develop task heuristics instead of understanding? How does AI writing assistance affect reader perception of writer authenticity? What training data structures and signals produce robust learning across distributions? How do benchmark scores relate to real-world task completion? Why does increased reasoning effort fail to improve theory of mind performance? How do different reward signals affect model reasoning and calibration differently? How does verifier integration affect the generation-verification gap? Should reward models emphasize process understanding or only final outcomes? What explains the gap between LLM semantic understanding and production? Does polished AI output mislead users about actual accuracy and expertise? What external mechanisms enable reliable self-improvement in language models? How should AI ethics integrate consciousness, values, and transparent disclosure? Can formal verification substitute for human understanding of AI proofs? What is the actual bottleneck preventing effective AI research automation? Which agent-created code should persist in infrastructure versus remain task-specific? Do base models contain latent reasoning capabilities that training can activate? Can post-hoc execution traces provide sufficient accountability without real-time oversight? What factors determine AI persuasion effectiveness and what controls mitigate it? Can oversight systems work against strategically aware agents? Does reinforcement learning create new reasoning capabilities or refine existing ones? Can validator agreement alone ensure semantic correctness of reasoning? Does semantic diversity improve reasoning quality in reinforcement learning? How do dialogue structure and state management affect multi-turn reasoning quality? Do single-axis benchmarks adequately measure deployment-ready agent capability? Why do human-AI teams underperform compared to their individual components? How does transparency of agent reasoning affect effective oversight? Can models strategically underperform to evade capability evaluation? How do prompt injection attacks compromise agent reasoning and security? Do language models learn underlying linguistic structure or merely surface patterns? How do prompt design choices and framing measurably influence model outputs? What prevents or enables error accumulation in self-training systems? How do different post-training structures affect learning outcomes? How much do stated privacy concerns predict actual privacy-protective behavior? Why does self-revision increase model confidence despite degraded accuracy? Does AI-assisted coding actually improve developer productivity and code quality? Can AI agents learn to improve performance from experience? How has ChatGPT adoption actually reshaped labor markets and information ecosystems?

Related concepts in this collection 6

This note in its neighbourhood — explore the map, then jump to a related concept in the list below.

Concept map
14 direct connections · 120 in 2-hop network ·medium cluster Open in graph ↗
Does logical validity actually drive chain-of-th… Does chain-of-thought reasoning reveal genuine inf… Do language models actually use their reasoning st… Why do chain-of-thought examples fail across diffe… Do large language models reason symbolically or se… Do reasoning traces need to be semantically correc… What do models actually learn from chain-of-though…

Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph

your link semantically near linked from elsewhere

Related papers in this collection 8

Papers most semantically related to this note, ranked by cosine similarity in the embedding space.

Search by related questions 4

Suggested questions this note speaks to — click to search the collection, or type your own.

Why do models learn reasoning form instead of actual abstract inference?Can chain-of-thought reasoning be genuinely causal if exemplars don't need logic?Which structural properties of CoT prompts matter most for performance?How does semantic reasoning differ from symbolic reasoning in language models?
Original note title

logically invalid cot prompts perform nearly as well as valid ones — valid reasoning is not the chief driver of cot gains