TOPIC
Task Planning
A subject the collection covers, read through 10 synthesis notes.
View as
Can delegation teach models to manage context more actively?
Does training models to decompose tasks and delegate to subagents—rather than passively compressing when context fills up—improve their ability to reason over long horizons? And does this skill transfer to single-agent work?
Eleven task axes determine delegation design and capability matching Agent memory quality depends on curation, not storage capacity alone Frontier LLMs corrupt 25 percent of document content across long delegated workflows Agents need an intermediate interface between planning and grounding layers. Multi-agent capability ceilings are structural, not scalable capability gains.
Can agent skills move from prompts to trainable parameters? When and what should a model actually decide to delegate? Why does delegation training help models that work alone? What makes passive prompt transfer fail as a substitute for auditable expertise? Can agents teach each other skills without human supervision?
Can command generation replace intent classification in dialogue systems?
Explores whether generating pragmatic commands in a DSL could outperform traditional intent classification for task-oriented dialogue, particularly regarding training data needs and scalability.
When should AI agents ask users instead of just searching? Why can't conversational AI agents take the initiative? Can dialogue planning balance fast responses with strategic depth? Why do protocol-based tool integrations fail in production workflows? What makes conversational recommenders hard to build well?
Insert-expansions from conversation analysis formalize when agents should probe users Conversational AI agents are structurally passive and reactive by design Dual-process dialogue planning balances efficiency with strategic depth through uncertainty switching Direct function calls outperform protocol-mediated tool access in production workflows. Conversational recommenders need mixed-initiative dialogue, not just language fluency
Can better AI interfaces eliminate the attention cost of prompt composition and evaluation? Why does dialogue-shaped text fail to produce dialogue-like operations in practice? Can systems guide users adaptively without imposing predetermined dialogue structures? Can real-time detection identify when users have incomplete or underdeveloped intent? What makes intent taxonomies unmanageable at hundreds of intents? How do probabilistic dialogue systems handle ASR errors differently? Can generative interfaces help users articulate what they actually want? What types of tasks benefit most from dynamically generated interfaces? How does API-first interaction compare to generative interface approaches? Can prompt engineering overcome the gulf between user intent and AI interpretation? How should task-oriented and socially-oriented dialogue acts receive different training signals? Why do traditional interfaces bypass the intention formation problem that language models expose?See all 20 inquiring lines on this note →
Can LLMs actually forecast time series better than we think?
Explores whether language models possess stronger forecasting ability than current benchmarks suggest, and what role workflow design plays in revealing or hiding that capability.
Can decomposing forecasting into stages unlock numerical and contextual reasoning? Do language models make rational strategic decisions in games? Can retrieval-augmented language models forecast like human experts? Can live benchmarks prevent contamination in prediction tasks? Can language models beat human venture capital experts?
Multi-stage decomposition enables forecasting by integrating numerical and contextual reasoning Language models deviate from rational strategies but workflows restore near-optimal play Retrieval-augmented language models forecast near human competitive levels Live contamination-free benchmarks reveal forecasting requires agentic reasoning LLMs surpass human venture capital experts where the expert bar is modest
Why do longer forecasting horizons degrade LLM accuracy in role-play? Why does combining natural language with numerical scores improve prediction accuracy? How does process supervision relate to execution-signaled feedback approaches? Can autoregressive models learn faithful translation to logical representations without semantic loss? Can simple diagnostic tests predict language model performance in production complexity? Why do hybrid paradigms outperform pure autoregressive or pure diffusion approaches? How do general language model benchmarks predict specialized domain performance? Do standard language benchmarks underestimate what LLMs can actually do? Do LLMs need world models to make accurate predictions? Can a model be strong at MMLU but weak at long-horizon tasks? Do monolithic prompts underutilize LLM strengths in forecasting workflows? What separates good workflow design from poor workflow design?See all 32 inquiring lines on this note →
Can large language models actually create executable plans?
Do LLMs genuinely assemble plans that work, or just generate planning-domain knowledge that sounds coherent? Understanding this distinction matters for deploying AI in real planning tasks.
Can language models understand without actually executing correctly? Why do language models fail to act on their own reasoning? Does self-revision actually improve reasoning in language models? Can symbolic solvers fix how LLMs reason about logic? Do foundation models learn world models or task-specific shortcuts?
LLMs show computational split-brain syndrome separating knowledge from execution Language models know what to do but act greedily instead Self-revision degrades reasoning accuracy in o1-like models Symbolic solver integration enables reliable logical reasoning through structured feedback Foundation models develop task-specific heuristics, not generalizable world models.
Why do LLMs generate ideas that sound novel but fail during execution? What distinguishes planning knowledge from an executable plan that works? What explains the 87 percent to 12 percent cliff in plan executability? What components must wrap an LLM to build a working CRS? What interference occurs when planning and synthesis happen in the same component? Why do LLMs understand efficient language but fail to produce it? Does LLM reasoning always match the outputs it generates? What planning tasks benefit most from combining LLM generation with external verification? Can the LLM-Modulo framework extend solver integration to domain planning? Which LLM backends produce the most executable research ideas? Do LLMs need world models to make accurate predictions? How does this differ from using LLMs as the policy itself?
Can decomposing forecasting into stages unlock numerical and contextual reasoning?
This explores whether breaking time-series forecasting into separate stages for contextualization, dual-resolution outlook, and synthesis allows systems to combine the strengths of numerical models and language models more effectively than either alone.
Can LLMs actually forecast time series better than we think? Does separating planning from execution improve reasoning accuracy? Can reasoning systems maintain memory across retrieval cycles? Can retrieval-augmented language models forecast like human experts? How should reasoning systems actually be architected?
Workflow architecture dominates raw model capability in LLM forecasting tasks Separating decomposer from solver improves accuracy and generalizability Stateful memory workspaces enable narrative reasoning across retrieval cycles Retrieval-augmented language models forecast near human competitive levels Reasoning systems should separate activation timing from execution capability.
Why does combining natural language with numerical scores improve prediction accuracy? Why do hybrid paradigms outperform pure autoregressive or pure diffusion approaches? How do ensemble methods apply within a single model? Should time always be a first-class ranking signal in temporally-extended sources? Can structured decomposition fix evaluation gaps in other research tasks? Do monolithic prompts underutilize LLM strengths in forecasting workflows? Why do macro and micro forecasting scales require different reasoning approaches? What real-world forecasting domains benefit most from contextual reasoning integration? How much does workflow architecture matter compared to raw model capability in forecasting? Do newer language model generations improve forecasting ability without additional training? Why are post-cutoff test sets essential for evaluating genuine forecasting ability? What role does retrieval mechanism design play in forecast accuracy?See all 20 inquiring lines on this note →
Does tree depth automatically produce supervision at multiple granularities?
Tree-search rollouts branch at different depths, potentially creating supervision signals ranging from coarse strategy-level to fine-grained detail-level choices. Does this depth variation naturally yield multi-granular process supervision without explicit annotation design?
Can tree structure alone convert outcome rewards into process supervision? Can shared-prefix trees reduce redundancy in agent rollouts? Can trajectory structure replace hand-annotated process rewards? How does RL training reshape reasoning and what gets lost? What actually changes inside a model during RL training?
Tree-search rollouts convert outcome rewards into process supervision automatically Shared-prefix tree rollouts yield more distinct trajectories per token budget Trajectory structure can substitute for separately trained process supervision RLVR activates pretraining strategies rather than teaching new reasoning RL training sparsely updates 5 to 30 percent of parameters through negative reinforcement
How does process supervision relate to execution-signaled feedback approaches? How does nesting optimization levels improve on traditional network depth? Does layer-wise prediction stabilization provide a stronger trace quality signal than confidence alone? What execution feedback signals drive context updates without supervision labels? Can next-state supervision work across different agent interaction types like conversations and tool calls? Does reverse-curriculum learning approximate process supervision using only outcome signals? What makes process-level supervision better than outcome-only rewards for RAG training? What tree depth is achievable before GPU memory becomes the bottleneck? Does self-supervised process supervision work for domains with ambiguous correctness? Can granular function calling tasks learn composition from graph-sampled data? Can trajectory structure alone provide process supervision without human annotation? How does relative progress estimation reduce dependence on hard labels for process supervision?See all 23 inquiring lines on this note →
Can shared-prefix trees reduce redundancy in agent rollouts?
Independent rollouts waste tokens regenerating similar early-turn sequences. Can structuring rollouts as shared-prefix trees instead preserve early computation across samples while maintaining statistical diversity for advantage estimation?
Can tree structure alone convert outcome rewards into process supervision? Does tree depth automatically produce supervision at multiple granularities? Can trajectory structure replace hand-annotated process rewards? How does RL training reshape reasoning and what gets lost? What actually changes inside a model during RL training?
Tree-search rollouts convert outcome rewards into process supervision automatically Tree expansion depth automatically yields multi-granular process supervision signals Trajectory structure can substitute for separately trained process supervision RLVR activates pretraining strategies rather than teaching new reasoning RL training sparsely updates 5 to 30 percent of parameters through negative reinforcement
Does the reversal curse stem from the same one-way commitment architecture? Why do zero-advantage rollouts destabilize training beyond just wasting compute? Why does multi-turn RL generate orders of magnitude more tokens than single-turn? Do multi-agent systems justify their token costs with genuine quality gains? Does upgrading model capability improve token efficiency in agentic systems? Can construction-time routing and runtime agent pruning be combined effectively? How does shared-memory parallelism compare to independent sampling and turn-based debate? How do shared KV caches enable emergent coordination between LLM agents? Which tokens actually change across different reasoning paths in rollouts? What metrics replace throughput per token for agent deployment? How do cache-dominant workflows change the marginal cost of agent tasks? Can two agents with identical token counts produce vastly different outputs?See all 22 inquiring lines on this note →
How much of LLM few-shot ability comes from training data?
Do large language models genuinely learn from a few examples, or are they mostly recognizing patterns from their training data? This matters for understanding what LLMs can actually do.
Does RLVR success on math benchmarks reflect genuine reasoning improvement? Do popular prompting techniques actually improve model performance? Are LLM emergent abilities real or measurement artifacts? Do reasoning traces show how models actually think? How do reasoning models actually break under pressure?
RLVR gains on contaminated benchmarks are primarily memorization, not reasoning Popular prompting techniques fail to replicate under controlled statistical testing Emergent abilities are metric artifacts not fundamental capability jumps Reasoning traces are stylistic mimicry, not verified reasoning. Reasoning models excel at formal tasks but fail at social cognition.
Can tree structure alone convert outcome rewards into process supervision?
Tree-based rollouts naturally create step-level preference signals by comparing sibling subtrees. Can this structural approach replace separate process reward models without explicit step-level annotation?
Can shared-prefix trees reduce redundancy in agent rollouts? Does tree depth automatically produce supervision at multiple granularities? Does supervising retrieval steps outperform final answer rewards? Can step-wise expert rewards help small models learn hard reasoning? Can trajectory structure replace hand-annotated process rewards?
Shared-prefix tree rollouts yield more distinct trajectories per token budget Tree expansion depth automatically yields multi-granular process supervision signals Process supervision substantially outperforms outcome only reward in agentic RAG Step-wise expert-similarity rewards enable small models to learn hard reasoning Trajectory structure can substitute for separately trained process supervision
How do outcome and process rewards differ in their treatment of intermediate steps? How does process supervision relate to execution-signaled feedback approaches? What execution feedback signals drive context updates without supervision labels? Do outcome-only reward signals miss step-level errors that compound later? What makes process-level supervision better than outcome-only reward signals? How does process-focused feedback compare to outcome-focused feedback in skill training? Why do process reward models need human annotation while MCTS intermediate nodes don't? How do process-level rewards compare to environment-extracted next-state signals? Can self-supervised methods replace human annotations for process reward models? Does reverse-curriculum learning approximate process supervision using only outcome signals? What information-theoretic framework explains why process rewards beat outcome only? What makes process-level supervision better than outcome-only rewards for RAG training?See all 57 inquiring lines on this note →
Why do unified image generators fail on non-Latin scripts?
GPT-4o excels at multimodal generation across 20+ tasks, but systematically fails to render non-Latin scripts and underrepresented cultures accurately. What explains this specific failure mode in otherwise capable systems?
Do LLMs represent low-resource cultures through dominant cultural proxies? Does multimodal zero-shot performance actually generalize or interpolate? Can we solve modality competition through architectural design? How do language models learn to think like humans? Can a single model generate all modalities without external encoders?
LLMs internalize cultural flattening as unidirectional representation pathways Multimodal zero-shot performance is driven by concept frequency in pretraining data Modality competition is architectural, not inherent to multimodal training Language models develop human-like cognitive biases and reasoning patterns Discrete multimodal tokens enable any-to-any generation and visual reasoning
No Inquiring Lines yet