Can LLMs actually forecast time series better than we think?
Explores whether language models possess stronger forecasting ability than current benchmarks suggest, and what role workflow design plays in revealing or hiding that capability.
The debate over whether LLMs can forecast time-series has been muddled by inconsistent evaluation. Some studies show LLMs underperform dedicated TSFMs; others show LLMs matching or exceeding them. The Nexus authors argue the variance comes from a methodological factor that has been under-attended: how the numerical and contextual reasoning are organized in the forecasting workflow.
Monolithic prompting — ask the LLM to read all the data and produce a forecast — produces uneven results. The model has to integrate seasonal numerical patterns and contextual event-driven catalysts simultaneously, and it does neither well. The intrinsic forecasting capability is real, but the workflow squashes it. Structured workflows — decompose the task into stages that separate numerical reasoning from contextual reasoning, then synthesize — surface the capability that the monolithic approach hides.
This reframes the "can LLMs forecast?" question. The answer is yes when the workflow respects what LLMs do well (contextual reasoning, integration of structured representations) versus what they do poorly (raw numerical extrapolation under noise). Architectures like Nexus that explicitly separate these contributions and use the right component for each get the LLM's strength without exposing its weaknesses.
The methodological consequence for forecasting benchmark design: evaluating "LLM forecasting ability" with a single prompt architecture undersamples the capability space. The right evaluation compares competing workflow designs against the same model, then compares the best workflow's performance against TSFMs. Mixing workflow effects with model effects in a single number obscures which contributes to performance.
The broader observation is that workflow architecture often dominates raw model capability in compound tasks. This shows up here for forecasting, but the pattern recurs: code generation (workflow with planning + execution beats one-shot), retrieval-augmented generation (workflow with retrieval + reranking + generation beats raw generation), reasoning (workflow with structured decomposition beats free-form CoT). For tasks above a complexity threshold, "which workflow" is a stronger lever than "which model."
Inquiring lines that read this note 32
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How can LLM user simulators model realistic goal-driven conversation? What properties determine whether reward signals teach genuine reasoning? Can self-supervised signals enable process supervision without human annotation? What structural advantages do diffusion language models offer over autoregressive methods?- Can autoregressive models learn faithful translation to logical representations without semantic loss?
- Why do hybrid paradigms outperform pure autoregressive or pure diffusion approaches?
- Can simple diagnostic tests predict language model performance in production complexity?
- Can a model be strong at MMLU but weak at long-horizon tasks?
- Why are post-cutoff test sets essential for evaluating genuine forecasting ability?
- What privacy-preserving evaluation methods best capture real-world forecasting ability?
- How do general language model benchmarks predict specialized domain performance?
- Do standard language benchmarks underestimate what LLMs can actually do?
- How should benchmarks evaluate workflow architecture versus raw model performance?
- Can automated benchmarks accurately capture progress on real-world long-horizon tasks?
- How should organizations redesign workflows if LLMs cannot solve optimization directly?
- Which model capabilities actually matter for sustained workflow delegation?
- Why does LLM performance improve when forecasting tasks include organized reasoning?
- How do you partition LLM experts by domain versus by time?
Related concepts in this collection 1
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can decomposing forecasting into stages unlock numerical and contextual reasoning?
This explores whether breaking time-series forecasting into separate stages for contextualization, dual-resolution outlook, and synthesis allows systems to combine the strengths of numerical models and language models more effectively than either alone.
same paper, the specific architectural instantiation
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Nexus: An Agentic Framework for Time Series Forecasting
- Approaching Human-Level Forecasting with Language Models
- Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Grading
- LLMs Corrupt Your Documents When You Delegate
- The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs
- Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks
- Rethinking Interpretability in the Era of Large Language Models
- From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AI
Original note title
LLM forecasting ability is stronger than recognized when numerical and contextual reasoning are organized properly — workflow architecture dominates raw model capability