INQUIRING LINE

Ask an AI to predict next week versus next year — does looking further ahead reliably make it worse?

Do longer prediction horizons systematically degrade LLM forecasting accuracy?

This explores whether asking an LLM to predict further into the future (next week versus next year, say) reliably makes its forecasts worse, and whether the collection holds evidence either way.


This explores whether predicting further out reliably makes LLM forecasts worse. The collection has no study that tests this directly. None of the retrieved notes compares accuracy at short and long horizons, so there is no measured decay curve to report. What the notes do give is indirect evidence, and it leans away from 'systematic degradation' and toward 'the result depends on how the forecast is built'.

The closest hint is that forecasting quality seems to depend more on structure than on raw model strength. One note finds that LLMs forecast better than expected once the workflow separates numerical reasoning from contextual reasoning, and that a single monolithic prompt hides this ability (Can LLMs actually forecast time series better than we think?). Another builds horizon into the design: its system produces a dual-resolution macro and micro outlook before synthesizing a forecast (Can decomposing forecasting into stages unlock numerical and contextual reasoning?). That treats long-range and short-range views as separate jobs. It suggests horizon is something to engineer around, but the summary doesn't report accuracy at each horizon, so it doesn't show that degradation happens or doesn't.

The evidence on real-world forecasting also blurs any horizon effect. A retrieval-augmented system reached near-parity with competitive human forecasters on questions published after the model's training cutoff, and newer model generations improved without any forecasting-specific tuning (Can retrieval-augmented language models forecast like human experts?). If accuracy shifts with model generation as much as with anything else, a horizon effect would be hard to isolate from it. In founder-success prediction, several LLMs beat human experts, but only because the human bar there was modest (Can language models beat human venture capital experts?). That shows a forward-looking task can be within reach. It says nothing about how the gap changes as the target moves further out.

Two other notes matter for how you would test the question. First, a fixed seed and zero temperature give a repeatable answer, but that answer is still one draw from the model's distribution (Does setting temperature to zero actually make LLM outputs reliable?). A horizon curve built from single runs could mistake noise for a trend. Second, 'long horizon' has a different meaning in agent work. On very long optimization tasks, success was best predicted by persistence in feedback loops, and most models quit early or wasted their budget (What predicts success in ultra-long-horizon agent tasks?). That concerns long tasks, not long predictions, but it shows that failures at long horizons can come from how a model behaves over time rather than from the prediction itself being harder.

The corpus can't say yes or no. A direct answer needs a horizon-by-horizon comparison that controls for model generation and workflow design, and that comparison isn't in this collection.


Sources 6 notes

Can LLMs actually forecast time series better than we think?

LLMs have stronger intrinsic forecasting ability than recognized, but only when workflows separate numerical reasoning from contextual reasoning. Monolithic prompting obscures this capability; structured decomposition surfaces it.

Can decomposing forecasting into stages unlock numerical and contextual reasoning?

Nexus outperforms pure TSFM and LLM baselines on real-world datasets by decomposing forecasting into contextualization, dual-resolution macro/micro outlook, and synthesis stages. Separating numerical extrapolation from event-driven contextual reasoning avoids forcing one model to handle both simultaneously.

Can retrieval-augmented language models forecast like human experts?

A retrieval-augmented LM system achieved near-parity with competitive human forecasters on real forecasting questions published after model training cutoffs, sometimes surpassing human crowds. Newer model generations naturally improved forecasting without domain-specific tuning.

Can language models beat human venture capital experts?

VCBench shows several LLMs exceed human baselines in founder-success prediction, with DeepSeek-V3 achieving 6× market-index precision. In sparse-signal forecasting where experts only modestly beat chance, even raw LLM capability suffices to clear the human bar.

Does setting temperature to zero actually make LLM outputs reliable?

Fixed seeds and zero temperature replicate the same output repeatedly, but that output remains one draw from the model's probability distribution. McDonald's omega testing across 100 repetitions reveals that consistency does not equal reliability.

Show all 6 sources
What predicts success in ultra-long-horizon agent tasks?

Across 17 frontier models on 36 expert-curated optimization tasks, repeated benchmark-edit-incorporate cycles within a wall-clock budget proved the dominant success predictor. Most models terminated early or burned budget unproductively; Claude Opus 4.6 stood out as persistent.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.