INQUIRING LINE

Different AI companies build separate models — so why do they all tend to give you the same answer to open-ended questions?

Why do different language models independently produce similar outputs in open-ended tasks?

This explores why separately built AI models (from different companies, with different architectures) end up giving strikingly similar answers to questions that have no single right answer, like writing a poem or brainstorming ideas, and what in their training and design pushes them toward the same place.


This explores why models built by different teams still give near-identical answers to open-ended prompts, where you'd expect variety. The clearest evidence comes from a study of more than 70 models across 26,000 open-ended queries. It found what the authors call an "Artificial Hivemind": models independently produce very similar, sometimes identical, responses Do different AI models actually produce diverse outputs?. The explanation is mundane but important. The models learn from largely overlapping data and are aligned with similar procedures, so the surface variety among AI providers hides a shared foundation. One practical consequence is easy to miss: if you ask several models for their opinions to get diverse perspectives, you may mostly get the same perspective back several times.

The convergence starts with what training data is. Models don't learn human expression as a whole. They learn a skewed slice of it, shaped by what was written down, posted online and collected Do large language models narrow human expression and thought?. Every model trained on that slice inherits the same regularities. A useful way to see this is to treat models as probability machines: they're best at producing whatever was most likely in their training data and worst at the unusual Can we predict where language models will fail?. Open-ended prompts don't override this pull. They give it more room, because nothing in the prompt anchors the model away from the most common answer.

You might hope a clever prompt would steer a model somewhere new. But research on context use shows that when a model's training-based associations are strong, they tend to override what's in the prompt. Instructions alone often can't shift them, and reliably changing the behavior required intervening directly in the model's internal representations Why do language models ignore information in their context?. Fine-tuning adds another narrowing force. In reinforcement learning, when a model's possible answers to a prompt all earn about the same reward, the training signal weakens and the model drifts toward generic templates that ignore the specific input Why do language models collapse into generic templates?. That paper studies reasoning tasks, but the mechanism fits open-ended writing well: when there's no clear best answer to reward, the safe default wins.

The surprising part is that the variety does exist inside the models. Regenerating the same response shows that a model holds a spread of possible characters and continuations and samples from them rather than committing to one Do large language models actually commit to a single character?. Models can even hold several different tasks in mind at once, but generating text word by word forces them to settle on one path almost immediately Can LLMs handle multiple tasks at once during inference?. So the sameness comes less from a lack of possibilities and more from how training and decoding keep choosing the most popular one.

The effect also reaches beyond the models. Co-writing studies find that people unconsciously adopt the stances and framings of the models they write with, so when many people rely on the same few similar models, the narrowing spreads into human writing Do large language models narrow human expression and thought?. The collection explains clearly why convergence happens but says much less about reliable fixes. If you're looking for proven ways to get real diversity out of models, the material here is thin.


Sources 7 notes

Do different AI models actually produce diverse outputs?

INFINITY-CHAT analyzed 70+ models across 26K open-ended queries and found an "Artificial Hivemind" effect: models independently generate strikingly similar or identical responses due to overlapping training data and alignment procedures, undermining the diversity benefits of model ensembles.

Do large language models narrow human expression and thought?

LLMs mirror skewed slices of human experience shaped by training data regularities, and widespread reliance on identical models amplifies convergence. Co-writing studies show users unconsciously adopt model stances and framings.

Can we predict where language models will fail?

By framing LLMs as autoregressive probability machines, researchers predicted tasks with low-probability target responses would be systematically harder, even when logically simple. Experiments confirmed predictions like backwards alphabet and letter counting.

Why do language models ignore information in their context?

Research demonstrates that LMs generate outputs inconsistent with their context because parametric knowledge from training dominates over in-context information. Textual prompting alone cannot override strong priors; causal intervention in representations is required.

Why do language models collapse into generic templates?

When within-prompt reward variance is low, task gradients weaken and regularization dominates, pushing policies toward generic outputs. SNR-Aware Filtering—selecting high-variance prompts before updates—recovers performance across tasks and scales.

Show all 7 sources
Do large language models actually commit to a single character?

Shanahan's 20-questions test shows LLMs maintain a superposition of consistent objects or characters and sample from that distribution at generation time. Regenerating the same response yields different outputs, each consistent with prior context, proving no fixed commitment exists.

Can LLMs handle multiple tasks at once during inference?

Large language models represent multiple complete, computationally distinct tasks simultaneously during inference—a macroscopic phenomenon separate from feature-level superposition. However, autoregressive decoding forces convergence to a single task after the first token, preventing practical multi-task generation.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.