Do LLMs represent low-resource cultures through dominant cultural proxies?
Explores whether language models internally represent cultures from data-poor regions by routing through high-resource cultural proxies rather than learning independent representations, and what this reveals about cultural bias in model architecture.
CultureScope is the first mechanistic interpretability method designed to probe how LLMs internally represent cultural knowledge. Using activation patching to extract cultural knowledge spaces, the paper reveals that cultural bias is not merely a surface output problem but a structural property of internal representations.
Cultural flattening as internal architecture. Visualization of the cultural flattening direction between cultures reveals unidirectional connections: low-resource cultures like Ethiopia and Algeria are internally represented through high-resource cultures like the United States and Iran. This means the model has not learned independent representations for these cultures — it has learned to route through dominant cultural proxies. When asked about Ethiopian customs, the model's internal representations partially activate American or Iranian cultural knowledge.
Hard-negative evaluation exposes the mechanism. Standard MCQ evaluation masks this because models can exploit surface-level elimination strategies without genuine cultural understanding. When culturally nuanced hard negatives are introduced (answers from similar but distinct cultures), models systematically favor culturally adjacent answers — explained by the unidirectional representation pathways CultureScope reveals.
Paradoxically, low-resource cultures are less susceptible. Cultures with very limited training data show less cultural flattening, likely because the model has insufficient data to form strong representational connections at all. The most affected cultures are those with moderate data — enough to trigger representation but insufficient to develop independent cultural knowledge structures.
This finding connects internal representation quality to downstream cultural harm. If a model represents Ethiopian culture as a variant of American culture internally, no amount of output-layer correction will fix the fundamental representational deficit. The bias is architectural, not behavioral.
Inquiring lines that read this note 50
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
What reasoning limitations and biases do language models systematically exhibit?- Do language models understand tacit workplace norms and unspoken social rules?
- Do language models build world models or just task-specific heuristics?
- Can large language models predict social norms better than individual script variation?
- Can LLMs predict social norms without deep integration into linguistic practices?
- Why do language models approximate collective human judgment better than individuals?
- Do language models consistently produce anachronistic output about historical periods?
- Can language models learn internal world models without explicit environment specifications?
- Do individual language models match particular human judges better than population averages?
- How do low-dimensional representation structures entangle multiple cultures together?
- How does mechanistic interpretability reveal ideological structures in language model weights?
- Can a world model have rich representations without adequate data coverage?
- How deeply are ideological structures represented in large language models?
- How do you measure the depth of political representation inside a language model?
- What happens when you remove core political features from a deep model?
- What substrate do supervised models lack that makes them weaker on low-resource languages?
- Why do moderately represented cultures show more flattening than data-poor cultures?
- Do rare cultural concepts fail predictably as model scale increases?
- What distinguishes genuine cultural understanding from exploited surface-level elimination strategies?
- Why do Western samples dominate studies claiming AI cultural competence?
- How do LLM biases reflect social classification schemas rather than random errors?
- Can LLMs recover true joint distributions from marginal census data?
- Why do language models reproduce human EPA structure despite different architecture?
- How do training regularities in LLMs overrepresent dominant languages and ideologies?
- Does LLM vocabulary become the cultural lexicon for how we think?
- Can LLMs infer user demographics from implicit signals alone?
- Why do language models successfully simulate political perspectives and social personas?
- What does zero-shot psychological profiling reveal about language model representations?
- Why does language compression via statistical dependencies capture cultural and situated language use?
- Can statistical learning from language alone capture all aspects of cultural competence?
- Can statistical learning from text replace embodied cultural experience?
- What makes internal embeddings useful as multimodal input for language model training?
- Do self-supervised speech models learn dialect-agnostic representations across languages?
- Does encoding information in LM representations guarantee it influences output?
- When does encoded knowledge fail to influence language model generation?
- Why might encoded world knowledge fail to actually influence language model outputs?
- How do description-based identifiers bias language model output distribution?
- What social information is missing from language data?
- How do corpus statistics shape the abstraction hierarchy in language model representations?
- What linguistic units do learned concepts correspond to in a language model?
- Do language models systematically underrepresent non English knowledge about local topics?
Related concepts in this collection 3
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can identical outputs hide broken internal representations?
Can neural networks produce correct outputs while having fundamentally fractured internal structure that prevents generalization and creativity? This challenges our assumptions about what performance benchmarks actually measure.
cultural flattening is a specific form of FER: two cultures that should be independently represented are entangled through shared high-resource proxies, with the fracture being the loss of culture-specific regularities
-
Do LLM semantic features organize along human evaluation dimensions?
Does the structure of meaning in language models match the three-dimensional semantic space (Evaluation-Potency-Activity) that humans use? If so, what are the implications for steering and alignment?
cultural representations may be entangled in similar low-dimensional structures, where steering toward one culture predictably activates others in the same representation cluster
-
Can we measure how deeply models represent political ideology?
This research explores whether LLMs vary not just in political stance but in the internal richness of their political representation. Understanding this distinction could reveal how deeply models have internalized ideological concepts versus merely parroting positions.
cultural depth (the richness of culture-specific features) determines whether the model can be steered toward authentic cultural representation or falls back on flattened proxies
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Entangled in Representations: Mechanistic Investigation of Cultural Biases in Large Language Models
- Computational structuralism: Toward a formal theory of meaning in the age of digital intelligence
- AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuances
- Grounding Multilingual Multimodal LLMs With Cultural Knowledge
- The Homogenizing Effect of Large Language Models on Human Expression and Thought
- Six misconceptions about large language models: A minimal model and diagnostic taxonomy
- LLMorphism: When humans come to see themselves as language models
- On Epistemic Diversity in Large Language Models
Original note title
LLMs internalize Western-dominance bias and cultural flattening as unidirectional representation pathways — low-resource cultures are represented through high-resource cultural proxies