Do LLMs represent low-resource cultures through dominant cultural proxies?
Explores whether language models internally represent cultures from data-poor regions by routing through high-resource cultural proxies rather than learning independent representations, and what this reveals about cultural bias in model architecture.
CultureScope is the first mechanistic interpretability method designed to probe how LLMs internally represent cultural knowledge. Using activation patching to extract cultural knowledge spaces, the paper reveals that cultural bias is not merely a surface output problem but a structural property of internal representations.
Cultural flattening as internal architecture. Visualization of the cultural flattening direction between cultures reveals unidirectional connections: low-resource cultures like Ethiopia and Algeria are internally represented through high-resource cultures like the United States and Iran. This means the model has not learned independent representations for these cultures — it has learned to route through dominant cultural proxies. When asked about Ethiopian customs, the model's internal representations partially activate American or Iranian cultural knowledge.
Hard-negative evaluation exposes the mechanism. Standard MCQ evaluation masks this because models can exploit surface-level elimination strategies without genuine cultural understanding. When culturally nuanced hard negatives are introduced (answers from similar but distinct cultures), models systematically favor culturally adjacent answers — explained by the unidirectional representation pathways CultureScope reveals.
Paradoxically, low-resource cultures are less susceptible. Cultures with very limited training data show less cultural flattening, likely because the model has insufficient data to form strong representational connections at all. The most affected cultures are those with moderate data — enough to trigger representation but insufficient to develop independent cultural knowledge structures.
This finding connects internal representation quality to downstream cultural harm. If a model represents Ethiopian culture as a variant of American culture internally, no amount of output-layer correction will fix the fundamental representational deficit. The bias is architectural, not behavioral.
Inquiring lines that read this note 41
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
How do language models establish social grounding in human dialogue?- Do language models understand tacit workplace norms and unspoken social rules?
- Can LLMs predict social norms without deep integration into linguistic practices?
- Why do language models presume common ground instead of building it?
- How do low-dimensional representation structures entangle multiple cultures together?
- Can statistical learning from language alone capture all aspects of cultural competence?
- Can statistical learning from text replace embodied cultural experience?
- What distinguishes genuine cultural understanding from exploited surface-level elimination strategies?
- How much cultural knowledge exists only in unwritten social rules?
- How do LLM biases reflect social classification schemas rather than random errors?
- Can large language models predict social norms better than individual script variation?
- Why do language models approximate collective human judgment better than individuals?
- Can LLMs recover true joint distributions from marginal census data?
- How does mechanistic interpretability reveal ideological structures in language model weights?
- How do you measure the depth of political representation inside a language model?
- What happens when you remove core political features from a deep model?
- How deeply are ideological structures represented in large language models?
- Does encoding information in LM representations guarantee it influences output?
- When does encoded knowledge fail to influence language model generation?
- Why might encoded world knowledge fail to actually influence language model outputs?
- Do language models consistently produce anachronistic output about historical periods?
- How do description-based identifiers bias language model output distribution?
- How do corpus statistics shape the abstraction hierarchy in language model representations?
- Do language models build world models or just task-specific heuristics?
- Why do language models reproduce human EPA structure despite different architecture?
- Can language models learn internal world models without explicit environment specifications?
- How can we probe LLM representations in channels that training did not target?
Related concepts in this collection 3
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Can identical outputs hide broken internal representations?
Can neural networks produce correct outputs while having fundamentally fractured internal structure that prevents generalization and creativity? This challenges our assumptions about what performance benchmarks actually measure.
cultural flattening is a specific form of FER: two cultures that should be independently represented are entangled through shared high-resource proxies, with the fracture being the loss of culture-specific regularities
-
Do LLM semantic features organize along human evaluation dimensions?
Does the structure of meaning in language models match the three-dimensional semantic space (Evaluation-Potency-Activity) that humans use? If so, what are the implications for steering and alignment?
cultural representations may be entangled in similar low-dimensional structures, where steering toward one culture predictably activates others in the same representation cluster
-
Can we measure how deeply models represent political ideology?
This research explores whether LLMs vary not just in political stance but in the internal richness of their political representation. Understanding this distinction could reveal how deeply models have internalized ideological concepts versus merely parroting positions.
cultural depth (the richness of culture-specific features) determines whether the model can be steered toward authentic cultural representation or falls back on flattened proxies
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Entangled in Representations: Mechanistic Investigation of Cultural Biases in Large Language Models
- Computational structuralism: Toward a formal theory of meaning in the age of digital intelligence
- Grounding Multilingual Multimodal LLMs With Cultural Knowledge
- LLMorphism: When humans come to see themselves as language models
- Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Control
- Large Language Models Reflect the Ideology of their Creators
- Beyond the Surface: Probing the Ideological Depth of Large Language Models
- Mechanistic Indicators of Understanding in Large Language Models
Original note title
LLMs internalize Western-dominance bias and cultural flattening as unidirectional representation pathways — low-resource cultures are represented through high-resource cultural proxies