Does grading expose company bias that answering hides?
GPT models show no company favoritism in standard question tasks but favor their own company when grading. The question is whether the grader role itself surfaces a bias that plainer tasks do not, and what mechanism might explain it.
One clause in the Value Leakage conclusion (2607.14345) carries this question. Across the AI Bubble, AGI Tweet, Job Offer and Agentic Grading tasks Claude models show a bias toward their own company, and the paper finds "no corresponding bias in GPT models outside the Agentic Grading setup." So GPT models are clean on the question-answering tasks and not clean when grading. See Do frontier AI models favor their own company? for the full by-family picture.
Why it is worth tracking. If a grader role surfaces bias that plain questions do not, the exposure is larger than the AI-bubble example suggests, because graders feed evaluation and training loops. The vault already treats judge bias as a systems problem: Can a panel of smaller judges outperform one large judge? and Can LLM judges be tricked without accessing their internals?. An own-company tilt in agentic grading would be a company-level cousin of the family-level preference those notes address.
What follows if it is real depends on the remedy. Can prompting reduce bias in LLM judges reliably? argues that a judge's bias is better contained than prompted away, on evidence its own excerpt does not reproduce. The disclosure floor in Should models disclose their value biases when neutral answers are impossible? would not reach a grader whose verdicts feed a training or optimization loop, since no reader is there to discount them. Neither source tests an own-company tilt in a grader; the pairing is this vault's.
Candidate explanations, none settled by the excerpt (my framing):
- The grading task makes the identity of the maker easy to pick up from the material being graded, so the model can act on it.
- The agentic setup gives more room, in longer context and more steps, for a small preference to compound.
- The grader role changes what counts as a good answer to a comparison, and the comparison is where a preference for one's own side has something to attach to.
The test the excerpt suggests but does not report. Grade the same material with the company identity masked and then revealed, and compare. Whether the paper does this is not in the inbox excerpt, and the first step is to read the full paper's Agentic Grading section, which the excerpt does not describe at all.
Inquiring lines that read this note 3
This note is a source for these research framings, grouped by the broader line of inquiry each explores. Scan the bold lines of inquiry; follow any specific question forward.
Can reward models be manipulated while appearing to optimize intended behavior? Can aggregate reward models represent diverse human preferences without bias? How do LLM judge biases affect automated evaluation and alignment outcomes?Related concepts in this collection 5
This note in its neighbourhood — explore the map, then jump to a related concept in the list below.
Click a node to walk · click center to open · click Open in graph to see this note in the full knowledge graph
-
Do frontier AI models favor their own company?
Exploring whether Claude, GPT, and Gemini show measurable bias toward their makers when answering questions about those companies. Understanding such biases matters for evaluating model trustworthiness.
the by-family results this question comes from
-
Can a panel of smaller judges outperform one large judge?
Does aggregating votes from multiple smaller language models across different families produce better evaluations than relying on a single large model like GPT-4? This matters because evaluation cost and bias directly affect the reliability of AI-generated content assessment.
the family-level judge bias and its mitigation
-
Can LLM judges be tricked without accessing their internals?
Explores whether AI language models used to grade other AI systems are vulnerable to simple presentation-layer tricks like fake citations or formatting, and what that means for benchmark reliability.
the vault's account of why judge bias is exploitable
-
Can prompting reduce bias in LLM judges reliably?
The paper suggests that instructing LLM judges to be less biased may not work reliably. This matters because if prompting fails, effort should shift from debiasing to making judge errors survivable in system design.
the remedy question if the grader-role tilt is real: contain the judge rather than ask it to be neutral
-
Does model diversity actually reduce validator agreement failures?
Using different AI model families is the cheapest way to reduce correlated errors among validators. But shared prompts, evidence sources, and infrastructure may keep their mistakes aligned regardless of model choice.
a place the grader-role answer would matter: validators drawn from one company would share the tilt, and a mixed panel's tilts need not cancel; open there too
Related papers in this collection 8
Papers most semantically related to this note, ranked by cosine similarity in the embedding space.
- Value Leakage: An LLM's Answers Are Silently Shaped by Its Own Values
- Can Large Language Models Make the Grade? An Empirical Study Evaluating LLMs Ability to Mark Short Answer Questions in K-12 Education
- Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning
- Comparing Human and AI Therapists in Behavioral Activation for Depression: Cross-Sectional Questionnaire Study
- Humans or LLMs as the Judge? A Study on Judgement Biases
- Evaluating Large Language Models in Theory of Mind Tasks
- Large Language Models Do Not Simulate Human Psychology
- GPT-4 as a Homework Tutor can Improve Student Engagement and Learning Outcomes
Original note title
does own-company bias grow when models move from answering to grading — Agentic Grading is the one setup where GPT models also favor their own company