Can intrinsic reward signals extend beyond mathematics to medicine and law?
This explores whether the 'intrinsic' or verifiable reward signals that power math and code reasoning (RLVR and its kin) can carry over to domains like medicine and law, where there's no clean right-answer to check against.
This explores whether the intrinsic reward signals that work so well in math — where an answer is verifiably right or wrong — can travel to medicine and law, where correctness is fuzzier and the stakes are higher. The corpus suggests the honest answer is "partly, but the bottleneck isn't the reward — it's what the model already knows and whether the domain can even be checked."
Start with what these rewards actually do. The research on RLVR dynamics finds that verifiable rewards don't teach new reasoning so much as activate strategies already latent from pretraining — a single example can trigger the effect, and even spurious rewards work nearly as well for a model with the right pretraining What does reward learning actually do to model reasoning?. That reframes the whole question: a reward signal can only surface capability that's already there. So extending to medicine or law depends less on inventing a clever reward and more on whether the model absorbed that knowledge in the first place. A sharp companion finding shows why this matters domain-by-domain: knowledge lives in the lower layers of the network and reasoning in the higher ones, which is exactly why reasoning-style training improves math but can actively degrade knowledge-heavy fields like medicine Why does reasoning training help math but hurt medical tasks?. The thing that helps math can hurt medicine.
The deeper obstacle is verifiability itself. Math gives you a free, automatic correctness signal; medicine and law mostly don't. Two lines of work try to manufacture that missing signal without an external checker. One uses the agent's own shifting belief — the log-ratio of how confident it grows toward a solution — as a dense, intrinsic reward that needs no critic network or verifier, and it generalized beyond its training setup Can an agent's own beliefs guide credit assignment without critics?. The other replaces the missing numeric signal with language: chain-of-thought critiques break through plateaus precisely because a number tells you *that* you failed but not *why*, while a critique does Can natural language feedback overcome numerical reward plateaus?. Both are domain-agnostic by construction, which is what you'd want for law and medicine.
For soft domains, the more promising path may be structuring the reward rather than purifying it. When answers can't be auto-graded, rubrics become the substitute — and the trick is to use them as gates that accept or reject whole rollouts rather than as scores to optimize, which prevents the model from gaming them Can rubrics and dense rewards work together without hacking?. Generative judges that reason step-by-step about the model's reasoning, instead of just classifying it, push further in the same direction and need far less training data Can judges that reason about reasoning outperform classifier rewards?. And critically for high-stakes fields: a ternary reward that explicitly values "I don't know" — rewarding correct answers, penalizing hallucinations, and giving abstention a middle value — cut hallucinations by nearly 29% Can three-way rewards fix the accuracy versus abstention problem?. In medicine and law, a model that knows when to abstain is worth more than one that's confidently wrong.
The thing you didn't know you wanted to know: the barrier to crossing into medicine and law isn't a missing reward function — those already exist in domain-agnostic forms. It's that verifiable-reward training quietly optimizes for the wrong shape of competence, sharpening reasoning at the expense of the factual recall those fields live on, while rewarding the confident-guess behavior they can least afford. The frontier work isn't building a stronger reward; it's teaching the reward to value abstention and to gate on rubrics where ground truth runs out.
Sources 7 notes
Research shows RLVR improves sampling efficiency within existing capability boundaries without expanding them. A single training example suffices for activation, and spurious rewards work nearly as well as correct ones for models with appropriate pretraining.
Two-phase inference model shows knowledge retrieval operates in lower network layers while reasoning adjustment happens in higher layers. This separation explains why reasoning training improves math but can degrade knowledge-intensive domains like medicine.
ΔBelief-RL uses log-ratios of sequential probability estimates to assign per-turn credit without critic networks or process reward models. Tested on 20 Questions, smaller models trained this way matched or exceeded prior SOTA and larger baselines while generalizing beyond training.
Critique-GRPO shows that models stuck on performance plateaus can generate correct solutions when given chain-of-thought critiques, revealing that numerical rewards lack critical information about why failures occur and how to improve.
DRO shows that using rubrics to accept or reject rollout groups—rather than converting rubric scores into dense rewards—prevents reward hacking. This separation preserves the categorical strength of rubrics while letting token-level rewards optimize within valid answers.
StepWiser demonstrates that training judges to produce reasoning chains about policy reasoning—rather than classify steps—yields better judgment accuracy and data efficiency. Independent confirmation from GenPRM and ThinkPRM shows generative PRMs outperform discriminative ones with orders of magnitude less training data.
TruthRL uses three distinct rewards (correct +1, hallucination -1, abstention intermediate) to make abstention learnable. Across four benchmarks, this reduced hallucinations by 28.9% and improved truthfulness by 21.1% compared to binary reward RL.