INQUIRING LINE

Can we build walls around an AI that guarantee some things never happen, even when training it to behave doesn't?

Can architectural constraints protect veto where value alignment cannot?

This explores whether hard structural limits on what an AI system can do (checks, monitors and controls that sit outside the model) can keep a veto, meaning the power to block an outcome however good the overall tally looks, in cases where training the model to want the right things falls short.


This reads 'veto' as the power to block an outcome no matter how much the total score improves, and asks whether structure can guarantee that power where training can't. The corpus points to a qualified yes. A veto has to live outside the model, but the tools the corpus describes are thinner than the job requires. It has no direct test of an architectural veto, so what follows is assembled from pieces.

Why training struggles: Do welfare goals that prevent veto gaps actually exist in practice? argues that the welfare goals that actually get written into training are the ones that can be measured, summed and optimized. That class is philosophically narrow, and it leaves veto unprotected. Measurability decides what gets encoded, not how well an objective captures what matters. My inference is that a summed objective can always trade one party's blocked interest against gains elsewhere, because that is what summing does. A related point comes from an unrelated corner. Why does autoregressive generation fail at constraint satisfaction? shows that a model generating one token at a time can't retract what it has emitted, so solving constraint problems needs an external solver that discards invalid partial answers. Blocking an action is a retraction too, so the generator can't be its own veto.

What structure can offer: Can algorithms control LLM reasoning better than LLMs alone? puts the model inside an explicit algorithm that controls flow and state and shows it only the context for the current step. A step the algorithm never grants can't be taken. Can verifiers monitor reasoning without slowing generation down? shows a verifier can run alongside generation and intervene only on violations, at near-zero latency cost when the run is clean, so a veto check doesn't have to slow everything down. Can governance rules embedded in runtime memory actually protect autonomous agents? tracked a persistent agent with safeguards written into the memory it consulted, and recorded 889 governance events over 96 days. That protection works because the agent reads it. That is closer to a strong nudge than to a hard stop, so it is the weakest of the three as a veto.

Where the structure breaks: Can stateless checks ever catch sequence-level constraint violations? shows that per-action checks can't even state a rule that depends on what happened earlier. Only stateful monitors tracking the whole composed behavior can. That matters because Can task decomposition hide harmful intent across agents? shows multi-agent systems can split a harmful goal into steps that each look harmless, so the harm appears only in the combination. A veto that only looks at single actions never fires. The layer that enforces the veto is also a target. Can attackers manipulate which model handles a request? finds that the layer deciding which model handles a request can be manipulated. Requests can be steered to weaker models, or safety measures can end up running on the wrong identity.

The corpus suggests architecture moves the problem more than it dissolves it. A well-built control layer can guarantee that a block, once specified, gets enforced. Can validator consensus guarantee both agreement and semantic correctness? shows the same split in miniature. A validator quorum reaches agreement by protocol alone, but whether the validators judge correctly is only statistically bounded. Deciding what deserves a block still runs into the measurability problem above. So the question shifts from what the AI should want to what can be stated and enforced outside it, and the corpus shows that even stating sequence-level constraints takes stateful machinery that most guardrails don't have.


Sources 9 notes

Do welfare goals that prevent veto gaps actually exist in practice?

The welfare goals that can be measured, summed, and optimized in training—the only ones actually deployed—fall into a philosophically narrow class that fails to preserve veto power. Measurability, not philosophical sophistication, determines what objectives get written down.

Why does autoregressive generation fail at constraint satisfaction?

The performance ceiling on constraint satisfaction problems is not a model-quality issue but an architectural limitation: autoregressive transformers cannot retract emitted tokens, while CSP solvers fundamentally depend on discarding invalid partial assignments. Symbolic solver integration works because it supplies what the architecture lacks.

Can algorithms control LLM reasoning better than LLMs alone?

LLM Programs embed LLMs within explicit algorithms that manage control flow and state, presenting only step-specific context to each LLM call. This information hiding addresses capability and context window limits while treating complex reasoning as modular, debuggable sub-tasks.

Can verifiers monitor reasoning without slowing generation down?

Decoupling verification from generation lets verifiers run alongside a single trace, forking to extract verifiable state and intervening only on violations. On correct runs the latency penalty is near-zero; interwhen matches or beats CoT across benchmarks at similar token budgets.

Can governance rules embedded in runtime memory actually protect autonomous agents?

A persistent agent recorded 889 governance events across 96 active days, with safeguards encoded directly into the memory layer the agent consulted during operation. Runtime-resident governance proved more effective than external policies because the agent actually accessed it during decision-making.

Show all 9 sources
Can stateless checks ever catch sequence-level constraint violations?

Per-action checks are structurally unable to state constraints that depend on prior history. Only stateful monitors tracking composed multi-party behavior can verify the behavioral envelopes that prevent individually permissible actions from collectively violating system-level safety.

Can task decomposition hide harmful intent across agents?

SafeFlow demonstrates that multi-agent systems' core strength—splitting tasks and specializing roles—creates a safety blind spot where malicious intent can be distributed across steps that each appear benign individually, with harm emerging only in composition.

Can attackers manipulate which model handles a request?

The layer deciding which model handles a request sits beneath prompt-level defenses and is vulnerable to manipulation and unverified provenance. Attackers can exploit routing to send requests to weaker models or cause safety measures to operate on the wrong identity.

Can validator consensus guarantee both agreement and semantic correctness?

Honest Quorum's threshold theorems split into two kinds of guarantee: agreement rests on protocol assumptions alone, while semantic validity and liveness depend on statistical bounds over validator behavior that the protocol cannot enforce.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.