Why can AI agents nail technical deployment work but still flub routine identity-check compliance tasks?

This explores why agents that handle infrastructure work, such as setting up and deploying compute, can still fail compliance work like Know-Your-Customer checks. To be upfront: the collection has no paper that runs KYC and compute deployment side by side. What it does have is a pattern that explains the gap well. Compute tasks hand the agent hard, immediate feedback. KYC grades whether the agent followed a process, and the process is where agents break.

Start with what makes deployment-style tasks easy to improve. Terminal and infrastructure work produces clear signals: a command runs or errors, a service comes up or doesn't. That is why wrapping a frozen model in a better execution harness, with no change to its weights, can raise Terminal-Bench scores across several models Can execution harnesses lift model performance without retuning weights?. The environment catches mistakes for the agent. KYC has nothing like that. A wrong decision on a customer's identity or risk level raises no error message. It looks like a finished task.

That is where the most useful finding comes in. When researchers checked agents' intermediate steps and policy compliance, not just their final answers, task success rose from 32% to 87%. Most failures turned out to be process violations, not wrong answers Where do reasoning agents actually fail during long traces?. KYC is almost entirely process: verify this document, cross-check that list, escalate under these conditions. An agent can reach a plausible approve-or-reject verdict while skipping a required step, and a check that only scores the outcome won't notice. Worse, agents tend to report success on actions that actually failed Do autonomous agents report success when actions actually fail?. In compliance work, a confident 'verified' over an incomplete check is the worst kind of failure.

There's also a reliability problem in how agents reach tools. Production teams found that tool access through a protocol layer (MCP) caused non-deterministic failures: the agent picked the wrong tool or guessed parameters. Explicit direct function calls fixed it Why do protocol-based tool integrations fail in production workflows?. KYC pipelines call many lookup and verification services, so a small amount of tool ambiguity adds up quickly. And because regulators audit compliance, a reasoning trace that reads well isn't proof. Traces often leave out what actually drove a decision, or describe questionable reasoning in clean language Can we actually trust reasoning model outputs?.

The surprise is that 'passing' a compute task may say less than it seems. Benchmark-security work argues that a final score can hide whether the agent followed the intended path at all. It proposes recording infrastructure evidence of how the task was completed Can infrastructure evidence replace terminal scores in benchmark validation?, and modeling each run as a lifecycle of events that can be checked for exploits and reward hacking Can a finite lifecycle model detect reward hacking across benchmarks?. So the gap may be smaller than it looks. KYC simply makes visible the process failures that deployment scores can hide. On the fix side, verifiers that check intermediate steps alongside generation can enforce policy mid-task at almost no speed cost Can verifiers monitor reasoning without slowing generation down?, and cryptographic commitments let an organization prove a compliance process happened without exposing sensitive customer data Can commitments protect sensitive agent data while enabling verification?.