Recognizing what something is seems to trip up computers far more than following step-by-step instructions — why is that?

This explores why recognizing what something is turns out to be harder for machines than following instructions someone spelled out. The library has no paper that takes on this classic puzzle directly. Economists sometimes call it Polanyi's paradox: we know more than we can tell. Read across the collection, though, the notes keep reaching the same answer: rules can be written down and checked, but recognition rests on fuzzy, shifting patterns that nobody can fully specify.

The clearest evidence comes from what happens when the two jobs get separated. Vision-only screen agents struggle when one model has to work out what each icon means and also decide what to click. Once a separate step labels the screenshot ("this is the search box, this is the settings gear"), choosing the action becomes manageable. That suggests the hard part was identifying things, not deciding what to do (Why do vision-only GUI agents struggle with screen interpretation?). LLM Programs follow the same logic from the other side. An explicit algorithm handles the control flow and the bookkeeping, and the model is called only for narrow steps that need judgment (Can algorithms control LLM reasoning better than LLMs alone?). The rules are the easy scaffolding. Interpretation is what has to be handed to a learned system.

There's also a hint about where the line falls. Researchers have managed to write ordinary code that reproduces the behavior of up to 30–40% of a transformer's attention heads (Can executable programs replace transformer attention heads?). Part of what a model does can be captured as explicit rules. Most of it so far hasn't been. Even a correct identification is often not enough. A robot that finds the object that looks most like a "cup" may still pick one it can't actually grasp or use. Recognition has to be re-ranked by what you can physically do with the object (Can visual similarity alone guide robot object retrieval?). Knowing what something is depends on context and purpose in a way that rule-following doesn't.

The part you may not expect to care about is fragility. Learned recognizers can score perfectly on tests while their internal organization is fractured, so they break under small changes that the metrics never reveal (Can models be smart without organized internal structure?). AI-text detectors show this in practice. They catch over 99% of machine rewrites right up until a new model is released, then drop to about 4%, because the word-choice patterns they had learned simply move (Why do AI detectors suddenly fail when models update?). A rule means the same thing tomorrow. A category in the world drifts. Even in an idealized math setting, telling decoys from real objects only becomes reliable when you already know, or can learn, how each one tends to respond (Can repeated quiet probes separate decoys from genuine objects?). Identification is a matter of statistics, and the statistics keep changing.

So the corpus suggests a sharper version of the question. Recognition isn't only harder to do. It's harder to verify you've done it, because it never comes with a complete specification to check against. If you want the history-of-automation angle, why machines took over rule-based clerical work long before perception tasks, the library is thin here, and that would be a good gap to fill.