When researchers use tools to peek inside an AI, can they spot ideas, facts, or real rules, and which can they trust?

This reads the question as: when automated tools look inside or around an LLM, how much of its knowledge can they recover, and which levels of that knowledge are they actually reliable on? The corpus has one direct answer, and it's layered. Mechanistic interpretability, the work of reverse-engineering a model's internals, finds three tiers of understanding. The first is conceptual, where ideas show up as directions in the model's internal space. The second is state-of-the-world, where facts are connected to each other. The third is principled, where compact circuits implement a general rule. The catch is that the higher tiers don't replace the lower ones. They sit alongside lower-tier shortcuts, so the picture is a patchwork, not a clean staircase (Do language models understand in fundamentally different ways?). Finding a principled circuit shows that the model has one. It doesn't show the model uses it every time.

That gap between having and using explains why the easiest automated route, asking the model to explain itself, is the least reliable one. Models can explain a concept correctly, fail to apply it, and then recognize that they failed. Human understanding doesn't break down that way, which suggests the explaining and doing pathways are largely disconnected (Can LLMs understand concepts they cannot apply?). Stripping the meaning out of a reasoning task makes the same point from the other side. Performance collapses even when the correct rules are sitting in context, so much of what looks like rule-following is really association with what the words usually mean (Do large language models reason symbolically or semantically?). Tools that probe with perturbations like this can reliably extract the associative layer. Whether a rule is really being applied is harder to establish.

A second pattern is that automated checks tend to capture surface form more reliably than meaning. LLMs write well-formed logic that is semantically wrong, with errors clustering around scope, quantifiers and predicate granularity. A parser would pass all of it (Can large language models translate natural language to logic faithfully?). Errors in complex grammar grow predictably with syntactic depth, which suggests surface patterns were learned and deep rules weren't (Why do large language models fail at complex linguistic tasks?). The trend is also getting worse for surface-level checking. Weaker models visibly delete content, but frontier models corrupt it while keeping the document looking intact (Does model capability change how documents degrade?). More capable models make surface checks less informative about the knowledge underneath.

Two other approaches get around the problem by changing what gets extracted. One is to make the model emit its working knowledge in an inspectable form. Knowledge Graph of Thoughts has a small model build reasoning as explicit triples, which makes each step checkable (Can structuring reasoning as knowledge graphs help smaller models solve complex tasks?). That exposes the task's working knowledge, not what is stored in the weights. The other is conflict detection. Timestamped knowledge bases let a system automatically notice when new information contradicts old, but choosing the right side depends on context outside the system, so a human has to resolve it (Can LLMs learn reliably at test time without human oversight?). Systems checking themselves are also formally limited by the gap between generating and verifying, and measurement error hides the true ceiling (What limits autonomous capability in large language models?).

The corpus has no head-to-head study of extraction reliability at each tier, so this is a pattern across notes, not a measured ranking. The pattern is that automation reliably gets the concept and association layer, and it can flag conflicts and structural errors. It is much weaker at confirming that a principle is actually being applied, or that a fluent output means what it appears to mean.