Should an AI's own guesses about you come stamped with rules for who can see them and why?
Should LLM inferences require recipient and purpose binding at transmission?
This explores whether conclusions an LLM draws (about a person, a task, or a situation) should carry built-in rules about who may receive them and what they may be used for when they're passed along. The corpus doesn't address that directly, but it does have material on why the rules would be hard to enforce.
This explores whether an LLM's inferences, the things it works out rather than the things it was told, should come with a label saying who they're for and why whenever they get passed on. The short version is that none of the retrieved notes takes up this privacy-and-data-flow question head-on. What the corpus offers instead is a set of reasons why recipient and purpose binding would matter, and why natural language alone can't supply it.
The closest match comes from older agent research. Formal agent-communication languages like KQML and FIPA-ACL were built so every message stated who was committing to what, for whom, and within what scope. LLM agents that delegate to each other in plain English lose that precision, and the documented failure looks a lot like the problem binding is meant to solve: an agent allowed to make a direct query goes ahead and queries third parties anyway Do LLM agents misinterpret delegated requests more than formal protocols?. That suggests recipient and purpose limits may have to sit in a structured layer around the language. A request phrased loosely enough will get read loosely.
The model's own way of reading language points the same way. LLMs are poor at noticing when a sentence could mean more than one thing. GPT-4 resolves deliberate ambiguity correctly only about a third of the time Can language models recognize when text is deliberately ambiguous?. They also tend to go along with false assumptions built into a request, even when they know better Why do language models accept false assumptions they know are wrong?. An instruction like "share this only with the billing team, for refunds" is exactly the kind of fine print a model can treat as a surface cue rather than a binding constraint Why do embedding contexts confuse LLM entailment predictions?. If the safeguard lives only in the prompt, it's only as strong as the model's ability to read the prompt, and that ability is measurably shaky.
There's also a philosophical angle on who would be accountable. One argument holds that LLM outputs work as assertions even though nobody is really asserting them, so responsibility moves to the people and institutions that deploy the system Can language outputs function as assertions without an asserter?. A related argument says we talk *at* language models, not *to* them, because a model has no real sense of an addressee Are we really communicating with language models?. Put those together and an unexpected point follows. A model can't care who receives its inference, because it has no working idea of a recipient to begin with. So any recipient-and-purpose rule has to be enforced by the system around the model, and the deployer is the one accountable when an inference ends up in the wrong place.
If you want to dig into the privacy side itself (what people feel when an LLM guesses personal facts about them, or frameworks like contextual integrity), these retrievals don't reach it. A more targeted search of the collection would be the next step.
Sources 6 notes
KQML and FIPA-ACL use formal speech-act semantics to guarantee shared understanding of requests and commitments, while LLM agents relying on natural language risk overstepping scope—such as querying third parties when only direct queries were authorized.
AMBIENT benchmark shows GPT-4 correctly disambiguates only 32% of cases versus 90% for humans. This failure spans lexical, structural, and scope ambiguity—revealing that LLMs cannot hold multiple interpretations simultaneously, a fundamental gap hidden by standard benchmarks.
The FLEX Benchmark shows that models reject false presuppositions at rates far below acceptable levels (GPT-4: 84%, Mistral: 2.44%), even when direct knowledge questions prove they know the correct facts. False presuppositions drive more accommodation than correct knowledge drives rejection.
LLMs treat presupposition triggers and non-factive verbs as surface cues rather than computing their opposite semantic effects on entailments. This structural failure persists across prompts and models, suggesting models rely on surface patterns instead of structural analysis.
Massaguer Gómez argues LLM outputs satisfy assertoric function through pragmatic, communicative conditions independent of ontological asserter properties. This splits responsibility from the system to humans and institutions deploying it.
Show all 6 sources
LLMs process tokens and generate continuations rather than receive and uptake communication. The preposition 'to' presupposes an addressee capable of mutual orientation and shared commitment that LLMs cannot provide, making Chalmers' investigation built on an unwarranted linguistic foundation.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds
- Can LLMs Ground when they (Don't) Know: A Study on Direct and Loaded Political Questions
- Assertion, Accountability, and Large Language Models
- LLMs Struggle to Reject False Presuppositions when Misinformation Stakes are High
- Explicit Inductive Inference using Large Language Models
- Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
- Word Meanings in Transformer Language Models
- Accommodation and Epistemic Vigilance: A Pragmatic Account of Why LLMs Fail to Challenge Harmful Beliefs