The ultimate human-machine interface is a direct brain link — but what does that intimacy actually do to your sense of control?
How do dense interfaces like brain-computer interfaces alter agency?
This explores what happens to a person's sense of control and authorship when the interface between them and a machine gets very close and very high-bandwidth, with brain-computer interfaces as the extreme case.
This explores what happens to human agency, meaning your sense that you are the one thinking, choosing and acting, when the interface between you and a machine becomes dense and intimate, with brain-computer interfaces as the furthest point. The collection has no papers on brain-computer interfaces themselves. It does cover the steps leading up to them: interfaces that read you without asking, and assistants so woven into your work that you lose track of where your contribution ends. Those notes suggest a dense interface threatens agency less through mind control than through quieter shifts in authorship, effort and attention.
The nearest thing to a brain-computer interface in the collection is AI that reads your cognitive state from how you behave. Systems can treat gaze, typing hesitation and interaction speed as a continuous signal of what you're thinking. That lets them time help so it doesn't break your flow, without asking you anything Can AI systems read cognitive state from interaction patterns alone?. The catch is that the same signal that enables well-timed help also enables manipulative profiling. Once an interface reads you instead of waiting for explicit commands, you no longer fully decide what you reveal. A brain-computer interface is that same move with a far richer signal. Research on world models adds a reason this matters: to predict what a person will do, a model has to track their beliefs, wants and intentions, not just the physical scene Can world models predict human action from physics alone?. A denser interface supplies exactly the mental-state data that makes people predictable, and potentially steerable.
The second thread is about who did the work. A four-month EEG study found that brain connectivity scaled down as people relied more on an LLM. Heavy users showed the weakest neural engagement and had trouble recalling essays they had just written Does AI assistance weaken our brain's ability to think independently?. A separate line of work names the "LLM Fallacy": people credit their own ability for what the AI produced, and this happens whether or not the output is accurate How does AI-assisted work reshape how people see their own abilities?. Together these point to a surprising risk. The closer the coupling, the harder it gets to tell your own thinking from the machine's. You might keep the feeling of agency while losing the substance, or the reverse. The proposed remedy is to make the boundary between human and machine contributions visible, not just to make the system more accurate. That design principle matters even more when the interface sits closer to the mind.
A third angle is how we perceive the system on the other side. When people treat an AI as a mind, the risks spread across several areas, including emotional dependence and erosion of their own autonomy. Interaction design that addresses this perception works better than fixes at the system level Does perceiving AI as conscious create multiple distinct risks?. Users already judge dialogue agents on competence, human-likeness and flexibility How do users mentally model dialogue agent partners?. A more seamless interface would likely push those judgments toward treating the system as an extension of yourself or as a partner, and neither feels like a tool you are operating.
The takeaway is that a dense interface changes agency in two directions at once. It extends what you can do, but it also blurs the boundary that lets you tell what you did. If you want primary research on brain-computer interfaces specifically, this collection doesn't have it yet. The behavioral-sensing and cognitive-debt notes are the best places to start.
Sources 6 notes
Research shows AI systems can instrument multimodal behavioral signals (gaze, hesitation, speed) to read cognitive state during interaction, preserving flow by avoiding disruptive explicit probes. However, the same substrate enables both helpful timing and manipulative profiling.
Research across eight LLM-based world models shows that tracking only the physical scene leads to wrong action predictions even when the scene looks correct. Mental World Modeling makes beliefs, wants, and intentions explicit state components coupled to physical simulation, and all three elements are required for accurate human decision prediction.
A four-month EEG study of 54 participants found that brain connectivity systematically scaled down with AI reliance—LLM users showed weakest neural engagement, poorest memory retention, and impaired ability to recall their own recent work.
Research shows the LLM Fallacy operates through misattribution of AI outputs to personal capability, independent of output accuracy or reliance behavior. It requires interventions that clarify human-machine contribution boundaries, not just better system accuracy or forced verification.
Research shows that consciousness attribution to AI drives multiple distinct risks—emotional dependence, autonomy erosion, status erosion, and political conflict—all stemming from treating systems as minds. Interaction design mitigations targeting this perceptual move are more directly effective than system-level alignment efforts.
Show all 6 sources
The Partner Modelling Questionnaire reveals that perceived competence dominates user impressions (49% of variance), followed by human-likeness (32%) and communicative flexibility (19%). This three-factor structure reflects how people evaluate dialogue partners against both functional and social standards.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- Mental World Modeling
- Seemingly Conscious AI Risks
- The Partner Modelling Questionnaire: A validated self-report measure of perceptions toward machines as dialogue partners
- The LLM Fallacy: Misattribution in AI-Assisted Cognitive Workflows
- Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task
- Can Language Models Serve as Text-Based World Simulators?
- Linguistic Alignment in Conversational AI: A Systematic Review of Cognitive-Linguistic Dimensions, Measurements, and User Outcomes (2020–2025)
- CloChat: Understanding How People Customize, Interact, and Experience Personas in Large Language Models