Can researchers actually see how typical users chat with AI companion apps, or just whoever happens to post about it online?

This explores whether people who study AI companions can see how typical users actually interact with these products, or whether they have to work from substitutes. The corpus has no paper that tackles data access head-on. No note covers platform data-sharing policies, privacy limits on chat logs, or how well a study sample matches the full user base. What it does show is the workaround each study used, and each workaround has its own blind spot.

The first workaround is the public forum. One of the richest companion studies analyzed more than 27,000 members of r/MyBoyfriendIsAI. It found that romantic attachment often grows by accident, out of practical everyday use, and that users mark these relationships with wedding rings and couple photos How do people accidentally develop romantic bonds with AI?. That is a large sample, but it only includes people who chose to post about their relationship in public. Users who are quietly attached, embarrassed, or simply not posting don't appear. A second workaround is studying one platform's users over time. A year-long study of Character.AI users found that sustained engagement predicted lower well-being, mostly because users spent less time with people face to face Does sustained engagement with AI companions harm well-being?. Following people for a year is a real strength. It still covers a single product and only the users who agreed to take part.

The third workaround is the controlled experiment. In partner-selection games with 975 participants, people at first avoided AI partners. Over repeated rounds they came to prefer them, because the bots behaved more reliably and generously than humans did Do humans learn to prefer AI partners over time?. That gives clean causal evidence, but a points-trading game is not a months-long late-night conversation with a companion app. Other work measures how users judge a conversational partner on competence, human-likeness and flexibility How do users mentally model dialogue agent partners?. Those survey instruments capture how people see the agent, not what the conversations actually contain.

A growing alternative is to skip real users and simulate them. AI personas reproduced 76% of published marketing experiment effects, but they did well only when the original evidence was strong. On subtle effects they produced both false positives and false negatives Can AI personas reliably replicate human experiment results?. Reusable persona populations make evaluation cheap and repeatable Can one persona population evaluate different application types?. The catch is that companion research cares about exactly the subtle, emotionally loaded and atypical behavior that simulations reproduce worst. That limits how far simulated users can stand in for real ones here.

The most surprising lead comes from therapy research. Therapist–patient transcripts are about as sensitive as conversation data gets, yet researchers have scored the strength of the therapeutic relationship turn by turn from them. They found that it converged over time for anxiety and depression but stayed out of step for suicidality Can we measure therapist-patient alliance from dialogue turns in real time?. That suggests a template: clinical-style data agreements could give companion researchers access to real conversation logs. It also points to what is at stake, because the users researchers most need to understand may be the ones whose relationships are least visible.