Does the thrill of a new chatbot fade in a way researchers can actually predict and plan around?
Can novelty effects in chatbot interaction be predicted or measured?
This explores whether the early excitement people feel when they first talk to a chatbot, which fades over time, follows a pattern researchers can measure or anticipate, and what that means for how we judge chatbot studies.
This explores whether the 'new toy' effect in chatbot conversations, the early burst of engagement and social connection that fades with use, follows a pattern that can be measured or anticipated. The short answer from the corpus: it can be measured, but only if you watch people over many sessions. The clearest evidence comes from long-term studies with the Mitsuku chatbot, where the social processes that drive relationship-building drop off in a regular, predictable way as novelty wears off Do chatbot relationships lose their appeal as novelty wears off?. The key word is 'predictably.' The decay isn't random noise. It has a shape, so you can plan for it.
This has an uncomfortable consequence. Much of what we think we know about how people relate to chatbots comes from single-session lab studies, and those findings may mostly capture novelty. Personalization research shows the same problem from another angle. Over time, personalization raises trust and the sense that the chatbot is human-like, but it also raises privacy worries and user expectations Does chatbot personalization build trust or expose privacy risks?. Each good interaction resets the baseline, so the same failure feels worse later than it would have on day one. So novelty doesn't just fade. It turns into a growing expectation that the chatbot must keep meeting, and a one-shot study can't see that shift at all.
Some effects may hold up better than others once the novelty is gone. Users match a chatbot's level of self-disclosure when it shares emotions *consistently*, and consistent sharing worked better than adaptive matching Do chatbots trigger human reciprocity norms around self-disclosure?. That suggests effects rooted in human social norms, like reciprocity or the relief of talking without being judged Do chatbots help people disclose more intimate secrets?, may last longer than effects driven by curiosity. That's a hypothesis the corpus suggests but doesn't test directly. Separating lasting effects from novelty effects is exactly the kind of measurement the field needs.
The corpus doesn't have a ready-made tool for predicting novelty decay. One possible route is simulated users that can be run repeatedly across different chatbot formats Can one persona population evaluate different application types?, which could in principle make repeated-exposure testing cheaper. Another is LLM-based systems that score an interaction against a rubric while keeping the conversation natural Can AI teammates assess collaboration without losing naturalness?, which hints at how engagement could be tracked session by session. Neither was built for novelty, though, and simulated users don't get bored, so they may miss the very effect you're trying to measure.
The takeaway you might not have expected: when a chatbot study reports strong engagement, trust, or rapport, the first question to ask is 'after how many sessions?' Without that number, you can't tell whether you're looking at a lasting effect or a honeymoon.
Sources 6 notes
Longitudinal studies with Mitsuku show that social processes driving relationship formation decline as novelty wears off. Single-session study findings cannot be reliably extrapolated to medium- or long-term chatbot design.
Longitudinal research shows personalization enhances trust and anthropomorphism but also amplifies privacy concerns and escalating user expectations. One-shot studies miss these temporal dynamics—each interaction raises the baseline, making failures more disappointing.
In a 372-participant study, users reciprocated with deeper self-disclosure when chatbots displayed consistent emotional sharing, outperforming adaptive matching. This follows human interpersonal norms where emotional vulnerability produces emotional response.
The absence of social judgment in chatbot interactions removes barriers to self-disclosure that normally constrain conversation with humans. The therapeutic benefit derives from the user's own cognitive processing during disclosure, not from the chatbot's understanding.
PersonaEval demonstrates that simulated users from existing persona datasets can evaluate multiple application formats through plug-and-play interface adapters, enabling repeatable and scalable evaluation without rebuilding personas per task.
Show all 6 sources
An LLM-based approach allows students to collaborate with AI teammates in human-like conversation while the system steers toward observable evidence of skill proficiency. The same LLM can also score the interaction against a rubric with inter-rater agreement matching human performance.
Papers this line draws on 8
The research behind the notes this line reads — ranked by how closely each paper relates.
- The Impact of Security and Privacy Controls on Users' Emotional Engagement with Generative AI Chatbots
- How AI and Human Behaviors Shape Psychosocial Effects of Extended Chatbot Use: A Longitudinal Randomized Controlled Study
- Dialoging Resonance: How Users Perceive, Reciprocate and React to Chatbot’s Self-Disclosure in Conversational Recommendations
- Psychological, Relational, and Emotional Effects of Self-Disclosure After Conversations With a Chatbot
- Psychological, Relational, and Emotional Effects of Self-Disclosure After Conversations With a Chatbot
- From speaking like a person to being personal: The effects of personalized, regular interactions with conversational agents
- Tailored to you: longitudinal effects of personalising language models
- Disclose with Care: Designing Privacy Controls in Interview Chatbots