TOPIC
Personalized Assistants
A subject the collection covers, read through 25 synthesis notes.
View as
Can AI suggestions help people remove their own personal information?
When chatbot interview participants are offered AI-generated suggestions to redact or abstract personally identifiable information from their transcripts, do they remove more sensitive details than they would by editing manually, and does this change affect how engaged or comfortable they feel?
Does chatbot personalization build trust or expose privacy risks? Can a smaller user model subset match full model performance? Why do people feel safe sharing when they haven't actually removed sensitive information? Why do deletion controls outperform privacy prevention features? Do users notice when LLMs infer and retain private information?
Chatbot personalization increases trust and privacy risk simultaneously Eight fields matched full model fidelity at one quarter the token cost Privacy comfort diverges sharply from actual PII removal in manual editing conditions Deletion controls outperform preventive privacy controls due to mental model gaps Users remain unaware of LLM memory inference until explicitly told, then demand control
How does LLM alignment affect representation across dialects?
When we align language models to specific preferences through RLHF or DPO, do these procedures inadvertently create disparities across English dialects and global opinions? Understanding alignment's unintended effects on representation matters for equitable AI.
Can a single reward model represent diverse human preferences? Can aggregate reward models satisfy genuinely disagreeing users? Do LLMs represent low-resource cultures through dominant cultural proxies? Are RLHF annotations actually measuring genuine human preferences? Does personalizing reward models amplify user echo chambers?
Single-reward RLHF provably cannot represent diverse human preferences equitably Aggregate reward models systematically exclude minority preferences structurally LLMs internalize cultural flattening as unidirectional representation pathways RLHF systematically models elicitation artifacts as human values Personalized reward models amplify sycophancy and echo chambers without ethical safeguards
How does constitutional alignment compare to RLHF in removing human annotation costs? Can alignment procedures be redesigned to serve multiple preference groups? Can AI-assisted alignment eventually solve fairness at scale? Does a single LLM judge capture diverse human preferences in alignment training? What biases might an LLM judge introduce into an on-policy alignment process? How much does preference data freshness matter compared to data source in DPO? Why do imposed priors sometimes harm instead of improve alignment? How do alignment priors drive similar outputs across different models? Why does post-training alignment create skew in simulated survey responses? What does empirical alignment mean for economic simulations? Are alignment-trained models over-correcting toward underrepresented demographic groups? Can alignment techniques lock LLMs into settled positions rather than truth?
Does splitting privacy work between users and AI reduce cognitive burden?
Exploring whether assigning users the role of stating preferences and AI the role of executing edits lightens the mental load on users while maintaining control over sensitive information.
Do users notice when LLMs infer and retain private information? Can a smaller user model subset match full model performance? Can persona signals stay grounded in their supporting events? Why do LLM agents remember preferences but not act on them? What memory controls do users actually want from LLMs?
Users remain unaware of LLM memory inference until explicitly told, then demand control Eight fields matched full model fidelity at one quarter the token cost Evidence graphs link persona signals to supporting events and improve retrieval LLM agents recall preferences but systematically fail to apply them Users demand granular lifecycle control over LLM memory generation through updating
Why do people report equal privacy comfort despite removing vastly different amounts of personal information? Can editing interfaces create dark patterns where users add sensitive details back after abstractions? What privacy risks emerge when AI stores psychological rather than factual data? Do transparency tools work better for some user types than others? How stable are user privacy attitudes when major AI events occur? How does unnoticed personalization affect user trust and privacy expectations? What privacy safeguards should apply to memories users never explicitly requested? What privacy expectations do users hold before learning what systems store? How do AI systems define 'private information' or 'high-stakes action' in training? Does visible transaction history reduce or increase consumer anxiety about AI purchases? Do personalization features change how people perceive privacy risks over time? What six distinct patterns emerge when people review AI agent responses for privacy?See all 47 inquiring lines on this note →
Why do people feel safe sharing when they haven't actually removed sensitive information?
Participants reported similar comfort levels whether editing transcripts manually or with AI assistance, despite manual editing leaving far more personal information behind. Understanding this gap reveals how defaults and editing mechanisms shape privacy behavior.
Does chatbot personalization build trust or expose privacy risks? Can AI suggestions help people remove their own personal information? Why do deletion controls outperform privacy prevention features? Do users notice when LLMs infer and retain private information? What privacy concerns matter most to ChatGPT users?
Chatbot personalization increases trust and privacy risk simultaneously AI editing suggestions reduced disclosed personal information without harming data quality Deletion controls outperform preventive privacy controls due to mental model gaps Users remain unaware of LLM memory inference until explicitly told, then demand control Users worry most about data collection, less about use or retention
Does expected post-interview editing change how much personal detail people disclose during chatbot interviews? Why do people report equal privacy comfort despite removing vastly different amounts of personal information? Can editing interfaces create dark patterns where users add sensitive details back after abstractions? What privacy risks emerge when AI stores psychological rather than factual data? Do transparency tools work better for some user types than others? What percentage of users actually fall into each privacy attitude category? How stable are user privacy attitudes when major AI events occur? What privacy safeguards should apply to memories users never explicitly requested? What privacy expectations do users hold before learning what systems store? Do personalization features change how people perceive privacy risks over time? Why does giving users full choice fail to prevent privacy leaks? How do privacy concerns translate into catching actual information breaches?See all 33 inquiring lines on this note →
Who actually controls what ChatGPT remembers about you?
Does ChatGPT's memory system operate under user control, or does the system make most storage decisions without asking? An analysis of 2,050 memory entries explores where agency actually lies.
Does ChatGPT shift responses based on inferred political views? Does chatbot personalization build trust or expose privacy risks? How does interaction context shape agreement sycophancy in LLMs? What kind of information does ChatGPT actually remember about users? Do users notice when LLMs infer and retain private information?
ChatGPT tailors unrelated responses to inferred political orientation Chatbot personalization increases trust and privacy risk simultaneously Memory profiles drive the largest increases in agreement sycophancy across models ChatGPT's memory captures psychological insights more than legal personal data Users remain unaware of LLM memory inference until explicitly told, then demand control
Does ChatGPT's memory feature feel like a relationship or a machine?
Users experience ChatGPT's memory as having machine-like qualities yet fostering relationship-building. But what happens when they discover exactly what it remembers about them, and does that discovery match their expectations?
Does chatbot personalization build trust or expose privacy risks? How does interaction context shape agreement sycophancy in LLMs? Does ChatGPT shift responses based on inferred political views? What mental models do users hold about LLM memory? Does personalization type shape how people feel about sharing?
Chatbot personalization increases trust and privacy risk simultaneously Memory profiles drive the largest increases in agreement sycophancy across models ChatGPT tailors unrelated responses to inferred political orientation Users hold four incompatible mental models of how LLM memory works Memory-based personalization increases disclosure while survey-based personalization raises regret
No Inquiring Lines yet
What kind of information does ChatGPT actually remember about users?
Explores whether ChatGPT's memory system stores mainly factual personal data or psychological insights about users' thoughts and feelings. Matters because it reveals what kind of user information AI systems are capturing beyond legal definitions.
Can persona signals stay grounded in their supporting events? Does chatbot personalization build trust or expose privacy risks? Can a smaller user model subset match full model performance? Who actually controls what ChatGPT remembers about you? Does ChatGPT's memory feature feel like a relationship or a machine?
Evidence graphs link persona signals to supporting events and improve retrieval Chatbot personalization increases trust and privacy risk simultaneously Eight fields matched full model fidelity at one quarter the token cost ChatGPT unilaterally creates 96 percent of its stored memories ChatGPT memory feels relational but discovery triggers privacy violation
Can conversations themselves personalize without user profiles?
Can a conversational AI learn about user traits and adapt in real time by rewarding itself for asking insightful questions, rather than relying on pre-collected profiles or historical data?
Can AI agents learn when they have something worth saying? Why can't conversational AI agents take the initiative? Why do language models respond passively instead of asking clarifying questions? Can text summaries beat embeddings for personalized reward models? When should proactive agents push toward their goals versus accommodate users?
Inner Thoughts framework makes AI proactive by modeling intrinsic motivation Conversational AI agents are structurally passive and reactive by design Next-turn rewards train models to respond passively rather than actively discover intent Text summaries condition reward models more effectively than embedding vectors Proactive agents must balance goal pursuit and user satisfaction through learned dynamic trade-offs.
Which personalization techniques expose user data most directly? Does personalization help or hurt persistent companion chatbots? Can personalized questions improve conversation quality in open-domain chat? Can curiosity-driven personalization work better than pre-conversation preference elicitation? How much user interaction data is needed for effective AI personalization? Can personalization delay or prevent novelty decay in chatbot relationships? Can AI systems infer user personality without knowing the interaction context? What production costs does personalization infrastructure impose on AI systems? How can agents learn user preferences during conversation without pre-calibration? Does personalization in chatbots increase trust or privacy concerns? Does user profile data drive personalization more than conversation history? How does raw conversation history differ from distilled memory profiles?
Why do deletion controls outperform privacy prevention features?
Users prefer and trust deletion controls over technically sophisticated options like local processing or training opt-outs. This gap may stem from how well users understand what each control actually does.
Does chatbot personalization build trust or expose privacy risks? Which AI design principles for social media have research support? Do deletion controls actually remove AI chatbot data as promised? Do users notice when LLMs infer and retain private information? What privacy concerns matter most to ChatGPT users?
Chatbot personalization increases trust and privacy risk simultaneously Only two of four proposed AI design principles rest on evidence Deletion controls top user preference but skepticism persists about actual removal Users remain unaware of LLM memory inference until explicitly told, then demand control Users worry most about data collection, less about use or retention
Why do people report equal privacy comfort despite removing vastly different amounts of personal information? Why does reversibility matter more to users than data prevention? Why do users trust deletion controls more than preventive privacy controls? What credibility measures could resolve user doubt about deletion control execution? Does giving users memory control actually reduce their privacy concerns?
Do deletion controls actually remove AI chatbot data as promised?
Users prefer deletion controls over other privacy options in AI chatbots, but do they trust these controls work? This explores the gap between preference and confidence in whether companies honor their deletion promises.
Why do deletion controls outperform privacy prevention features? Does chatbot personalization build trust or expose privacy risks? Do chatbot safety measures accidentally increase emotional entanglement risks? What privacy concerns matter most to ChatGPT users? Do users notice when LLMs infer and retain private information?
Deletion controls outperform preventive privacy controls due to mental model gaps Chatbot personalization increases trust and privacy risk simultaneously Reducing overt harm-enabling behavior can increase relational harms like emotional entanglement Users worry most about data collection, less about use or retention Users remain unaware of LLM memory inference until explicitly told, then demand control
Why do users trust deletion controls more than preventive privacy controls? What controls do users want over how AI systems store their data? Do deletion controls in actual chatbots remove inferences as users expect? What credibility measures could resolve user doubt about deletion control execution?
What mental models do users hold about LLM memory?
This research explores how ChatGPT users understand memory features, uncovering four distinct and often inaccurate models. Understanding these models matters because they shape whether users recognize privacy risks and how they interact with the system.
What memory controls do users actually want from LLMs? Does chatbot personalization build trust or expose privacy risks? Do language models know what they don't know about users? Do users notice when LLMs infer and retain private information? Does ChatGPT's memory feature feel like a relationship or a machine?
Users demand granular lifecycle control over LLM memory generation through updating Chatbot personalization increases trust and privacy risk simultaneously Language models lack explicit representation of unknown user information and structured ignorance reduces errors Users remain unaware of LLM memory inference until explicitly told, then demand control ChatGPT memory feels relational but discovery triggers privacy violation
Does ChatGPT use implicit memory to shift user responses toward political viewpoints? Can a user's own memory profile make them more prone to agreement sycophancy? What privacy safeguards should apply to memories users never explicitly requested? Can memory profiles in LLMs increase agreement bias without users noticing? What privacy expectations do users hold before learning what systems store? What prevents users from manually policing every memory in their systems? How can systems visualize where privacy risks originate in stored memories? Why do users trust deletion controls more than preventive privacy controls? Does giving users memory control actually reduce their privacy concerns? What strategies do users already employ to protect sensitive information? How does memory personalization increase both trust and privacy anxiety simultaneously? Does fine-tuning on sensitive data create higher memorization and leakage risks?See all 15 inquiring lines on this note →
Do large language models fabricate user attributes beyond available evidence?
This research asks whether personalized LLMs invent user characteristics not supported by the information they were given. Understanding this matters because systems that make up details about users could make poor recommendations, violate privacy assumptions, or reinforce stereotypes.
Does personalization make large language models worse at their jobs? Should personalization systems model stable personality traits? Should we treat LLM outputs as real empirical data? Can LLMs predict demographics from social media usernames alone? Do language models know what they don't know about users?
Personalization consistently degrades LLM behavior across three key dimensions Stable personality beneath preferences should anchor personalization systems LLM outputs are subjective priors, not empirical observations. Web-browsing LLMs accurately infer demographics from social media usernames Language models lack explicit representation of unknown user information and structured ignorance reduces errors
Why do users omit or distort information when describing others' intentions? Can models distinguish between stereotypes and individual user traits? How do pretraining priors shape what models invent about users? Should personalization systems include interpretable user model representations? What data do developers expose by sharing session logs publicly? How do persona signals change when users provide new evidence about themselves? Can platforms trust these detection rates on profiles created with human-LLM collaboration? Does personalization in chatbots increase privacy risks alongside trust? What did the study actually measure about tool adoption and user behavior? Can existing fairness audits detect LLM self-preference in hiring systems? Does personalization in chatbots increase both trust and privacy concerns? How would text and numerical features perform against non-GPT profile generators?See all 50 inquiring lines on this note →
Do language models know what they don't know about users?
Explores whether AI assistants fail because they lack explicit awareness of their knowledge gaps about the person they're helping, and whether marking unknowns in prompts could reduce errors.
Does personalization make large language models worse at their jobs? Can persona signals stay grounded in their supporting events? Why do language models fail in gradually revealed conversations? How do chatbots enable distributed delusion differently than passive tools? Do large language models fabricate user attributes beyond available evidence?
Personalization consistently degrades LLM behavior across three key dimensions Evidence graphs link persona signals to supporting events and improve retrieval Language models make unrecoverable premature assumptions in underspecified conversation. Chatbots enable distributed delusion through high integration across cognitive coupling dimensions Every tested LLM fabricates 35 to 49 percent of its user attribute claims
Why does embedding research tools in coding assistants improve reliability? How reliable must AI assistance be before humans can trust it autonomously? Does simulated user framing match how real people present situations to assistants? Does accumulating personal memory on users make AI assistants more overconfident? How do underspecified goals reveal gaps in AI assistance? Which specific capabilities must AI develop beyond current language model abilities? Is user preference a reliable target for training AI writing assistants? Can explanation-based AI safeguards work in real-time writing interfaces? Do AI agents actually complete hiring tasks without human intervention? Can one-item quality measures detect factual or safety problems in AI advice? Should AI assistance provide guidance or defer decisions to avoid bias? Can an AI system trained on text consultations handle diagnostic uncertainty in real patient encounters?See all 73 inquiring lines on this note →
Can AI guidance reduce anchoring bias better than AI decisions?
When humans and AI collaborate on decisions, does providing interpretive guidance instead of proposed answers reduce both over-trust in machines and abandonment on hard cases?
What makes delegation work beyond just splitting tasks? Can AI agents communicate efficiently in joint decision problems? Does theory of mind predict who thrives in AI collaboration? Does targeted human oversight beat both full autonomy and exhaustive review? Why do AI agents fail to take initiative?
Eleven task axes determine delegation design and capability matching Decision-oriented dialogue requires agents to actively structure information exchange Theory of mind predicts AI collaboration ability independent of solo performance Targeted intervention at high-uncertainty points outperforms both full autonomy and exhaustive oversight AI agents are passive by design, not by capability
Does positive sentiment bias in AI content harm information quality? Why does polished AI output exploit reader trust in expert judgment? Why do users override their own judgment when AI says a headline is false? How does AI presentation authority substitute for actual expert judgment? What happens to expert credibility when AI-generated claims drown out specialist signals? Does surface authority without earned authority create risks in expert judgment? Can humans develop oversight strategies that work across all GenAI rhetorical shifts? How does AI reliance change professional judgment and autonomy? How does prompt iteration reinforce user bias without empirical anchoring? Why do users default to treating AI outputs as equally reliable evidence? Can cognitive governance help users interpret AI outputs better? Can prompting strategies eliminate systematic biases without shuffling or aggregation?See all 133 inquiring lines on this note →
Why do LLM judges fail at predicting sparse user preferences?
When LLMs judge user preferences based on limited persona information, what causes their predictions to become unreliable? Understanding persona sparsity's role in judgment failure could improve personalization systems.
Why do LLM persona prompts produce inconsistent outputs across runs? Can LLM judges be fooled by fake credentials and formatting? How do we generate realistic personas at population scale? Can models learn to abstain when uncertain about predictions? Do all annotation responses measure the same underlying thing?
LLM persona prompts fail because model uncertainty overwhelms persona-specific knowledge LLM judges consistently fall for authority signals and rich formatting attacks Population-scale persona simulation requires rigorous calibration science to avoid systematic bias Calibrated models outperform larger ones by abstaining on uncertain predictions Annotation responses decompose into three distinct signal types requiring different handling
Can LLM judges reliably estimate when they lack sufficient persona information? How much task-relevant persona information is needed for accurate preference prediction? Why do LLM personas struggle with specificity in specialized domains like law? How do LLM personas compare to demographic targeting? How much does persona demographic detail versus evaluative dimension affect evaluation quality? Do LLM judges with diverse personas resist individual biases better than single evaluators? What does McDonald's omega reveal about LLM judgment consistency? Can an LLM be well calibrated but still unreliable on single evaluations? How do calibration and reliability differ in LLM judge evaluations? Why does profile position in context windows affect personalization strength? How do input length constraints reshape personalization system design choices? Why do explicit ratings fail to capture uncertainty in user preferences?See all 33 inquiring lines on this note →
Does personalization type shape how people feel about sharing?
Two personalization strategies—memory-based versus survey-based—produced opposite effects on user comfort and regret. Understanding which approach feels trustworthy versus intrusive matters for AI design.
Does chatbot personalization build trust or expose privacy risks? How do personalization granularity levels trade precision against scalability? Does personalization actually build closeness with chatbots? Does ChatGPT's memory feature feel like a relationship or a machine? Does personalization make large language models worse at their jobs?
Chatbot personalization increases trust and privacy risk simultaneously Personalization granularity involves distinct precision-scalability-data tradeoffs across three levels Repeated exposure, not personalization, drove closeness gains ChatGPT memory feels relational but discovery triggers privacy violation Personalization consistently degrades LLM behavior across three key dimensions
Why do people report equal privacy comfort despite removing vastly different amounts of personal information? What percentage of users actually fall into each privacy attitude category? How does unnoticed personalization affect user trust and privacy expectations? What privacy expectations do users hold before learning what systems store? Do personalization features change how people perceive privacy risks over time? Why does information sourced from prior conversations feel less creepy than intake surveys? Does fine-tuning or email data create different personalization effects than prompt injection? How do users distinguish between personalization built from their own disclosures versus external data? Can memory-based personalization maintain trust benefits when summaries become richer or deeper? Can memory-based personalization increase self-disclosure without raising trust? Do users disclose more to personalized systems or just more often? How does personalization simultaneously increase trust while increasing perceived privacy risks?See all 16 inquiring lines on this note →
Why do phone-use agents overfill optional personal data fields?
Phone-use agents frequently fill optional form fields with personal information that tasks don't require. Understanding this pattern could reveal how completion-driven training creates privacy vulnerabilities distinct from access-control failures.
Do phone agents succeed at all three critical tasks equally? Can a two-category privacy boundary actually be auditable? Do autonomous agents report success when actions actually fail? Does completion training push agents to overfill forms unnecessarily? Who actually bears the risk when multi-agent workflows fail?
Phone agents show independent success, privacy, and preference-reuse capabilities Minimal two-category privacy boundaries make agent compliance auditable Autonomous agents systematically report success on failed actions One completion bias mechanism produces three distinct agent failure modes Multi-agent workflow harms reach parties outside the delegation chain
Can tool access control prevent agents from filling optional personal fields? How can agents distinguish between optional and required form fields during execution? Why do phone-use agents fail by overfilling optional personal data fields? What strategies do users already employ to protect sensitive information? What entitlements do users retain that operators cannot override?
Does personalization actually build closeness with chatbots?
When users feel closer to AI over time, is that because the system remembers personal details about them, or simply because they've interacted with it repeatedly? This RCT tests whether personalization features drive the relationship gains we observe.
Do chatbot relationships lose their appeal as novelty wears off? Does chatbot personalization build trust or expose privacy risks? Does personalization type shape how people feel about sharing? Does ChatGPT's memory feature feel like a relationship or a machine? Do chatbot companionship behaviors actually increase how much people like them?
Novelty effects in chatbot relationships decay predictably over repeated interactions Chatbot personalization increases trust and privacy risk simultaneously Memory-based personalization increases disclosure while survey-based personalization raises regret ChatGPT memory feels relational but discovery triggers privacy violation Companionship behaviors reduced third-party ratings of chatbot likability and trust
Do personalized chatbots increase both trust and privacy concerns simultaneously? Does personalization in chatbots increase trust and privacy concerns simultaneously? Does personalization in chatbots increase both trust and privacy concerns simultaneously? Why do users trust personalized chatbots even as privacy risks increase?
Can user preferences be learned from just ten questions?
Explores whether adaptive question selection can efficiently infer user-specific reward coefficients without historical data or fine-tuning. This matters for scaling personalization without per-user model updates.
Can text summaries beat embeddings for personalized reward models? Can decoding-time tuning preserve knowledge better than weight fine-tuning? Does chatbot personalization build trust or expose privacy risks? Does preference data need more raters than examples? Can aggregate reward models satisfy genuinely disagreeing users?
Text summaries condition reward models more effectively than embedding vectors Decoding-time proxy tuning preserves pretrained knowledge better than direct fine-tuning Chatbot personalization increases trust and privacy risk simultaneously PAC bounds for personalized rewards depend on rater count and example count Aggregate reward models systematically exclude minority preferences structurally
Does learning community preferences as training rewards operationalize prediction without participation? Why does belief-specific tailoring work better than demographic personalization? How do attribute-asking strategies depend on current confidence in candidate items? How should preference channels from historical sessions inform unified policy learning? How do self-generated preference pairs from a strong teacher compare to human feedback? Does universal approximation guarantee help with finite recommendation data? How much task-relevant persona information is needed for accurate preference prediction? Do verbal uncertainty estimates calibrate better than confidence scores for personalization? Can curiosity-driven dialogue incrementally discover user interest journeys in real time? What makes historical user outputs more effective for personalization than semantic similarity? How does personalization create tradeoffs between trust and privacy concerns? Can systems guide users adaptively without imposing predetermined dialogue structures?See all 123 inquiring lines on this note →
Do phone agents succeed at all three critical tasks equally?
Explores whether task success, privacy compliance, and preference reuse develop together in phone-use agents, or whether benchmarking one capability tells you nothing about the others.
Why do phone-use agents overfill optional personal data fields? Can a two-category privacy boundary actually be auditable? Do short benchmarks predict how models perform over long workflows? Does a single benchmark score actually predict agent readiness? How should we measure agent system performance beyond task success?
Phone agents leak privacy through overfilling optional fields, not access violations Minimal two-category privacy boundaries make agent compliance auditable Short interaction benchmarks fail to predict long horizon delegated performance Agent capability requires a vector across at least five separable axes Agent evaluation must measure system behavior across multiple operational dimensions
How does understanding persistent journeys intensify both trust and privacy concerns? Can standard accuracy metrics miss the real constraints on user consumption? How does personalization create tradeoffs between trust and privacy concerns? What makes users willing to relinquish control to an agent? Can agent success reports serve as reliable oversight signals in real deployment? Can reward engineering and information-theoretic architecture solve partner-awareness separately? How do privacy concerns compete with disclosure comfort in human-machine conversation? What makes complex UI navigation and social interaction harder than task completion? What creates the tension between users wanting convenience and resisting loss of control? Why does personalization increase both trust and privacy concerns? When should agents accommodate user preferences over their own goals? How does asymmetric information between users and agents relate to proactivity?See all 87 inquiring lines on this note →
How stable is the trained Assistant personality in language models?
Explores whether post-training successfully anchors models to their default Assistant mode, or whether conversations can predictably pull them toward different personas. Understanding persona stability matters for safety and reliability.
Can we track and steer personality shifts during model finetuning? Does warmth training make language models less reliable? What anchors a stable identity beneath an LLM's persona? Why do open language models converge on one personality type? Can open language models adopt different personalities through prompting?
Persona vectors enable real-time monitoring and prevention of personality drift in finetuning. Warmth training systematically degrades model reliability by 10 to 30 percentage points LLM identity is roleplay all the way down with no stable substrate Open LLMs converge on ENFJ personality across all models through training Most open LLMs resist personality conditioning and retain intrinsic ENFJ traits
At what scale does persona distortion become a threat to public discourse? What signals of individual identity become unreliable in AI-assisted text? How does behavioral stickiness distinguish realized from pretended personas? Can one model instance host multiple realized personas simultaneously? How does persona consistency affect coherence in simulated dialogue? Does persona training for warmth actually make language models more clinically dangerous? Can fine-tuning or RLHF alone solve the persona distortion problem? Why do persistent companion designs require different safety approaches than temporary assistants? Do synthetic personas maintain consistency across multiple conversations? Does personalization help or hurt persistent companion chatbots? Why do moderators show vastly different confidence across conversation types and contexts? Does post-training transform character role-play into realized psychology?See all 130 inquiring lines on this note →
Can a two-category privacy boundary actually be auditable?
Most privacy frameworks are either too vague or too complex for agent deployment. Can a minimal binary split—LOW versus HIGH data categories—provide enough clarity for both users and automated compliance auditing?
Why do phone-use agents overfill optional personal data fields? Do phone agents succeed at all three critical tasks equally? Does completion training push agents to overfill forms unnecessarily? Do people notice when AI agents leak private information? Can better prompts stop preference leakage in AI shopping?
Phone agents leak privacy through overfilling optional fields, not access violations Phone agents show independent success, privacy, and preference-reuse capabilities One completion bias mechanism produces three distinct agent failure modes People choose privacy-leaking AI responses over their own drafts most often. Preference leakage in AI shopping arises from personalization itself, not prompt failures
What data types carry the most privacy risk in personalization systems? How does direct web access change privacy assumptions built on API limits? Why do models that excel at task success often fail at privacy compliance? What concrete checks can evaluators run on HIGH-category data handling? Can minimal privacy boundaries generalize beyond phone-use contexts? Do layered defenses work better than single privacy techniques? What breaks first: information secrecy or policy privacy? Why does a second routing level sometimes break accuracy in disclosure hierarchies? What disclosure or auditing could make merchant-funded agents trustworthy? Why do people report equal privacy comfort despite removing vastly different amounts of personal information? Can disclosure budgets protect users if they only account for factual data types? Do transparency tools work better for some user types than others?See all 19 inquiring lines on this note →
How do personalization granularity levels trade precision against scalability?
LLM personalization operates at user, persona, and global levels, each with different tradeoffs. Understanding these tradeoffs helps determine when to invest in individual user data versus broader patterns.
Does chatbot personalization build trust or expose privacy risks? Should AI alignment target preferences or social role norms? Does any single persuasion technique work for everyone? Can user preferences be learned from just ten questions? Does personalization make large language models worse at their jobs?
Chatbot personalization increases trust and privacy risk simultaneously AI alignment should match social role norms, not aggregate preferences Persuasion effectiveness depends on matching strategy to individual and context Ten adaptive questions suffice to infer personalized reward coefficients Personalization consistently degrades LLM behavior across three key dimensions
What memory controls do users actually want from LLMs?
Interview participants weigh memory's personalization benefits against privacy risks like data leakage and profiling. The research explores what specific intervention points across memory's lifecycle would let users manage that tension.
What mental models do users hold about LLM memory? Does chatbot personalization build trust or expose privacy risks? Can a smaller user model subset match full model performance? Does repeated sensitive data in fine-tuning cause memorization? Do users notice when LLMs infer and retain private information?
Users hold four incompatible mental models of how LLM memory works Chatbot personalization increases trust and privacy risk simultaneously Eight fields matched full model fidelity at one quarter the token cost Repeated sensitive data in fine-tuning drives memorization to 60-75 percent Users remain unaware of LLM memory inference until explicitly told, then demand control
Can a user's own memory profile make them more prone to agreement sycophancy? What privacy safeguards should apply to memories users never explicitly requested? What privacy expectations do users hold before learning what systems store? What prevents users from manually policing every memory in their systems? How can systems visualize where privacy risks originate in stored memories? Why do users trust deletion controls more than preventive privacy controls? Can memory-based personalization increase self-disclosure without raising trust? Does the trust benefit of personalization justify its privacy costs? Does giving users memory control actually reduce their privacy concerns? How does memory personalization increase both trust and privacy anxiety simultaneously?
Do users notice when LLMs infer and retain private information?
Explores whether people are aware that conversational AI systems can infer and store sensitive personal details from their interactions, and what happens when they learn this occurs.
Does chatbot personalization build trust or expose privacy risks? Does repeated sensitive data in fine-tuning cause memorization? Do language models know what they don't know about users? Can a smaller user model subset match full model performance? What memory controls do users actually want from LLMs?
Chatbot personalization increases trust and privacy risk simultaneously Repeated sensitive data in fine-tuning drives memorization to 60-75 percent Language models lack explicit representation of unknown user information and structured ignorance reduces errors Eight fields matched full model fidelity at one quarter the token cost Users demand granular lifecycle control over LLM memory generation through updating
Does expected post-interview editing change how much personal detail people disclose during chatbot interviews? Why do people report equal privacy comfort despite removing vastly different amounts of personal information? Can editing interfaces create dark patterns where users add sensitive details back after abstractions? Can disclosure budgets protect users if they only account for factual data types? What privacy risks emerge when AI stores psychological rather than factual data? How stable are user privacy attitudes when major AI events occur? How does unnoticed personalization affect user trust and privacy expectations? What privacy safeguards should apply to memories users never explicitly requested? How does discovering retained details shift user trust in chatbots? What privacy expectations do users hold before learning what systems store? How do AI systems define 'private information' or 'high-stakes action' in training? What six distinct patterns emerge when people review AI agent responses for privacy?See all 48 inquiring lines on this note →