Detecting Cognitive Distortions from Patient-Therapist Interactions
An important part of Cognitive Behavioral Therapy (CBT) is to recognize and restructure certain negative thinking patterns that are also known as cognitive distortions. This project aims to detect these distortions using natural language processing. We compare and contrast different types of linguistic features as well as different classification algorithms and explore the limitations of applying these techniques on a small dataset.
Emotional Reasoning: Believing “I feel that way, so it must be true”
Overgeneralization: Drawing conclusions with limited and often un negative experience.
Mental Filter: Focusing only on limited negative aspects and not the excessive positive ones.
Should Statements: Expecting things or personal behavior should be a certain way.
All or Nothing: Binary thought pattern. Considering anything short of perfection as a failure.
Mind Reading: Concluding that others are reacting negatively to you, without any basis in fact.
Fortune Telling: Predicting that an event will always result in the worst possible outcome.
Magnification: Exaggerating or Catastrophizing the outcome of certain events or behavior.
Personalization: Holding oneself personally responsible for events beyond one’s control.
Labeling: Attaching labels to oneself or others (ex: “loser”, “perfect”).
In particular, this research aims to answer the following questions:
Which type of NLP features is more suitable for cognitive distortion detection: semantic or syntactic? Simply put, to compare what is said and how is it said in the context of this task. And, how important is word order in this context?
How well do these NLP features and ML classification algorithms perform this task with a limited-sized dataset?
the annotators were asked to determine a dominant distortion for each of the entries, and an optional secondary distortion if it is too hard to determine a dominant distortion
On the other hand, some findings are more unexpected. The expectation with the “should statement” was to have a higher probability of having auxiliary verbs such as ‘should’, ‘must’, ‘ought to’ etc. However, the results show that should statement have a lower than average probability of having auxiliary verbs. An example of this distortion without using any of the words listed above could be “While others my age are busy with their jobs and life I am just wasting my time”.
Lines of inquiry this paper opens 21
Research framings built by reading the notes related to this paper — the questions it feeds into.
Can AI systems balance emotional competence with factual reliability? Why do LLM chatbots fail as independent therapeutic agents?- Can models succeed at mental health tasks without integrating multiple psychological traditions?
- What other therapy constructs could be measured from transcripts using this approach?
- What makes Beck's diagram effective for constraining simulated patient behavior?
- Can large language models actually deliver cognitive behavioral therapy techniques?
- Do problem-solving defaults in LLM therapists actually undermine therapeutic effectiveness?
- Do worksheet-based structured formats work as well as embodied agents for therapy?
- What makes clinical theory grounding more effective than pattern matching alone?
- Why do Llama models struggle with cognitively distorted user expressions in therapy?
- Can AI provide therapy without challenging users to confront cognitive distortions?
- Do self-correction and chain-of-thought prompting reduce hallucination rates?
- How do cognitive load dimensions interact with hallucination awareness in prompts?