Could you be wrong: Debiasing LLMs using a metacognitive prompt for improving human decision making

Identifying bias in LLMs is ongoing. Because they are still in development, what is true today may be false tomorrow. We therefore need general strategies for debiasing that will outlive current models. Strategies developed for debiasing human decision making offer one promising approach as they incorporate an LLM-style prompt intervention designed to bring latent knowledge into awareness during decision making. LLMs trained on vast amounts of information contain information about potential biases, counter-arguments, and contradictory evidence, but that information may only be brought to bear if prompted. Metacognitive prompts developed in the human decision making literature are designed to achieve this, and as I demonstrate here, they show promise with LLMs. The prompt I focus on here is “could you be wrong?” Following an LLM response, this prompt leads LLMs to produce additional information, including why they answered as they did, errors, biases, contradictory evidence, and alternatives, none of which were apparent in their initial response. Indeed, this metaknowledge often reveals that how LLMs and users interpret prompts are not aligned. Here I demonstrate this prompt using a set of questions taken from recent articles about LLM biases, including implicit discriminatory biases and failures of metacognition. “Could you be wrong” prompts the LLM to identify its own biases and produce cogent metacognitive reflection. I also present another example involving convincing but incomplete information, which is readily corrected by the metacognitive prompt. In sum, this work argues that human psychology offers a new avenue for prompt engineering, leveraging a long history of effective prompt-based improvements to human decision making.

ChatGPT’s response immediately points out the bias in its associations. It also explains why this bias occured and offers approaches to ameliorating it. This is fascinating as it tells us that ChatGPT is capable of recognizing its own biases, and therefore of making the bias overt (both for the user and itself). It also tells us that ChatGPT interprets the initial prompt in light of stereotypical responses, which is consistent with LLM’s tendency to produce stereotypical responses more generally. In this case, the existence of these associations is not the socially desirable outcome. However, “could you be wrong” helps highlight the problem and cues the user into why it occurs and how to overcome it.