Showing an AI a few examples seems like an easy way to get what you want — so why do non-experts still struggle to pick good ones?

This explores why people without technical training find it hard to pick and write good examples to show an AI what they want. One caveat up front: no note in the collection studies non-engineers using few-shot examples directly. But several nearby lines of research add up to a clear picture, and they suggest the problem isn't really about examples.

The first barrier sits underneath any prompting technique: writing. Nielsen estimates that only 10 to 20 percent of adults in rich countries write well enough to prompt AI effectively, based on literacy data Can most adults write prompts good enough for AI?. Few-shot prompting looks like a shortcut around this, since you show the model what you want instead of describing it. But choosing an example means knowing which features of it matter, and that is the same act of putting your intent into words, just hidden. A second problem goes deeper. Research on the 'gulf of envisioning' finds that people often don't know exactly what they want until they see options, because intent forms through interaction, not before it Why can't users articulate what they want from AI?. Asking someone for representative examples assumes they already have a clear target.

Here is the surprising part: picking good examples is hard even for algorithms. One line of work treats example selection as an experimental design problem, choosing the examples that most reduce the model's uncertainty. It beats the intuitive approach of picking examples that look similar to the task Can optimal experimental design improve few-shot example selection?. 'Pick ones that look like what I want' is exactly how a non-engineer would do it, and that turns out to be the weaker strategy. Prompt compression research adds another twist. Demonstrations can be compressed far more heavily than instructions or questions with little loss in accuracy Can prompts be compressed without losing model performance?. So the examples people work hardest on may matter less than the plain instruction around them.

Iteration doesn't fully rescue things either. When people refine prompts over and over, they pull the output toward what they already expected, so the result reflects the user's assumptions as much as the model's ability How much does the user shape what a model generates?. In research settings, this kind of ad hoc tinkering lets one person's bias shape the results and slowly bends the success criteria toward whatever the model happens to do well Does iterative prompt engineering undermine scientific validity?. A non-engineer swapping examples until the output 'looks right' is running the same loop with no check on it. There's also a hard ceiling. Examples can only activate knowledge the model already has, never teach it something new Can prompt optimization teach models knowledge they lack?. Some failures that look like 'bad examples' have nothing to do with the examples.

Taken together, the research suggests the fix is better interfaces, not better-trained users. Proposals include previewing likely output before you submit, so you can catch a mismatch early Can previewing AI output before submission reduce prompt iteration?. Another is having the AI offer options to react to, turning the task from 'describe what you want' into 'pick which is closer' Why can't users articulate what they want from AI?. There is also a framework that rates prompt quality on six dimensions drawn from communication theory Can we measure prompt quality independent of model outputs?. A framework like that could eventually give non-experts feedback on their examples before they ever run them.