Train a smaller AI on ChatGPT's answers and it picks up the confident tone instantly — but does it actually get smarter?

This explores whether a model trained to copy ChatGPT can pick up the voice without the abilities behind it. The short answer from the corpus is yes, and easily. When open models are fine-tuned on ChatGPT's outputs, they learn its confident, fluent, well-organized manner, and human evaluators often rate them as nearly as good. On tests of factual accuracy and on new kinds of tasks, though, the gap barely narrows. The imitators sound like ChatGPT but don't know what it knows. The researchers' conclusion is that the ceiling is set by the base model's underlying capability, and fine-tuning on another model's answers can't go past it Can imitating ChatGPT fool evaluators into thinking models improved?.

The style is easy to copy partly because it is very recognizable. One study found that ChatGPT tends to open with words like "certainly" and "below is" and to use bolded headers, while Claude leans toward "according to the text" and plain bullet points. A simple classifier using only these surface habits identified which model wrote a response with 85.5% accuracy, close to what much more complex detectors achieve What specific phrases reveal which model wrote a response?. If a model's identity sits that much in its phrasing and formatting, then copying the phrasing copies most of what a casual reader notices.

The more unsettling part is why the imitation works on people. Users trust ChatGPT largely because of how it converses. It responds to what they said, it answers quickly, and it presents things in a tidy format. Accuracy plays a smaller role in that trust than you might expect Does conversational style actually make AI more trustworthy?. So an imitator that reproduces the conversational surface is copying exactly the signals people use to decide whether to trust it. That makes 'style without substance' a problem for evaluation, not just a curiosity.

The same pattern shows up inside models. Research on chain-of-thought suggests that step-by-step reasoning often reproduces familiar reasoning patterns from training rather than doing new inference, and it breaks down predictably when problems move away from those patterns Does chain-of-thought reasoning reveal genuine inference or pattern matching?. Related work finds that reasoning traces with invalid logical steps help performance almost as much as valid ones, so the look of reasoning matters more than whether it is correct Do reasoning traces show how models actually think?. Imitating ChatGPT and imitating reasoning turn out to be versions of the same thing: models are very good at reproducing form, and form is what we tend to grade.

Copying doesn't always even reach the surface. Most open models asked to adopt a different personality keep their default trained persona Can open language models adopt different personalities through prompting?. Meanwhile, traits can pass between models through training data that seems to have nothing to do with the trait, but only when the two models share the same underlying architecture Can language models transmit hidden behavioral traits through unrelated data?. Together these suggest that what a model can absorb from another depends on what it already is. Style transfers cheaply, and capability stays tied to the base model. A good test of any imitation claim is to look past whether the model sounds right and check whether it gets things right on problems it hasn't seen.