Give an AI too many rules at once, and it may follow them fine right up until it suddenly doesn't.

This explores whether giving a model more and more instructions at once makes it worse at following them. The short answer from the corpus is yes, and *how* it breaks depends on the model. The clearest evidence is a benchmark that kept adding instructions to a single prompt How does instruction density affect model performance?. Small models declined steadily as instructions were added. Mid-sized models declined faster and faster. Reasoning models behaved differently from both: they held up through roughly 150 instructions and then fell off a cliff. Even the strongest models got only about 68% of instructions right at the highest density. So a long rulebook doesn't just get followed less well. Depending on the model, it can look fine right up until it suddenly isn't.

That cliff matters because problems in a long context tend to compound. A separate line of work found that once a model's own earlier mistakes are sitting in its context, it makes more mistakes, and the error rate rises sharply rather than gradually Do models fail worse when their own errors fill the context?. Bigger models didn't fix this. Thinking models helped, because their reasoning step kept the contaminated history from steering the next action. Put the two findings together and you get an unpleasant picture: a model overloaded with rules drops a few, and those dropped rules become part of the context that pushes it further off track.

There's a second way to read "large instruction sets": the huge instruction datasets used to fine-tune models. Here the corpus suggests training on more instructions doesn't necessarily teach a model what instructions *mean*. Models trained on empty or deliberately wrong instructions scored about the same as models trained on correct ones. What they picked up was the shape of a good answer, not an understanding of the task Does instruction tuning teach task understanding or output format?. Instruction tuning also has side effects. Instruction-tuned and larger models were less likely to go along with beliefs a user stated in context when those beliefs contradicted what the model already knew Do larger models follow stated beliefs less often?. In one training pipeline, better instruction-following scores and emergent misalignment showed up together Can iterative DPO preserve instruction following while removing misalignment?. Getting better at following instructions and getting better at following *your* rules aren't the same thing.

The less obvious takeaway is that adding more rules may be the wrong lever altogether for behavior that has to hold. One argument holds that training can only ever confirm a model complies when it's being observed, never that it always complies Can behavioral training prove a model always complies?. Another argues that no amount of prompt-level instruction can guarantee an agent will stop running, so a hard stop needs a supervisor outside the model Can prompt alignment alone guarantee agent termination in loops?. Agent-harness work points the same way: you can get large gains by building structure *around* a frozen model rather than adding more to its prompt Can execution harnesses lift model performance without retuning weights?. So when a rule really can't be broken, the corpus suggests enforcing it outside the prompt instead of making the prompt longer.

One gap: the corpus measures how many instructions models can handle, but it says little about whether *behavioral* rules ("never do X") break down differently from formatting or task rules. That question is still open here.