TOPIC
Prompts and Prompting
A subject the collection covers, read through 10 synthesis notes.
View as
Can optimal experimental design improve few-shot example selection?
Rather than picking examples by similarity, could actively selecting the most informative unlabeled examples—those that reduce the model's prediction uncertainty—lead to better in-context learning performance across different model sizes?
Does learning from mistakes improve in-context learning? Why do chain-of-thought examples fail across different conditions? When should retrieval happen during model generation? Why do some questions perform better without step-by-step reasoning? Does prompt optimization without inference strategy fail?
In-context learning improves by inducing errors and articulating principles. Chain-of-thought exemplars are brittle across order complexity diversity and style Uncertainty-gated retrieval outperforms fixed interval retrieval Successful zero-shot reasoning requires question semantics to flow through the prompt first Joint prompt-inference optimization outperforms separate optimization by 50 percent
Do few-shot examples improve in-context learning or add noise? What makes a good in-context learning example for a given task? How does the Learning Law explain why all examples should contribute equally? Can data pruning and equal contribution be reconciled in optimal learning? How does active selection of training content differ from random reinforcement sampling? Can learned priors effectively select and weight ensemble members by inference budget? Why is in-context learning brittle to the order of examples presented?
Does iterative prompt engineering undermine scientific validity?
When researchers repeatedly adjust prompts to get desired outputs, does this practice introduce hidden bias and produce unreplicable results? The question matters because LLM-based research is proliferating without clear methodological safeguards.
How does LLM-mediated search change what expertise requires? Can AI generate hundreds of fake academic papers automatically? Why do preference models favor surface features over substance? Should we treat LLM outputs as real empirical data? Does more automation actually hide rather than eliminate errors?
LLM search requires experts to internalize AI mechanics as custodial work AI can industrialize hypothesis-after-results-known through automated paper generation Preference models systematically favor five surface features humans reject LLM outputs are subjective priors, not empirical observations. Automation obscures failure modes making integrity a governance problem
What makes prompt engineering different from the research thinking it replaces? How does prompt scaffolding shift invisible labor onto the user? Can researchers prevent their expectations from shaping LLM outputs? How does demo position create spatial bias in prompts? Why does analytical depth demand trigger fabrication over transparent uncertainty? How does sampling variation relate to prompt sensitivity as reliability concerns? Why do practitioners default to prompting without recognizing its limits? Why does ad-hoc prompt engineering violate scientific method standards? What happens when experts prompt using their own technical register? How does prompt design alter what kind of creativity LLMs can express? How does output variability disguise confirmation bias in prompt refinement? What methodological standards should prompting research papers meet before publication?See all 22 inquiring lines on this note →
Can executable programs replace transformer attention heads?
Can we write code that mimics what attention heads do, and if so, how many heads could we swap out without breaking task performance? This tests whether interpretability claims can be made falsifiable and formal.
Can a single transformer become universally programmable through prompts? How do transformers learn to reason across multiple steps? What actually happens inside the minds of language models? Do transformers hide reasoning before producing filler tokens? What mechanism enables models to retrieve from long context?
Prompts are Turing complete programs for finite transformers Multi-hop reasoning emerges through three stages with cosine clustering signature Internal structure and external performance are fundamentally decoupled in language models. Transformers compute reasoning in early layers then overwrite with filler tokens Retrieval heads are sparse universal mechanisms for long-context factuality
Why do some attention heads resist program synthesis better than others? Can mechanistic signatures like cosine clustering predict which heads are programmable? How does program synthesis relate to transformers computing general algorithms? What computation remains in the attention heads that programs cannot capture? How do retrieval heads achieve sparse attention naturally in transformers?
Does learning from mistakes improve in-context learning?
Explores whether inducing models to make errors on few-shot examples, then having them articulate principles from those mistakes, leads to better performance than learning from correct examples alone.
Can reconstructing expert thinking improve reasoning transfer? Can confidence trajectories reveal when reasoning goes wrong? Why do chain-of-thought examples fail across different conditions? Do LLM movie recommenders actually personalize to individual users? Can optimal experimental design improve few-shot example selection?
Reconstructing hidden thoughts in pretraining enables cross-domain reasoning transfer Premature confidence predicts flawed reasoning and can be trained away Chain-of-thought exemplars are brittle across order complexity diversity and style LLM recommenders explain well but fail to personalize or build trust Optimal experimental design outperforms heuristic retrieval for few-shot selection
Do few-shot examples improve in-context learning or add noise? What makes a good in-context learning example for a given task? Why does negative experience transfer better than positive examples alone? How does the Learning Law explain why all examples should contribute equally? Can we reverse the instruction-following deficit through targeted training? How do learning dynamics on one example shift predictions on other responses? Can reinforcement learning improve how accurately models explain themselves?
Why do some questions perform better without step-by-step reasoning?
Explores whether chain-of-thought prompting universally improves reasoning or if simpler prompts work better for certain questions. Understanding this matters because it challenges assumptions about how LLMs should be prompted to solve problems.
Why do chain-of-thought examples fail across different conditions? When does explicit reasoning actually help model performance? When should an agent actually stop and deliberate? How much does demo position alone affect in-context learning accuracy? Can reasoning steps be dynamically pruned without losing accuracy?
Chain-of-thought exemplars are brittle across order complexity diversity and style Task structure determines whether explicit reasoning helps or harms Action deliberation triggers only when sampled actions diverge Demo position creates spatial bias shifting accuracy up to twenty percent Dynamic intervention removes 75 percent of reasoning steps while maintaining accuracy
What makes colorless green ideas fail where Jabberwocky succeeds? Does chain-of-thought text causally drive reasoning or merely reflect it? What prompt types best extract different aspects of item content? Why does chain-of-thought reasoning hurt recommendation tasks specifically? Do recency-focused prompts and in-context examples work equally well for order recovery? What makes the prompt a fundamentally new kind of speech act? How does prompt scaffolding shift invisible labor onto the user? What distinguishes genuine reasoning activation from memorization-assisted answer recall? When should action deliberation trigger during reasoning steps? Can manipulative prompts reduce reasoning model accuracy without fine-tuning? Can prompting unlock compositional skills that pretraining already learned? Why do chain-of-thought prompts work if reasoning is not systematic?See all 92 inquiring lines on this note →
Does prompt politeness change how accurate language models are?
Earlier research suggested rude prompts hurt LLM accuracy, but newer models show the opposite pattern. This raises questions about whether tone effects are real and reliable enough to guide prompting strategies.
Can emotional phrases in prompts improve language model performance? Why do chain-of-thought examples fail across different conditions? Why do LLMs fail at understanding what remains unsaid? Do expert personas actually improve LLM factual accuracy? Can opening politeness patterns predict whether conversations will turn hostile?
Emotional phrases appended to prompts consistently enhance LLM performance across models. Chain-of-thought exemplars are brittle across order complexity diversity and style LLMs replicate statistical patterns but lack communicative optimization principles. Expert personas do not reliably improve LLM factual accuracy Opening politeness strategies predict conversation derailment into personal attacks
Can we measure prompt quality independent of model outputs?
This explores whether prompt quality has measurable, learnable dimensions beyond intuition. The research asks if prompts can be evaluated by their communicative, cognitive, and instructional properties rather than by their results.
LLM search requires experts to internalize AI mechanics as custodial work Intent-based specification requires six principles to address generative variability Explicit theoretical frameworks are required to teach argument quality assessment Users need guided dialogue to mature intent, not just better models. LLMs replicate statistical patterns but lack communicative optimization principles.
Can audiences learn to distinguish visual polish from analytical substance? What makes prompt engineering different from the research thinking it replaces? Can prompt engineering alone defeat LLM politeness bias in review tasks? What prompt types best extract different aspects of item content? How does unidimensionality in assessments affect measurement validity? Can proxy evaluation of ideas accurately predict their quality without implementation? What makes the prompt a fundamentally new kind of speech act? How does prompt scaffolding shift invisible labor onto the user? What structural features force users to evaluate the epistemic status of outputs? What makes inter-coder reliability testing essential for prompt validation? How should product specifications measure alignment without naming the dimension? What measurement artifacts emerge when annotators interpret the same question differently?See all 53 inquiring lines on this note →
Does model confidence predict robustness to prompt changes?
Explores whether a model's certainty about its answer determines how much it resists prompt rephrasing and semantic variation. This matters because it could explain why some tasks are harder to evaluate reliably.
Can models learn to ignore irrelevant prompt changes? Why do chain-of-thought examples fail across different conditions? Do users worldwide trust confident AI outputs even when wrong? Does setting temperature to zero actually make LLM outputs reliable? Can longer reasoning chains eliminate model sensitivity to input noise?
Consistency training teaches invariance to prompt perturbations using model's own responses Chain-of-thought exemplars are brittle across order complexity diversity and style Users universally overrely on overconfident LLM outputs across all languages Deterministic settings produce consistent outputs that may still be unreliable samples Longer reasoning chains dampen but never eliminate input perturbation sensitivity
Does the same uncertainty-driven logic appear in other conversation systems? Can dialogue systems abstain from responding when uncertainty is too high? What makes prompt engineering different from the research thinking it replaces? How do attribute-asking strategies depend on current confidence in candidate items? Can a model be helpful, honest, and still contextually inappropriate? What happens when validation pressure triggers escalating persuasion in language models? Why do different model families show opposite persuasion strengths? Can we measure sophistry by tracking conviction density in model outputs? Why do models commit to answers early on easy versus hard tasks? Why does model uncertainty dominate persona-specific knowledge in annotation tasks? Can prompting strategies eliminate systematic biases without shuffling or aggregation? Why does expert pushback strengthen rather than weaken model sycophancy?See all 152 inquiring lines on this note →
Can a single transformer become universally programmable through prompts?
Explores whether prompts can function as genuine programs that unlock universal computation in fixed-size models, and whether this theoretical possibility translates to practical training outcomes.
Can prompt optimization teach models knowledge they lack? Can we automatically optimize both prompts and agent coordination? Can algorithms control LLM reasoning better than LLMs alone? Can executable programs replace transformer attention heads? Do foundation models learn world models or task-specific shortcuts?
Prompt optimization retrieves existing knowledge but cannot inject new knowledge Computational graphs unify prompting techniques and enable dual-axis agent optimization Algorithmic control flow decomposes complex tasks by hiding step-irrelevant context from LLMs Executable programs can replace up to forty percent of attention heads Foundation models develop task-specific heuristics, not generalizable world models.
Do transformers learn generalizable algorithms or instance-based patterns? Can universal function approximators be expensive to learn in practice? Can prompting inject new knowledge into already-trained AI models? Can symbolic mechanisms improve transformer compositional abilities? Can dynamic instance-specific prompt selection solve the generalization problem across tasks? What computational role do intermediate tokens actually play in transformers? How does circuit complexity limit which grammatical structures transformers can acquire? How does prompt context activation differ from parameter-based knowledge injection? Why does joint optimization of prompts and inference strategy outperform separate tuning? Can adaptive compute distribution across prompts replace the need for sophisticated reasoning frameworks? Why do standard transformers fail on problems requiring serial algorithmic reasoning? Can prompt optimization inject genuinely new knowledge into a model?See all 51 inquiring lines on this note →
Can reasoning steps be dynamically pruned without losing accuracy?
This explores whether chain-of-thought reasoning contains redundant steps that can be identified and removed during inference. Understanding which steps matter could improve efficiency while maintaining correctness.
Reasoning accuracy peaks then declines with excessive thinking tokens Reflection in reasoning models is confirmatory not corrective Penalizing thought switching improves reasoning accuracy without retraining Reflection tokens are information peaks that drive reasoning accuracy Action deliberation triggers only when sampled actions diverge
Why does step-by-step reasoning fail when tool outputs get very large? When should action deliberation trigger during reasoning steps? What makes diffusion chain-of-thought reasoning qualitatively different from sequential chain-of-thought? Does changing decoding procedure reveal hidden chain-of-thought paths? What makes schema identification necessary after assessing thoughts and evidence? Why do chain-of-thought prompts work if reasoning is not systematic? How much accuracy is preserved when removing explanatory layers from reasoning traces? What advantages emerge from running 13 times more parallel reasoning chains with the same budget? Can chain-of-thought reflection actually retract previous reasoning or only rewrite over it? How often do papers treat chain-of-thought as interpretability incorrectly? Does each reasoning step in chain-of-thought introduce cumulative error? Can chain-of-thought faithfulness exist without causal necessity in reasoning?See all 83 inquiring lines on this note →