Line of inquiry
Inquiring lines›How do language models construct a…›How does AI persuasion undermine h…›this line of inquiry
What role does compression play in language model capability and generalization?
A broader line of inquiry — a family of 32 specific questions the research asks around this. Follow one into its inquiring-line page, or move sideways to a related line below.
Questions in this line of inquiry 32
Specific inquiring lines the field asks around this — ordered from the most general framing down to the most specific angle.
- Can context compression preserve what matters without introducing bias?
- How does the compression view extend from trained models to training objectives?
- Can compression length really indicate how well a model generalizes?
- Can model compression size predict generalization better than parameter count?
- Why does keeping full key-value blocks matter more than compressing them?
- Why does language compression via statistical dependencies capture cultural and situated language use?
- How does modeling capability relate to lossless compression in language models?
- Why do student models learn better from internal pruning versus external compression?
- Why do parameter-based compressors fail to measure true model simplicity?
- What compression explains why syntax fits in low-dimensional subspaces?
- Why does statistical compression destroy literary connotation and meaning?
- Can fixed-size latent states losslessly store arbitrary input context?
- What is the connection between model compression and data compression?
- Can task-agnostic compression of documents remain broadly useful for later queries?
- Why does adjusted compression performance degrade as models scale larger?
- Can steering vectors be combined with other compression techniques?
- Can linguistic compression be a fundamental mechanism for representing psychology?
- How does compressing memory between iterations prevent overthinking?
- Does recurrent memory or gist compression work better for ultra-long context?
- How do memory hierarchies and compression reduce context management demands?
- Can KV cache pruning serve as an alternative to consolidation?
- How does data entropy inflate compression estimates in prequential coding?
- Why does each rewrite cycle degrade domain-specific details differently than compression?
- Can compressive memory track what matters most across 35 conversation sessions?
- When should architects prioritize consolidation compute over larger context windows?
- How does reducing activation precision further extend context length?
- Can entropy signatures alone detect whether context was model-generated or externally prefilled?
- Can differential privacy during generation eliminate leakage at scale?
- Why does token redundancy and poor readability emerge at trillion-parameter scale?
- How does epiplexity measure extractable value differently from compression codelength?
- Does ternary weight quantization simplify deployment of mixture of experts?
- How much does schema bloat actually degrade reasoning in large language models?