Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning

Reinforcement learning with verifiable rewards (RLVR) without human-annotated data, often referred to as “zero RL”, has emerged as a powerful paradigm for eliciting chain-of-thought (CoT) reasoning. However, due to computational constraints, existing studies are largely restricted to small models, leaving the training dynamics and emergent capabilities at a large scale unexplored. To meaningfully explore this frontier, we aim to elicit high-quality reasoning behaviors from the model. However, we find that naive scaling often suffers from poor readability, token redundancy, and a lack of adaptive reasoning depth. To address these challenges, we present a stable and efficient training pipeline, incorporating algorithmic and system optimizations such as clipped importance sampling, traininginference ratio correction, and mixed-precision control.

Introduction. Alongside the scaling of model parameters, chain-of-thought (CoT) reasoning (Wei et al., 2022; Kojima et al., 2022; Lightman et al., 2024) has emerged as a critical scaling dimension for test-time compute, establishing itself as a foundational capability for large language models (LLMs) (Zhao et al., 2026; Brown et al., 2020; Kaplan et al., 2020) to solve complex tasks. Recent work (Shao et al., 2024) has demonstrated that reinforcement learning with verifiable rewards (RLVR) can effectively enhance the reasoning abilities. In particular, the paradigm of “zero RL” (Guo et al., 2025) diverges from traditional imitation learning by initiating RL directly from a pretrained base model, entirely bypassing the need for supervised CoT data, which has proven capable of incentivizing emergent problem-solving strategies. This breakthrough indicates Over the past year, the research community has shown considerable interest in eliciting reasoning behaviors directly from base models.

Discussion / Conclusion. The history of AI research repeatedly demonstrates a “bitter lesson”: general methods that leverage computation and scale ultimately overtake human-engineered heuristics. In the context of LLM reasoning, researchers have invested heavily in crafting specialized prompts, curated SFT datasets, and complex pipelines to force models to present in a structured format, verify their own answers, or explore parallel paths (Lightman et al., 2024; Wang et al., 2025; Zheng et al., 2025b). Indeed, in our own experiments on the 104B model, we found it necessary to introduce a hand-designed reward to successfully guide the model toward structured formatting and self-verification. However, when scaling to the 1-Trillion parameter model, such hand-crafted heuristics become completely redundant. Under pure zero RL, the 1T model spontaneously discovers these advanced cognitive strategies from scratch, without any explicit supervision or auxiliary rewards. In this section, we showcase five striking emergent behaviors that validate this bitter lesson. In this work, we addressed a fundamental question: How did zero reinforcement learning behave and scale at the trillion-parameter level?