This paper tackles one of the most important open questions in LLM training methodology: when should you use reinforcement learning with verifiable rewards (RLVR) versus knowledge distillation to improve a model's reasoning capabilities? The authors draw a critical distinction between two metrics — pass@1 (accuracy: does the model get it right on the first try?) and pass@k (capability: can the model produce a correct answer among k attempts?). Their core finding is that RLVR reliably boosts pass@1 but consistently fails to improve pass@k, while distillation can improve both — but only under the right conditions.
The training recipe for modern reasoning models like o1, DeepSeek-R1, and their successors heavily relies on RLVR. This paper reveals a fundamental limitation of that approach: RLVR works by optimizing the easier end of the difficulty distribution, essentially helping the model be more consistently correct on problems it already has some ability to solve. But the hardest problems — the ones that require genuine new reasoning capability — actually suffer. This is a sobering finding for anyone betting on RLVR as the primary path to superintelligent reasoning. It suggests we may hit a ceiling with pure RLVR scaling.
The paper's treatment of distillation is equally nuanced. The authors show that distillation only improves capability (pass@k) when it introduces genuinely new knowledge to the student model. When you distill reasoning patterns from a teacher that the student's base distribution already covers, you get accuracy gains (pass@1 improves) but no capability expansion — and you actually hurt performance on the hardest problems, mirroring RLVR's failure mode. This is a crucial insight: the source of improvement matters, not just the technique.
These findings have direct implications for how labs should think about training pipelines. If RLVR and distillation both fail to expand true capability unless novel knowledge or reasoning patterns are introduced, the field needs to invest more in curriculum design, harder problem generation, and teacher models that genuinely exceed student capability on difficult tasks. The paper also highlights a methodological gap — response length and reflection keywords are not reliable indicators of response quality — pushing for better evaluation frameworks. This is a must-read for anyone working on post-training for reasoning models.