This paper introduces a fundamentally different approach to scaling reasoning in language models. Rather than generating longer chains of tokens (the approach used by o1, DeepSeek-R1, QwQ, and similar models), the authors propose iterating a recurrent depth block within the model itself — effectively giving the model more "thinking time" by running its internals multiple times before producing output. A 3.5B parameter model trained on 800B tokens can punch well above its weight class, achieving performance equivalent to a 50B parameter model on reasoning benchmarks by simply unrolling deeper at inference time.
The dominant paradigm for scaling test-time compute right now is chain-of-thought: make models generate more tokens, and those tokens constitute visible reasoning steps. This approach has real costs — longer context windows, more memory, higher latency, and a dependency on high-quality reasoning trace data during training. Latent reasoning sidesteps all of these. The model reasons in embedding space, not token space, which means it doesn't need specialized training data, doesn't blow up context windows, and can potentially capture reasoning patterns that are fundamentally hard to express in natural language. This is a qualitatively different kind of "thinking" from what we see in today's reasoning models.
If latent recurrent depth scales well, it could represent a significant architectural shift in how we build reasoning-capable models. The ability to tune compute at inference time without changing model weights is extremely valuable for deployment — operators could dial up reasoning depth for hard problems and dial it back for easy ones. This also raises interesting questions about what kinds of reasoning are best suited to latent vs. token-level computation. Mathematical reasoning, spatial reasoning, and abstract pattern matching might all benefit differently from this approach compared to CoT. The authors position this as a proof-of-concept at 3.5B parameters; scaling behavior at 70B+ will be the critical test of this idea's viability.
The key unknowns are interpretability and scaling laws. With CoT, we can read the reasoning steps. With latent reasoning, the "thought process" is opaque — a sequence of high-dimensional vectors. This makes it harder to audit, debug, or align. Additionally, we don't yet know whether recurrent depth scales favorably compared to simply training larger transformer models or scaling CoT length further. The paper demonstrates the concept compellingly at 3.5B, but the research community will want to see systematic scaling experiments before treating this as a viable alternative paradigm.