SLPO

Outcome-reward RL for continuous latent reasoners — a tractable surrogate policy, a learned stopping gate, and adaptive latent test-time scaling.

Runyang You^*1, Zhiyuan Liu^*2, Yongqi Li^†1, Wenjie Li¹

¹Hong Kong Polytechnic University · ²Sichuan University
^*Equal contribution · ^†Corresponding author

Code Models Paper soon

In one line

Latent reasoners think in continuous vectors, so standard token-level RL does not apply. SLPO defines a surrogate likelihood over latent transitions, cold-starts a stop head from answer correctness, and optimizes with RLOO/GRPO — unlocking Pass@k style scaling without decoding every thought as text.

What SLPO adds

Stop-gate cold start

Supervise continue-vs-stop from verifiable answers over candidate latent lengths, before any RL.

Surrogate latent policy

Score continuous transitions with a diagonal-Gaussian density from MC-dropout forwards, jointly with answer and stopping likelihoods.

Latent test-time scaling

Adaptive stopping reallocates latent compute at inference, improving Acc and Pass@k on COCONUT and CODI.

Method

Overview of the SLPO training and inference pipeline — Advantages weight a joint surrogate over latent transitions, answer tokens, and the first-stop event.

Resources

Code

ModalityDance/SLPO Two-stage training (stop-gate → SLPO) and evaluation scripts.

Open →

Model

slpo-coconut-gpt2 COCONUT + SLPO · recommended gate threshold 0.6

Open →

Model

slpo-codi-gpt2 CODI + SLPO · recommended gate threshold 0.7

Open →

Hub

SLPO collection All public checkpoints in one place.

Open →

Citation

If you find SLPO useful, please cite:

@misc{you2026slpo,
  title   = {SLPO: Scaling Latent Reasoning with Surrogate Policy Optimization},
  author  = {You, Runyang and Liu, Zhiyuan and Li, Yongqi and Li, Wenjie},
  year    = {2026},
  note    = {Code: https://github.com/ModalityDance/SLPO}
}