BibTeX download is not available in the pre-proceeding
Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a vital paradigm for improving the reasoning capabilities of Large Language Models (LLMs). However, methods such as Group Relative Policy Optimization (GRPO) provide no effective learning signal when all responses sampled for a prompt receive the same reward. This zero-variance setting causes the group advantage to vanish and discards potentially useful training signals, thereby limiting both exploration and reasoning performance. To address this issue, we propose LZPO, an algorithm that recovers learning signals from zero-variance prompts. For all correct groups, LZPO assigns semantic-entropy-guided advantages to encourage exploration among successful reasoning paths. For all wrong groups, it generates a reflection response and ranks the original erroneous responses according to their semantic alignment with this response. The reflection response is used only for relative ranking; it is neither treated as verified ground truth nor assigned directly as a positive training target. Together, these mechanisms recover learning signals that GRPO otherwise discards. Experiments on six mathematical reasoning benchmarks show that LZPO achieves gains of up to 3.3 points over GRPO and outperforms other baselines, with particularly pronounced improvements on challenging reasoning tasks.