---
格式版本: 2
标题: "ReOrder-OPD:Reliability-Aware Prompt Ordering for On-Policy Distillation"
原文链接: "https://arxiv.org/abs/2608.10905"
发布日期: "2026-08-11"
发布时间校准状态: "found"
发布时间需复核: "否"
发布时间来源: "rule:local:strict_original_body"
发布时间证据: "**\\[v1\\]** Tue, 11 Aug 2026 13:27:56 UTC (706 KB)"
发布时间校准原因: "规则确认唯一严格发布时间，来源 local:strict_original_body"
发布时间校准置信度: "high"
发布时间候选数量: 18
发布时间严格候选数量: 6
发布时间原页读取状态: "source template page reused from URL open"
发布时间未找到原因: ""
发布时间校准时间: "2026-08-12T10:58:30+08:00"
发布时间仲裁状态: "skipped"
发布时间仲裁尝试次数: 0
发布时间仲裁耗时毫秒: 0
发现时间: "2026-08-12T10:57:49+08:00"
入库时间: "2026-08-12T02:58:30.120Z"
来源平台: "arXiv 学术论文搜索"
搜索渠道: "source_template"
搜索词: "https://arxiv.org/search/?query=Oracle&searchtype=all"
匹配关键词:
  []
相关厂家:
  - "Oracle"
相关专家:
  []
内容类型: "网页"
抓取工具: "Free Fetch + Defuddle"
清洗工具: "Defuddle Markdown + Defuddle/Readability 正文提取"
原始附件:
  []
AI优质: "否"
AI打分: 0
AI分档: "非优质"
AI质检状态: "不通过"
AI打分理由: "论文内容为机器学习领域的prompt ordering和知识蒸馏，与超节点、AI Rack、机柜级AI基础设施等主题完全无关，不提供任何有效信息。"
AI质检模型: "ali-deepseek-v4-flash"
AI质检时间: "2026-08-12T10:58:37+08:00"
AI主题相关性: 0
AI来源权威性: 0
AI新颖性: 0
AI技术细节: 0
AI商业部署信号: 0
AI完整性: 0
采集批次: "2026年8月12日2点04分20秒"
采集批次ID: "20260812-020420-289"
去重键: "https://arxiv.org/abs/2608.10905"
---

## Computer Science > Machine Learning

## Title:ReOrder-OPD:Reliability-Aware Prompt Ordering for On-Policy Distillation

Authors:[Ximo Zhu](https://arxiv.org/search/cs?searchtype=author&query=Zhu,+X), [Ruiqi Liu](https://arxiv.org/search/cs?searchtype=author&query=Liu,+R), [Rong Wang](https://arxiv.org/search/cs?searchtype=author&query=Wang,+R), [Ping Wu](https://arxiv.org/search/cs?searchtype=author&query=Wu,+P), [Xiang Zheng](https://arxiv.org/search/cs?searchtype=author&query=Zheng,+X), [Wenzhuo Xu](https://arxiv.org/search/cs?searchtype=author&query=Xu,+W), [Xubin Yao](https://arxiv.org/search/cs?searchtype=author&query=Yao,+X), [Zhiyuan Yan](https://arxiv.org/search/cs?searchtype=author&query=Yan,+Z), [Bo Li](https://arxiv.org/search/cs?searchtype=author&query=Li,+B), [Jun Gao](https://arxiv.org/search/cs?searchtype=author&query=Gao,+J), [Xiaolei Lv](https://arxiv.org/search/cs?searchtype=author&query=Lv,+X)

[View PDF](https://arxiv.org/pdf/2608.10905) [HTML (experimental)](https://arxiv.org/html/2608.10905v1)

> Abstract:On-policy distillation (OPD) applies token-level teacher supervision to student-generated trajectories, but this supervision is not always reliable. Existing methods use local confidence or teacher-student agreement to weight, filter, or truncate the sampled trajectory. These signals do not directly determine whether the teacher can continue a student prefix to a correct answer, and trajectory-level interventions can conflate one rollout's unreliability with low expected training value of its prompt. We define prompt-level teacher continuation reliability $R$ as the teacher's probability of reaching a correct answer from a student prefix, averaged over prefixes and trajectories induced by the current student. Oracle experiments show that high-$R$ prompts yield larger OPD gains and that descending-$R$ training outperforms random and ascending orders on a fixed prompt pool. Because estimating $R$ requires many teacher continuations, we use the maximum ROUGE-5 F1 between one independent student rollout and verifier-correct same-prompt teacher trajectories. Across ten equal-frequency bins of this actual score, mean $R$ rises monotonically, showing that the proxy separates coarse reliability levels. ReOrder-OPD sorts prompts by the proxy, then draws independent on-policy training trajectories for vanilla OPD. It improves every matched aggregate comparison across Qwen3 and Gemma4 mathematics settings and Qwen3 code settings. Gains in all six FiRe-OPD and ExOPD settings show that prompt ordering complements within-trajectory supervision.

| Subjects: | Machine Learning (cs.LG) |
| --- | --- |
| Cite as: | [arXiv:2608.10905](https://arxiv.org/abs/2608.10905) \[cs.LG\] |
|  | (or [arXiv:2608.10905v1](https://arxiv.org/abs/2608.10905v1) \[cs.LG\] for this version) |
|  | [https://doi.org/10.48550/arXiv.2608.10905](https://doi.org/10.48550/arXiv.2608.10905) |

## Submission history

From: Ximo Zhu \[[view email](https://arxiv.org/show-email/3fa69a2d/2608.10905)\]  
**\[v1\]** Tue, 11 Aug 2026 13:27:56 UTC (706 KB)

[Which authors of this paper are endorsers?](https://arxiv.org/auth/show-endorsers/2608.10905) | Disable MathJax ([What is MathJax?](https://info.arxiv.org/help/mathjax.html))