---
格式版本: 2
标题: "JudgeStealer: Extracting LLM Judging Capabilities across Evaluation Protocols"
原文链接: "https://arxiv.org/abs/2608.26982"
发布日期: "2026-08-27"
发布时间校准状态: "found"
发布时间需复核: "否"
发布时间来源: "rule:local:strict_original_body"
发布时间证据: "**\\[v1\\]** Thu, 27 Aug 2026 11:28:32 UTC (494 KB)"
发布时间校准原因: "规则确认唯一严格发布时间，来源 local:strict_original_body"
发布时间校准置信度: "high"
发布时间候选数量: 18
发布时间严格候选数量: 6
发布时间原页读取状态: "source template page reused from URL open"
发布时间未找到原因: ""
发布时间校准时间: "2026-08-30T15:13:13+08:00"
发布时间仲裁状态: "skipped"
发布时间仲裁尝试次数: 0
发布时间仲裁耗时毫秒: 0
发现时间: "2026-08-30T15:11:38+08:00"
入库时间: "2026-08-30T07:13:14.058Z"
来源平台: "arXiv 学术论文搜索"
搜索渠道: "source_template"
搜索词: "https://arxiv.org/search/?query=Scale-up&searchtype=all"
匹配关键词:
  - "Scale-up"
相关厂家:
  []
相关专家:
  []
内容类型: "网页"
抓取工具: "Free Fetch + Defuddle"
清洗工具: "Defuddle Markdown + Defuddle/Readability 正文提取"
原始附件:
  []
AI优质: "否"
AI打分: 20
AI分档: "非优质"
AI质检状态: "不通过"
AI打分理由: "正文主线是窃取LLM评判能力的模型提取框架，新增了跨点式、成对和列表式评估协议的方法及最高73.3%、87.0%、71.6%的实验结果，但不涉及超节点、AI机架、互连、供电、散热或机架级部署。来源为作者提交的arXiv一手预印本，当前页面仅提供摘要，尚非正式标准或生产部署材料。固定知识库未发现同一研究，但未命中不能证明首次出现；其新增事实也不属于业务情报范围。商业客户、量产和部署信号均缺失，命中正文主线非机架级AI基础设施的强否决条件。"
AI质检模型: "gpt-5.6-sol"
AI质检时间: "2026-08-30T15:13:29+08:00"
AI主题相关性: 0
AI来源权威性: 10
AI新颖性: 4
AI技术细节: 0
AI商业部署信号: 0
AI完整性: 6
AI评分提示词版本: "v17-精简生产版"
AI评分提示词SHA256: "48fb9777f386026761b4873eaff30807694fb11e9b352d7c69bf2dfde750cc7d"
AI评分知识库版本: "knowledge_base_v1-20260819+runtime.55"
AI评分知识库SHA256: "ca56afec1ff616b1fa4394cd1d08b1e2a6af47eccbe70c802c9f1fea6037e803"
AI评分知识库检索词: "[\"Scale-up\",\"NPU\",\"Intel\",\"LLM\",\"JUDGESTEALER\",\"PDF\",\"arxiv.org/pdf/2608.26982\",\"HTML\",\"arxiv.org/html/2608.26982v1\",\"LLM-as-a-judge\",\"arxiv.org/abs/2608.26982\",\"arxiv.org/abs/2608.26982v1\"]"
AI评分知识库命中: "[{\"id\":\"july-correct-0111\",\"title\":\"StrataCL: Fabric-Native Communication Library for Production Supernodes\",\"sourceType\":\"labeled_article\",\"time\":\"2026-07\",\"matchedTerms\":[\"NPU\",\"LLM\",\"PDF\",\"HTML\"],\"rank\":-14.493749653978089},{\"id\":\"july-correct-0131\",\"title\":\"DeepSeek-V4如何在昇腾超节点高效完成全参数后训练？SLAI T-Rex技术报告解读\",\"sourceType\":\"labeled_article\",\"time\":\"2026-07\",\"matchedTerms\":[\"NPU\",\"LLM\",\"PDF\",\"HTML\"],\"rank\":-11.988553635560807},{\"id\":\"july-correct-0081\",\"title\":\"AAI 2026: 6th Gen AMD EPYC Server CPUs Power the Agentic Data Center\",\"sourceType\":\"labeled_article\",\"time\":\"2026-07\",\"matchedTerms\":[\"Intel\",\"PDF\",\"HTML\"],\"rank\":-10.511406570061126},{\"id\":\"runtime-569639751a0dbe7ef3ffccc5\",\"title\":\"[2608.17503] Predict Before Replay: Joint FEC and Flight Control for Reliable Scale-Up Links\",\"sourceType\":\"ai_excellent_article\",\"time\":\"2026-08-18\",\"matchedTerms\":[\"Scale-up\",\"PDF\",\"HTML\"],\"rank\":-10.481970259177816},{\"id\":\"july-correct-0088\",\"title\":\"StrataCL: Fabric-Native Communication Library for Production Supernodes - 智源社区论文\",\"sourceType\":\"labeled_article\",\"time\":\"2026-07\",\"matchedTerms\":[\"NPU\",\"LLM\",\"HTML\"],\"rank\":-10.465171912639054}]"
AI摘要: "研究者提出 JUDGESTEALER，这是首个跨评估协议（pointwise、pairwise、listwise）高效提取 LLM 评判能力的模型窃取框架，利用跨协议一致性并动态选择输入来减少受害者查询。"
AI摘要模型: "ali-deepseek-v4-flash"
AI摘要时间: "2026-08-30T18:08:54.527Z"
采集批次: "2026年8月30日14点11分37秒"
采集批次ID: "20260830-141137-130"
去重键: "https://arxiv.org/abs/2608.26982"
---

## Computer Science > Computation and Language

## Title:JudgeStealer: Extracting LLM Judging Capabilities across Evaluation Protocols

Authors:[Chen Chen](https://arxiv.org/search/cs?searchtype=author&query=Chen,+C), [Yaolin Chen](https://arxiv.org/search/cs?searchtype=author&query=Chen,+Y), [Xuehan Sun](https://arxiv.org/search/cs?searchtype=author&query=Sun,+X), [Juan Lin](https://arxiv.org/search/cs?searchtype=author&query=Lin,+J), [Xueluan Gong](https://arxiv.org/search/cs?searchtype=author&query=Gong,+X), [Yuhang Zheng](https://arxiv.org/search/cs?searchtype=author&query=Zheng,+Y), [Qian Wang](https://arxiv.org/search/cs?searchtype=author&query=Wang,+Q), [Kwok-Yan Lam](https://arxiv.org/search/cs?searchtype=author&query=Lam,+K)

[View PDF](https://arxiv.org/pdf/2608.26982) [HTML (experimental)](https://arxiv.org/html/2608.26982v1)

> Abstract:Large language model (LLM) judges are increasingly used across various evaluation scenarios, making their judgment capabilities valuable intellectual property. However, black-box access exposes these capabilities to model extraction attacks. Existing extraction methods do not specifically target LLM judges and provide limited support for multiple evaluation protocols under restricted query budgets. In this study, we propose JUDGESTEALER, the first query-efficient model extraction framework for replicating judging capabilities across pointwise scoring, pairwise comparison, and listwise ranking protocols. JUDGESTEALER exploits the strong cross-protocol agreement to acquire pointwise scores and transform them into pairwise and listwise supervisions without additional victim queries. To capture informative judge patterns and improve query efficiency, JUDGESTEALER dynamically selects pointwise inputs based on semantic diversity, predictive uncertainty, and potential judge biases. It further applies score smoothing and multi-protocol review to preserve the ordinal structure of scores and mitigate catastrophic forgetting during surrogate adaptation. Extensive experiments on state-of-the-art LLM-as-a-judge and reward models show that JUDGESTEALER consistently outperforms existing extraction baselines, achieving up to 73.3%, 87.0%, and 71.6% accuracy for pointwise, pairwise, and listwise evaluation, respectively. JUDGESTEALER also remains effective across different sur- rogate model scales, adaptation strategies, and reasoning settings. Moreover, JUDGESTEALER demonstrates robustness against representative extraction defenses.

| Comments: |  |
| --- | --- |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | [arXiv:2608.26982](https://arxiv.org/abs/2608.26982) \[cs.CL\] |
|  | (or [arXiv:2608.26982v1](https://arxiv.org/abs/2608.26982v1) \[cs.CL\] for this version) |
|  | [https://doi.org/10.48550/arXiv.2608.26982](https://doi.org/10.48550/arXiv.2608.26982) |

## Submission history

From: Chen Chen \[[view email](https://arxiv.org/show-email/b8f03aab/2608.26982)\]  
**\[v1\]** Thu, 27 Aug 2026 11:28:32 UTC (494 KB)

[Which authors of this paper are endorsers?](https://arxiv.org/auth/show-endorsers/2608.26982) | Disable MathJax ([What is MathJax?](https://info.arxiv.org/help/mathjax.html))