---
格式版本: 2
标题: "The Lifecycle of LLM-as-a-Judge for Large-Scale Recommendation Explanations"
原文链接: "https://arxiv.org/abs/2608.18300"
发布日期: "2026-08-18"
发布时间校准状态: "found"
发布时间需复核: "否"
发布时间来源: "rule:local:strict_original_body"
发布时间证据: "**\\[v1\\]** Tue, 18 Aug 2026 20:27:09 UTC (495 KB)"
发布时间校准原因: "规则确认唯一严格发布时间，来源 local:strict_original_body"
发布时间校准置信度: "high"
发布时间候选数量: 18
发布时间严格候选数量: 6
发布时间原页读取状态: "source template page reused from URL open"
发布时间未找到原因: ""
发布时间校准时间: "2026-08-20T14:38:19+08:00"
发布时间仲裁状态: "skipped"
发布时间仲裁尝试次数: 0
发布时间仲裁耗时毫秒: 0
发现时间: "2026-08-20T14:35:51+08:00"
入库时间: "2026-08-20T06:38:19.344Z"
来源平台: "arXiv 学术论文搜索"
搜索渠道: "source_template"
搜索词: "https://arxiv.org/search/?query=Meta&searchtype=all"
匹配关键词:
  - "deployment"
  - "AI"
相关厂家:
  - "Meta"
相关专家:
  []
内容类型: "网页"
抓取工具: "Free Fetch + Defuddle"
清洗工具: "Defuddle Markdown + Defuddle/Readability 正文提取"
原始附件:
  []
AI优质: "否"
AI打分: 10
AI分档: "非优质"
AI质检状态: "不通过"
AI打分理由: "论文主题为LLM-as-a-Judge在推荐解释中的应用，与超节点/AI Rack/机柜级AI基础设施完全无关，无相关技术或商业信号。"
AI质检模型: "ali-deepseek-v4-flash"
AI质检时间: "2026-08-20T14:38:56+08:00"
AI主题相关性: 0
AI来源权威性: 5
AI新颖性: 0
AI技术细节: 0
AI商业部署信号: 0
AI完整性: 5
AI摘要: "Netflix 提出 LLM-as-a-Judge 的生命周期框架，用于评估面向数千万会员的推荐解释，涵盖构建、训练、部署与持续监控四阶段。五周 A/B 测试显示，法官对齐后的解释使会员更倾向观看未看过的内容，并提升浏览到播放的成功率。"
AI摘要模型: "ali-deepseek-v4-flash"
AI摘要时间: "2026-09-07T03:22:06.112Z"
采集批次: "2026年8月20日14点19分32秒"
采集批次ID: "20260820-141932-079"
去重键: "https://arxiv.org/abs/2608.18300"
---

## Computer Science > Artificial Intelligence

## Title:The Lifecycle of LLM-as-a-Judge for Large-Scale Recommendation Explanations

[View PDF](https://arxiv.org/pdf/2608.18300) [HTML (experimental)](https://arxiv.org/html/2608.18300v1)

> Abstract:LLM-as-a-Judge, which leverages a large language model to evaluate natural language generated by another AI application or model, has become a standard, scalable approach for accelerating and extending costly human evaluation. However, most work treats a judge as a static artifact, evaluating it once at construction or against a fixed benchmark. In contrast, we argue that an LLM judge running in a production system is better understood as having a lifecycle: it must be built, trained, deployed, and continuously maintained as the surrounding data evolves, and each phase poses distinct technical and operational challenges.  
> We present such a lifecycle for the LLM judges that evaluate user-facing recommendation explanations at Netflix, where our pipeline generates and the judges assess hundreds of thousands of distinct show-level explanations per week, served across the mobile experience to millions of members. Our framework has four phases: (I) Birth, defining multiple evaluation criteria and building curated benchmark datasets with human labels and rationales; (II) Training, refining the judges' rubrics via Reasoning-Aligned Rubric Tuning (RART), a rubric-tuning procedure that uses a meta-judge over reasoning output as the learning signal; (III) Deployment, in which one judge serves two production roles: quality gating and reflective generation; and (IV) Monitoring, a continuous Human-in-the-Loop alignment process that detects drift and triggers re-tuning behind a human review gate. We report post-launch results from a five-week A/B test over tens of millions of members, in which the judge-aligned explanations shifted member viewing toward novel content (previously unwatched) and increased successful browse-to-play sessions relative to a no-explanation control, with no quality-related takedowns.

| Subjects: | Artificial Intelligence (cs.AI) |
| --- | --- |
| Cite as: | [arXiv:2608.18300](https://arxiv.org/abs/2608.18300) \[cs.AI\] |
|  | (or [arXiv:2608.18300v1](https://arxiv.org/abs/2608.18300v1) \[cs.AI\] for this version) |
|  | [https://doi.org/10.48550/arXiv.2608.18300](https://doi.org/10.48550/arXiv.2608.18300) |

## Submission history

From: Emma Yanyang Kong \[[view email](https://arxiv.org/show-email/34ef5311/2608.18300)\]  
**\[v1\]** Tue, 18 Aug 2026 20:27:09 UTC (495 KB)

[Which authors of this paper are endorsers?](https://arxiv.org/auth/show-endorsers/2608.18300) | Disable MathJax ([What is MathJax?](https://info.arxiv.org/help/mathjax.html))