---
格式版本: 2
标题: "Efficient Hypergradient Descent for Inverse Reinforcement Learning"
原文链接: "https://arxiv.org/abs/2608.11052"
发布日期: "2026-08-11"
发布时间校准状态: "found"
发布时间需复核: "否"
发布时间来源: "rule:local:strict_original_body"
发布时间证据: "**\\[v1\\]** Tue, 11 Aug 2026 15:22:16 UTC (900 KB)"
发布时间校准原因: "规则确认唯一严格发布时间，来源 local:strict_original_body"
发布时间校准置信度: "high"
发布时间候选数量: 18
发布时间严格候选数量: 6
发布时间原页读取状态: "source template page reused from URL open"
发布时间未找到原因: ""
发布时间校准时间: "2026-08-12T11:19:35+08:00"
发布时间仲裁状态: "skipped"
发布时间仲裁尝试次数: 0
发布时间仲裁耗时毫秒: 0
发现时间: "2026-08-12T11:16:29+08:00"
入库时间: "2026-08-12T03:19:36.029Z"
来源平台: "arXiv 学术论文搜索"
搜索渠道: "source_template"
搜索词: "https://arxiv.org/search/?query=performance&searchtype=all"
匹配关键词:
  - "performance"
相关厂家:
  []
相关专家:
  []
内容类型: "网页"
抓取工具: "Free Fetch + Defuddle"
清洗工具: "Defuddle Markdown + Defuddle/Readability 正文提取"
原始附件:
  []
AI优质: "否"
AI打分: 5
AI分档: "非优质"
AI质检状态: "不通过"
AI打分理由: "该文章为逆强化学习算法论文，与超节点/AI Rack/机柜级AI基础设施完全无关，无技术细节或商业信号。"
AI质检模型: "ali-deepseek-v4-flash"
AI质检时间: "2026-08-12T11:19:53+08:00"
AI主题相关性: 0
AI来源权威性: 5
AI新颖性: 0
AI技术细节: 0
AI商业部署信号: 0
AI完整性: 0
采集批次: "2026年8月12日2点04分20秒"
采集批次ID: "20260812-020420-289"
去重键: "https://arxiv.org/abs/2608.11052"
---

## Computer Science > Machine Learning

## Title:Efficient Hypergradient Descent for Inverse Reinforcement Learning

[View PDF](https://arxiv.org/pdf/2608.11052) [HTML (experimental)](https://arxiv.org/html/2608.11052v1)

> Abstract:Inverse reinforcement learning (IRL) aims to recover a reward function under which the resulting policy reproduces the behavior observed in expert demonstrations. A natural approach is to formulate IRL as a bilevel optimization problem, in which the inner level corresponds to policy optimization under the learned reward and the outer level measures the discrepancy between the induced policy and expert data. However, this formulation is computationally challenging in practice because the outer update requires a hypergradient involving an inverse-Hessian-vector product for the inner objective. We address this challenge by showing that, at the inner optimum, the Hessian of the inner objective is proportional to the Fisher information matrix of the policy, yielding a structured Fisher-based hypergradient closely related to Natural Hypergradient Descent. To address the resulting scalability bottleneck associated with large Fisher matrices, we approximate the required inverse-Fisher-vector product using a streaming spectral sketch, avoiding explicit construction of the Fisher matrix. We evaluate our approach against a first-order stochastic bilevel baseline across discrete- and continuous-control environments. The results demonstrate competitive policy performance and strong reward-ranking quality, while Fisher sketching reduces curvature-storage complexity and can improve computational efficiency relative to an explicit Fisher solver.

| Subjects: | Machine Learning (cs.LG); Machine Learning (stat.ML) |
| --- | --- |
| Cite as: | [arXiv:2608.11052](https://arxiv.org/abs/2608.11052) \[cs.LG\] |
|  | (or [arXiv:2608.11052v1](https://arxiv.org/abs/2608.11052v1) \[cs.LG\] for this version) |
|  | [https://doi.org/10.48550/arXiv.2608.11052](https://doi.org/10.48550/arXiv.2608.11052) |

## Submission history

From: Ilya Levin \[[view email](https://arxiv.org/show-email/2fdf10d7/2608.11052)\]  
**\[v1\]** Tue, 11 Aug 2026 15:22:16 UTC (900 KB)

[Which authors of this paper are endorsers?](https://arxiv.org/auth/show-endorsers/2608.11052) | Disable MathJax ([What is MathJax?](https://info.arxiv.org/help/mathjax.html))