---
格式版本: 2
标题: "Self-Normalized Inference for Constant-Stepsize Temporal-Difference Learning under Markovian Sampling"
原文链接: "https://arxiv.org/abs/2608.10896"
发布日期: "2026-08-11"
发布时间校准状态: "found"
发布时间需复核: "否"
发布时间来源: "rule:local:strict_original_body"
发布时间证据: "**\\[v1\\]** Tue, 11 Aug 2026 13:19:28 UTC (148 KB)"
发布时间校准原因: "规则确认唯一严格发布时间，来源 local:strict_original_body"
发布时间校准置信度: "high"
发布时间候选数量: 18
发布时间严格候选数量: 6
发布时间原页读取状态: "source template page reused from URL open"
发布时间未找到原因: ""
发布时间校准时间: "2026-08-12T11:24:27+08:00"
发布时间仲裁状态: "skipped"
发布时间仲裁尝试次数: 0
发布时间仲裁耗时毫秒: 0
发现时间: "2026-08-12T11:23:04+08:00"
入库时间: "2026-08-12T03:24:27.817Z"
来源平台: "arXiv 学术论文搜索"
搜索渠道: "source_template"
搜索词: "https://arxiv.org/search/?query=bandwidth&searchtype=all"
匹配关键词:
  - "bandwidth"
相关厂家:
  []
相关专家:
  []
内容类型: "网页"
抓取工具: "Free Fetch + Defuddle"
清洗工具: "Defuddle Markdown + Defuddle/Readability 正文提取"
原始附件:
  []
AI优质: "否"
AI打分: 5
AI分档: "非优质"
AI质检状态: "不通过"
AI打分理由: "该资料为机器学习论文，主题完全无关超节点/AI Rack等机柜级AI基础设施，仅命中搜索词bandwidth，无任何技术或商业信号。"
AI质检模型: "ali-deepseek-v4-flash"
AI质检时间: "2026-08-12T11:24:32+08:00"
AI主题相关性: 0
AI来源权威性: 5
AI新颖性: 0
AI技术细节: 0
AI商业部署信号: 0
AI完整性: 0
采集批次: "2026年8月12日2点04分20秒"
采集批次ID: "20260812-020420-289"
去重键: "https://arxiv.org/abs/2608.10896"
---

## Statistics > Machine Learning

## Title:Self-Normalized Inference for Constant-Stepsize Temporal-Difference Learning under Markovian Sampling

Authors:[Min Zeng](https://arxiv.org/search/stat?searchtype=author&query=Zeng,+M), [Yichen Zhang](https://arxiv.org/search/stat?searchtype=author&query=Zhang,+Y), [Xiaofeng Shao](https://arxiv.org/search/stat?searchtype=author&query=Shao,+X)

[View PDF](https://arxiv.org/pdf/2608.10896) [HTML (experimental)](https://arxiv.org/html/2608.10896v1)

> Abstract:Constant-stepsize temporal-difference (TD) learning is attractive for policy evaluation, but inference from a single Markov trajectory must account for serial dependence and a stepsize-dependent stationary target. For fixed-stepsize linear TD, we establish a functional central limit theorem whose covariance retains the multiplicative component induced by the random TD matrix and the stationary iterate error. We then derive a joint functional limit for parallel Richardson--Romberg (RR) recursions driven by the same trajectory. A Brownian-bridge self-normalizer yields asymptotically pivotal confidence regions for prespecified state-value contrasts without estimating the long-run covariance or selecting a bandwidth or batch length. For such a contrast, the procedure admits a one-pass implementation whose memory does not grow with the trajectory length. At a fixed stepsize, the inferential center is the RR stationary target. We also study horizon-indexed designs in which the stepsize remains constant within each run and decreases across longer horizons. Under an explicit RR-dependent rate window, the residual RR target shift, multiplicative remainder, and initialization effect are negligible at the root-$n$ scale, yielding inference for the projected Bellman solution. Experiments on FrozenLake and Garnet illustrate stationary-target coverage, RR target correction, and the finite-sample behavior of the horizon-indexed design.

| Subjects: | Machine Learning (stat.ML); Machine Learning (cs.LG) |
| --- | --- |
| Cite as: | [arXiv:2608.10896](https://arxiv.org/abs/2608.10896) \[stat.ML\] |
|  | (or [arXiv:2608.10896v1](https://arxiv.org/abs/2608.10896v1) \[stat.ML\] for this version) |
|  | [https://doi.org/10.48550/arXiv.2608.10896](https://doi.org/10.48550/arXiv.2608.10896) |

## Submission history

From: Min Zeng \[[view email](https://arxiv.org/show-email/e763fae1/2608.10896)\]  
**\[v1\]** Tue, 11 Aug 2026 13:19:28 UTC (148 KB)

[Which authors of this paper are endorsers?](https://arxiv.org/auth/show-endorsers/2608.10896) | Disable MathJax ([What is MathJax?](https://info.arxiv.org/help/mathjax.html))