---
格式版本: 2
标题: "Multi-Turn LLM Conversations under the Least-Recently-Used Policy: Mean-Field Asymptotics and Hit Ratio Approximation"
原文链接: "https://arxiv.org/abs/2609.02027"
发布日期: "2026-09-02"
发布时间校准状态: "found"
发布时间需复核: "否"
发布时间来源: "rule:local:strict_original_body"
发布时间证据: "**\\[v1\\]** Wed, 2 Sep 2026 02:57:16 UTC (3,104 KB)"
发布时间校准原因: "规则确认唯一严格发布时间，来源 local:strict_original_body"
发布时间校准置信度: "high"
发布时间候选数量: 18
发布时间严格候选数量: 6
发布时间原页读取状态: "source template page reused from URL open"
发布时间未找到原因: ""
发布时间校准时间: "2026-09-04T01:40:00+08:00"
发布时间仲裁状态: "skipped"
发布时间仲裁尝试次数: 0
发布时间仲裁耗时毫秒: 0
发现时间: "2026-09-04T01:39:08+08:00"
入库时间: "2026-09-03T17:40:00.504Z"
来源平台: "arXiv 学术论文搜索"
搜索渠道: "source_template"
搜索词: "https://arxiv.org/search/?query=HBM&searchtype=all"
匹配关键词:
  - "HBM"
  - "performance"
  - "bandwidth"
相关厂家:
  []
相关专家:
  []
内容类型: "网页"
抓取工具: "Free Fetch + Defuddle"
清洗工具: "Defuddle Markdown + Defuddle/Readability 正文提取"
原始附件:
  []
AI优质: "否"
AI打分: 12
AI分档: "非优质"
AI质检状态: "不通过"
AI打分理由: "论文主题为LLM多轮对话缓存命中率与LRU策略，仅背景提及HBM，与超节点/AI Rack/机柜级AI基础设施无关。"
AI质检模型: "zj-deepseek-v4-flash"
AI质检时间: "2026-09-04T01:40:26+08:00"
AI主题相关性: 0
AI来源权威性: 5
AI新颖性: 0
AI技术细节: 5
AI商业部署信号: 0
AI完整性: 2
AI摘要: "该研究针对多轮大语言模型对话系统的KV缓存命中率问题，在最近最少使用（LRU）替换策略下建立平均场渐近模型，证明当请求到达率与内存容量成比例增长时命中率收敛于闭式极限，并提出实用命中率估计器。"
AI摘要模型: "ali-deepseek-v4-flash"
AI摘要时间: "2026-09-04T00:17:29.645Z"
采集批次: "2026年9月3日22点43分34秒"
采集批次ID: "20260903-224334-406"
去重键: "https://arxiv.org/abs/2609.02027"
---

## Computer Science > Performance

## Title:Multi-Turn LLM Conversations under the Least-Recently-Used Policy: Mean-Field Asymptotics and Hit Ratio Approximation

Authors:[Heyuan Yao](https://arxiv.org/search/cs?searchtype=author&query=Yao,+H), [Chutong Gao](https://arxiv.org/search/cs?searchtype=author&query=Gao,+C), [Yuan Lyu](https://arxiv.org/search/cs?searchtype=author&query=Lyu,+Y), [Izzy Grosof](https://arxiv.org/search/cs?searchtype=author&query=Grosof,+I), [David Simchi-Levi](https://arxiv.org/search/cs?searchtype=author&query=Simchi-Levi,+D)

[View PDF](https://arxiv.org/pdf/2609.02027) [HTML (experimental)](https://arxiv.org/html/2609.02027v1)

> Abstract:The major workloads in modern large language model (LLM) serving systems have shifted from single-shot LLM calls to multi-turn conversations, where new responses are generated based on the whole conversation history across all previous turns. The hit ratio, i.e., the average fraction of KV caches accessed directly from existing caches stored in high-bandwidth memory (HBM), is hence a crucial metric that governs system performance. Estimating the hit ratio is a highly nontrivial task due to the complex system dynamics, where the KV cache prefixes grow with turns and some must be evicted due to finite memory capacity. We formulate the system as a multi-turn conversation model under the least-recently-used (LRU) policy. Through a mean-field asymptotic framework, we prove that as the conversation arrival rate and the memory capacity grow proportionally to infinity, the hit ratio converges to a closed-form limit. Based on the characterization of the limit, we further propose a practical hit ratio estimator, and validate its accuracy by real LLM serving experiments on the Qwen3-8B model implemented on Ascend NPUs. Our results provide a theoretical foundation for the analysis of multi-turn LLM serving systems and a practical guideline for memory capacity provisioning.

| Subjects: | Performance (cs.PF); Probability (math.PR) |
| --- | --- |
| MSC classes: | 60K25, 68M20, 90B22 |
| Cite as: | [arXiv:2609.02027](https://arxiv.org/abs/2609.02027) \[cs.PF\] |
|  | (or [arXiv:2609.02027v1](https://arxiv.org/abs/2609.02027v1) \[cs.PF\] for this version) |
|  | [https://doi.org/10.48550/arXiv.2609.02027](https://doi.org/10.48550/arXiv.2609.02027) |

## Submission history

From: Heyuan Yao \[[view email](https://arxiv.org/show-email/fb0b692c/2609.02027)\]  
**\[v1\]** Wed, 2 Sep 2026 02:57:16 UTC (3,104 KB)

[Which authors of this paper are endorsers?](https://arxiv.org/auth/show-endorsers/2609.02027) | Disable MathJax ([What is MathJax?](https://info.arxiv.org/help/mathjax.html))