---
格式版本: 2
标题: "Cacheable by Design? Training Mixture-of-Experts Routers for Locality Against the Edge Memory-Bandwidth Wall: A Pre-Registered Negative Result with a Systems Measurement Study"
原文链接: "https://arxiv.org/abs/2608.18261"
发布日期: "2026-08-18"
发布时间校准状态: "found"
发布时间需复核: "否"
发布时间来源: "rule:local:strict_original_body"
发布时间证据: "**\\[v1\\]** Tue, 18 Aug 2026 19:23:32 UTC (41 KB)"
发布时间校准原因: "规则确认唯一严格发布时间，来源 local:strict_original_body"
发布时间校准置信度: "high"
发布时间候选数量: 18
发布时间严格候选数量: 6
发布时间原页读取状态: "source template page reused from URL open"
发布时间未找到原因: ""
发布时间校准时间: "2026-08-20T14:22:45+08:00"
发布时间仲裁状态: "skipped"
发布时间仲裁尝试次数: 0
发布时间仲裁耗时毫秒: 0
发现时间: "2026-08-20T14:19:36+08:00"
入库时间: "2026-08-20T06:22:45.125Z"
来源平台: "arXiv 学术论文搜索"
搜索渠道: "source_template"
搜索词: "https://arxiv.org/search/?query=Scale-up&searchtype=all"
匹配关键词:
  - "Scale-up"
  - "GPU"
  - "bandwidth"
  - "AI"
相关厂家:
  []
相关专家:
  []
内容类型: "网页"
抓取工具: "Free Fetch + Defuddle"
清洗工具: "Defuddle Markdown + Defuddle/Readability 正文提取"
原始附件:
  []
AI优质: "否"
AI打分: 15
AI分档: "非优质"
AI质检状态: "不通过"
AI打分理由: "论文专注MoE路由器缓存与内存带宽，未涉及超节点/AI Rack/机柜级基础设施、供电、液冷或互连等核心主题，属于模型推理优化研究，与技术范围完全无关。"
AI质检模型: "tx-deepseek-v4-flash"
AI质检时间: "2026-08-20T14:23:36+08:00"
AI主题相关性: 0
AI来源权威性: 10
AI新颖性: 2
AI技术细节: 3
AI商业部署信号: 0
AI完整性: 0
AI摘要: "研究者在8GB GPU上服务Qwen3-235B时测量到内存带宽墙，并通过预注册实验训练137M MoE模型加入局部性损失，发现缓存未命中最多可降60%，但所有配置均未通过困惑度变化不超过1%的预设门槛。"
AI摘要模型: "ali-deepseek-v4-flash"
AI摘要时间: "2026-09-07T03:21:38.704Z"
采集批次: "2026年8月20日14点19分32秒"
采集批次ID: "20260820-141932-079"
去重键: "https://arxiv.org/abs/2608.18261"
---

## Computer Science > Artificial Intelligence

## Title:Cacheable by Design? Training Mixture-of-Experts Routers for Locality Against the Edge Memory-Bandwidth Wall: A Pre-Registered Negative Result with a Systems Measurement Study

Authors:[Shriniwas Ramesh Suram](https://arxiv.org/search/cs?searchtype=author&query=Suram,+S+R)

[View PDF](https://arxiv.org/pdf/2608.18261) [HTML (experimental)](https://arxiv.org/html/2608.18261v1)

> Abstract:Serving a 235B-parameter Mixture-of-Experts (MoE) model on a single 8 GB GPU is bottlenecked not by compute but by memory bandwidth: decode must stream each token's active experts from whichever tier holds them, and on consumer hardware most experts sit on an SSD far slower than RAM. We quantify this bandwidth wall on Qwen3-235B (Q4\_K\_M, 134 GB): measured decode is 0.44 tok/s warm, matching a bytes-per-token / bandwidth model, while a batching scheme that should amortize one disk sweep instead collapses at batch 32 from paging thrash. We build llama-moe-trace, a zero-surgery router-telemetry tool, and measure routing on Qwen3-30B: adjacent-token expert reuse is 2.0x chance, 95% of traffic uses 52.5% of experts, and an LRU cache of 13.4% of experts serves 66% of requests. We then ask whether cacheability is trainable: we pre-register training of 137M MoE language models with auxiliary locality and domain router losses, under joint criteria on cache-miss reduction and perplexity. The mechanism works (misses down up to 60%; a 99% static-pin hit rate) but every configuration fails the pre-registered <=1% perplexity gate -- miss reduction and quality are tightly coupled. Concurrent StickyMoE reports the same loss as near-free on single-domain sub-25M models; on multi-domain 137M we find the tax real. Our contribution is this pre-registered, stricter-criterion, multi-domain evaluation plus edge-serving measurements. A 340M rung shows the tax does not shrink with scale (it rises slightly). We further show training-free cache-aware rerouting stacks with trained locality -- together ~80% miss reduction at <=3.4% perplexity at both sizes, far cheaper than either alone -- while domain-primed prefetching does not help. All code, traces, and the pre-registration are released.

| Comments: |  |
| --- | --- |
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | [arXiv:2608.18261](https://arxiv.org/abs/2608.18261) \[cs.AI\] |
|  | (or [arXiv:2608.18261v1](https://arxiv.org/abs/2608.18261v1) \[cs.AI\] for this version) |
|  | [https://doi.org/10.48550/arXiv.2608.18261](https://doi.org/10.48550/arXiv.2608.18261) |

## Submission history

From: Shriniwas Ramesh Suram \[[view email](https://arxiv.org/show-email/929fc20c/2608.18261)\]  
**\[v1\]** Tue, 18 Aug 2026 19:23:32 UTC (41 KB)

[Which authors of this paper are endorsers?](https://arxiv.org/auth/show-endorsers/2608.18261) | Disable MathJax ([What is MathJax?](https://info.arxiv.org/help/mathjax.html))