---
格式版本: 2
标题: "Every Kernel Is a Join: Automatic Multi-GPU Parallelism for AI Computations in Einsummable"
原文链接: "https://arxiv.org/abs/2609.03905"
发布日期: "2026-09-03"
发布时间校准状态: "found"
发布时间需复核: "否"
发布时间来源: "rule:local:strict_original_body"
发布时间证据: "**\\[v1\\]** Thu, 3 Sep 2026 14:24:02 UTC (174 KB)"
发布时间校准原因: "规则确认唯一严格发布时间，来源 local:strict_original_body"
发布时间校准置信度: "high"
发布时间候选数量: 18
发布时间严格候选数量: 6
发布时间原页读取状态: "source template page reused from URL open"
发布时间未找到原因: ""
发布时间校准时间: "2026-09-06T21:24:00+08:00"
发布时间仲裁状态: "skipped"
发布时间仲裁尝试次数: 0
发布时间仲裁耗时毫秒: 0
发现时间: "2026-09-06T21:21:19+08:00"
入库时间: "2026-09-06T13:24:00.886Z"
来源平台: "arXiv 学术论文搜索"
搜索渠道: "source_template"
搜索词: "https://arxiv.org/search/?query=GPU&searchtype=all"
匹配关键词:
  - "GPU"
  - "AI"
相关厂家:
  []
相关专家:
  []
内容类型: "网页"
抓取工具: "Free Fetch + Defuddle"
清洗工具: "Defuddle Markdown + Defuddle/Readability 正文提取"
原始附件:
  []
AI优质: "否"
AI打分: 32
AI分档: "非优质"
AI质检状态: "不通过"
AI打分理由: "该论文是关于多GPU服务器上自动并行化的学术预印本，聚焦软件编译优化，未涉及超节点/AI Rack硬件架构、互连、供电、散热或量产部署，与项目主题明显无关。"
AI质检模型: "zj-deepseek-v4-flash"
AI质检时间: "2026-09-06T21:24:32+08:00"
AI主题相关性: 3
AI来源权威性: 8
AI新颖性: 12
AI技术细节: 4
AI商业部署信号: 0
AI完整性: 5
AI摘要: "Einsummable是一个自动多GPU并行原型系统，接受PyTorch式AI计算描述，无需人工设备分配、分片或通信标注，将每个算子建模为关系连接并聚合，自动搜索分解和生成通信方案。"
AI摘要模型: "ali-deepseek-v4-flash"
AI摘要时间: "2026-09-06T23:44:26.530Z"
采集批次: "2026年9月6日19点27分09秒"
采集批次ID: "20260906-192709-237"
去重键: "https://arxiv.org/abs/2609.03905"
---

## Computer Science > Distributed, Parallel, and Cluster Computing

## Title:Every Kernel Is a Join: Automatic Multi-GPU Parallelism for AI Computations in Einsummable

[View PDF](https://arxiv.org/pdf/2609.03905) [HTML (experimental)](https://arxiv.org/html/2609.03905v1)

> Abstract:Distributing an AI computation across the GPUs of a multi-GPU server is one of the central problems in systems-for-AI. We present Einsummable, a prototype system that accepts a PyTorch-like description of an AI computation and automatically distributes it across a multi-GPU server, with no device assignments, sharding annotations, or communication operations written by the programmer. Einsummable models every operation as a relational join followed by an aggregation over tensor relations, in which the tuples contain sub-tensors. Each operation exposes its possible decompositions through what we call "join-agg specs". An optimizer then selects decompositions across the whole computation to minimize a communication-cost proxy. Because it searches decompositions rather than a menu of named strategies, Einsummable discovers plans that mesh-based auto-parallelizers cannot. Each decomposed operation is implemented by synthesizing an exchange program, which is a topology-aware generalization of Volcano's exchange operator. Einsummable invokes no canned collectives: all communication and aggregation is special-purpose, derived at compile time. Despite being fully automatic, Einsummable can outperform custom-designed implementations. For example, on LLaMA transformer blocks on an eight-GPU A100 server, Einsummable achieves a geometric-mean runtime of 8.97 ms, versus 13.80 ms for hand-tuned PyTorch and 15.90 ms for vLLM.

| Subjects: | Distributed, Parallel, and Cluster Computing (cs.DC) |
| --- | --- |
| Cite as: | [arXiv:2609.03905](https://arxiv.org/abs/2609.03905) \[cs.DC\] |
|  | (or [arXiv:2609.03905v1](https://arxiv.org/abs/2609.03905v1) \[cs.DC\] for this version) |
|  | [https://doi.org/10.48550/arXiv.2609.03905](https://doi.org/10.48550/arXiv.2609.03905) |

## Submission history

From: Zhimin Ding \[[view email](https://arxiv.org/show-email/ddc2159c/2609.03905)\]  
**\[v1\]** Thu, 3 Sep 2026 14:24:02 UTC (174 KB)

[Which authors of this paper are endorsers?](https://arxiv.org/auth/show-endorsers/2609.03905) | Disable MathJax ([What is MathJax?](https://info.arxiv.org/help/mathjax.html))