---
格式版本: 2
标题: "SQuad: Sub-Quadratic Attention Distillation for Efficient Video Generation"
原文链接: "https://arxiv.org/abs/2608.16585"
发布日期: "2026-08-17"
发布时间校准状态: "found"
发布时间需复核: "否"
发布时间来源: "rule:scrape:strict_html_body"
发布时间证据: "citation_date: 2026/08/17"
发布时间校准原因: "规则确认唯一严格发布时间，来源 scrape:strict_html_body"
发布时间校准置信度: "high"
发布时间候选数量: 6
发布时间严格候选数量: 6
发布时间原页读取状态: ""
发布时间未找到原因: ""
发布时间校准时间: "2026-08-18T19:33:45+08:00"
发布时间仲裁状态: "skipped"
发布时间仲裁尝试次数: 0
发布时间仲裁耗时毫秒: 0
发现时间: "2026-08-18T19:33:29+08:00"
入库时间: "2026-08-18T11:34:15.393Z"
来源平台: "arXiv 学术论文搜索"
搜索渠道: "source_template"
搜索词: "https://arxiv.org/search/?query=latency&searchtype=all"
匹配关键词:
  - "latency"
相关厂家:
  []
相关专家:
  []
内容类型: "网页"
抓取工具: "Free Fetch + Defuddle"
清洗工具: "Defuddle Markdown + Defuddle/Readability 正文提取"
原始附件:
  []
AI优质: "否"
AI打分: 28
AI分档: "非优质"
AI质检状态: "不通过"
AI打分理由: "论文主题为视频生成中的注意力蒸馏，与超节点/AI Rack/机柜级AI基础设施完全无关，仅命中'latency'一词，属于明显无关内容。"
AI质检模型: "tx-deepseek-v4-flash"
AI质检时间: "2026-08-18T19:36:25+08:00"
AI主题相关性: 0
AI来源权威性: 10
AI新颖性: 5
AI技术细节: 5
AI商业部署信号: 0
AI完整性: 8
AI摘要: "SQuad提出一种次二次注意力蒸馏方法，将预训练视频扩散Transformer蒸馏为低复杂度注意力结构，计算量从O(n²)降至O(n√n)。"
AI摘要模型: "ali-deepseek-v4-flash"
AI摘要时间: "2026-09-07T03:25:34.023Z"
采集批次: "2026年8月18日15点52分05秒"
采集批次ID: "20260818-155205-090"
去重键: "https://arxiv.org/abs/2608.16585"
---

## Computer Science > Computer Vision and Pattern Recognition

## Title:SQuad: Sub-Quadratic Attention Distillation for Efficient Video Generation

[View PDF](https://arxiv.org/pdf/2608.16585)

> Abstract:Video Diffusion Transformers (DiTs) spend most of their compute inside the Self-Attention operation, whose cost grows quadratically, $\\mathcal{O}(n^2)$, with the number of latent tokens $n$. For the task of video generation, the token count is large, so this term dominates runtime and memory, and thereby caps the resolution and duration we can generate. Linear $\\mathcal{O}(n)$ and low-rank $\\mathcal{O}(nk)$ surrogates of Self-Attention trade the full softmax $QK^T$ for cheaper kernels, but rarely recover the original's expressivity, leaving a stubborn quality gap. Motivated by this, we propose SQuad, a Sub-Quadratic Attention Distillation framework that achieves a complexity of $\\mathcal{O}(n\\sqrt{n})$ in the resulting distilled Attention, naturally balancing the efficiency v/s expressivity trade-off. Instead of training our own Video DiT from scratch, which is prohibitively expensive, we fit a pretrained full softmax Self-Attention DiT into our proposed SQuad-Attention one by distilling the former in two stages: Flow-Matching Supervised Fine-Tuning (SFT), followed by improved Distribution Matching Distillation (DMD2) which additionally makes the sampling more efficient. On the Wan~2.2 5B text-to-video model, SQuAD matches the quadratic teacher on VBench ($83.20$ v/s $83.08$) while cutting the per-step per-block attention FLOPs by $\\sim$$67\\times$ and attention latency by $\\sim$$11\\times$, and end-to-end DiT latency by 2$\\times$, all while also generating a video in only $6$ Neural Functional Evaluations (NFEs) instead of the default $100$.

| Subjects: | Computer Vision and Pattern Recognition (cs.CV) |
| --- | --- |
| Cite as: | [arXiv:2608.16585](https://arxiv.org/abs/2608.16585) \[cs.CV\] |
|  | (or [arXiv:2608.16585v1](https://arxiv.org/abs/2608.16585v1) \[cs.CV\] for this version) |
|  | [https://doi.org/10.48550/arXiv.2608.16585](https://doi.org/10.48550/arXiv.2608.16585) |

## Submission history

From: Mohsen Ghafoorian \[[view email](https://arxiv.org/show-email/72534a3d/2608.16585)\]  
**\[v1\]** Mon, 17 Aug 2026 13:47:44 UTC (2,921 KB)

[Which authors of this paper are endorsers?](https://arxiv.org/auth/show-endorsers/2608.16585) | Disable MathJax ([What is MathJax?](https://info.arxiv.org/help/mathjax.html))