---
格式版本: 2
标题: "Instella-MoE Technical Report"
原文链接: "https://arxiv.org/abs/2609.00791"
发布日期: "2026-09-01"
发布时间校准状态: "found"
发布时间需复核: "否"
发布时间来源: "rule:local:strict_original_body"
发布时间证据: "**\\[v1\\]** Tue, 1 Sep 2026 06:38:17 UTC (338 KB)"
发布时间校准原因: "规则确认唯一严格发布时间，来源 local:strict_original_body"
发布时间校准置信度: "high"
发布时间候选数量: 18
发布时间严格候选数量: 6
发布时间原页读取状态: "source template page reused from URL open"
发布时间未找到原因: ""
发布时间校准时间: "2026-09-02T18:52:13+08:00"
发布时间仲裁状态: "skipped"
发布时间仲裁尝试次数: 0
发布时间仲裁耗时毫秒: 0
发现时间: "2026-09-02T18:51:36+08:00"
入库时间: "2026-09-02T10:52:13.837Z"
来源平台: "arXiv 学术论文搜索"
搜索渠道: "source_template"
搜索词: "https://arxiv.org/search/?query=GPU&searchtype=all"
匹配关键词:
  - "GPU"
  - "AI"
相关厂家:
  - "AMD"
相关专家:
  []
内容类型: "网页"
抓取工具: "Free Fetch + Defuddle"
清洗工具: "Defuddle Markdown + Defuddle/Readability 正文提取"
原始附件:
  []
AI优质: "否"
AI打分: 34
AI分档: "非优质"
AI质检状态: "不通过"
AI打分理由: "正文主线是Instella-MoE语言模型的架构、训练流程与模型基准，MI300X/MI325X仅作为训练硬件，未提供机架拓扑、互连带宽、功耗散热、RAS或可复用机架级基础设施机制。arXiv论文来源可追溯，但当前页面仅有摘要。固定知识库未见该模型既有记录，但未命中不能证明首次出现；可确认的新增主要是16B总参数、每token激活2.8B及Gated MLA、FarSkip-Collective等模型信息，无客户、量产或部署信号。命中“应用与模型效率”及摘要壳否决项，当前页面不值得作为超节点业务信息源。"
AI质检模型: "gpt-5.6-sol"
AI质检时间: "2026-09-02T18:53:00+08:00"
AI主题相关性: 3
AI来源权威性: 12
AI新颖性: 10
AI技术细节: 3
AI商业部署信号: 0
AI完整性: 6
AI评分提示词版本: "v17-精简生产版"
AI评分提示词SHA256: "48fb9777f386026761b4873eaff30807694fb11e9b352d7c69bf2dfde750cc7d"
AI评分知识库版本: "knowledge_base_v1-20260819+runtime.81"
AI评分知识库SHA256: "3b93d12e47749b3512f545f51c44c011bdc0931677c2cfe61e4df59a2b1a5a48"
AI评分知识库检索词: "[\"GPU\",\"AMD\",\"Intel\",\"PDF\",\"arxiv.org/pdf/2609.00791\",\"HTML\",\"arxiv.org/html/2609.00791v1\",\"MI300X\",\"MI325X\",\"GPUs\",\"v1\",\"MLA\"]"
AI评分知识库命中: "[{\"id\":\"july-correct-0081\",\"title\":\"AAI 2026: 6th Gen AMD EPYC Server CPUs Power the Agentic Data Center\",\"sourceType\":\"labeled_article\",\"time\":\"2026-07\",\"matchedTerms\":[\"GPU\",\"AMD\",\"Intel\",\"PDF\",\"HTML\",\"GPUs\"],\"rank\":-16.166559443408286},{\"id\":\"july-correct-0075\",\"title\":\"From EPYC to Helios, AMD and Meta are Scaling the Future of AI Together\",\"sourceType\":\"labeled_article\",\"time\":\"2026-07\",\"matchedTerms\":[\"GPU\",\"AMD\",\"HTML\",\"MI300X\",\"GPUs\"],\"rank\":-13.388268694472337},{\"id\":\"july-correct-0077\",\"title\":\"AMD and OpenAI: Collaborating Across Every Layer of the Stack\",\"sourceType\":\"labeled_article\",\"time\":\"2026-07\",\"matchedTerms\":[\"GPU\",\"AMD\",\"HTML\",\"MI300X\",\"GPUs\"],\"rank\":-13.018723815850631},{\"id\":\"july-correct-0010\",\"title\":\"Schneider Electric and AMD release first Helios platform reference design to accelerate AI Factory deployment\",\"sourceType\":\"labeled_article\",\"time\":\"2026-07\",\"matchedTerms\":[\"GPU\",\"AMD\",\"PDF\",\"GPUs\",\"v1\"],\"rank\":-12.741008981700478},{\"id\":\"july-correct-0111\",\"title\":\"StrataCL: Fabric-Native Communication Library for Production Supernodes\",\"sourceType\":\"labeled_article\",\"time\":\"2026-07\",\"matchedTerms\":[\"PDF\",\"HTML\",\"v1\"],\"rank\":-11.543771882503226}]"
AI摘要: "Instella-MoE是一个完全开源的混合专家语言模型，总参数160亿、每 token 激活28亿，基于AMD MI300X/MI325X GPU从零训练；"
AI摘要模型: "ali-deepseek-v4-flash"
AI摘要时间: "2026-09-02T17:13:28.869Z"
采集批次: "2026年9月2日18点28分48秒"
采集批次ID: "20260902-182848-894"
去重键: "https://arxiv.org/abs/2609.00791"
---

## Computer Science > Computation and Language

## Title:Instella-MoE Technical Report

[View PDF](https://arxiv.org/pdf/2609.00791) [HTML (experimental)](https://arxiv.org/html/2609.00791v1)

> Abstract:In this work, we introduce Instella-MoE, a fully open Mixture-of-Experts (MoE) language model with 16 billion total parameters and 2.8 billion active parameters per token, trained entirely from scratch on AMD Instinct MI300X and MI325X GPUs. Instella-MoE combines a sparsely activated MoE design with architectural and system-level innovations, including Gated Multi-head Latent Attention (Gated MLA) and FarSkip-Collective connectivity, enabling efficient large-scale training and inference. The model is developed through a multi-stage pipeline comprising pre-training, mid-training, long-context extension, supervised fine-tuning with feedback-driven data curation, direct preference optimization, and reinforcement learning with Multi-Teacher On-Policy Distillation. Instella-MoE achieves an average score of 76.7 across standard pre-training benchmarks, outperforming prior fully open models including OLMo-3-7B, SmolLM3-3B, and OLMoE-1B-7B, while remaining competitive with open-weight MoE and dense baselines at comparable active-parameter scales, including Moonlight-16B-A3B and Qwen3.5-4B. After post-training, our final Think checkpoint achieves an average score of 73.2 across instruction-following, reasoning, math, coding, and chat benchmarks, outperforming both fully open and open-weight models with comparable or larger active parameter counts in our evaluation. To support transparent and reproducible research, we release the complete Instella-MoE model flow, including model weights, training configurations, data mixtures, and training code. Together, these contributions establish Instella-MoE a strong, fully open foundation for efficient, high-performing MoE models and reproducible research.

| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| --- | --- |
| Cite as: | [arXiv:2609.00791](https://arxiv.org/abs/2609.00791) \[cs.CL\] |
|  | (or [arXiv:2609.00791v1](https://arxiv.org/abs/2609.00791v1) \[cs.CL\] for this version) |
|  | [https://doi.org/10.48550/arXiv.2609.00791](https://doi.org/10.48550/arXiv.2609.00791) |

## Submission history

From: Jiang Liu \[[view email](https://arxiv.org/show-email/09a0664d/2609.00791)\]  
**\[v1\]** Tue, 1 Sep 2026 06:38:17 UTC (338 KB)

[Which authors of this paper are endorsers?](https://arxiv.org/auth/show-endorsers/2609.00791) | Disable MathJax ([What is MathJax?](https://info.arxiv.org/help/mathjax.html))