---
格式版本: 2
标题: "Serve Qwen3.8-2.4T-A95B, a 2.4T-Parameter Model, with Configurable Reasoning on NVIDIA GB300 NVL72"
原文链接: "https://developer.nvidia.com/blog/serve-qwen3-8-2-4t-a95b-a-2-4t-parameter-model-with-configurable-reasoning-on-nvidia-gb300-nvl72/"
发布日期: "2026-08-12"
发布时间校准状态: "found"
发布时间需复核: "否"
发布时间来源: "rule:scrape:strict_html_metadata"
发布时间证据: "article:published_time: 2026-08-12T18:23:13+00:00"
发布时间校准原因: "规则确认唯一严格发布时间，来源 scrape:strict_html_metadata"
发布时间校准置信度: "high"
发布时间候选数量: 8
发布时间严格候选数量: 1
发布时间原页读取状态: "source template page reused from URL open"
发布时间未找到原因: ""
发布时间校准时间: "2026-08-13T23:02:27+08:00"
发布时间仲裁状态: "skipped"
发布时间仲裁尝试次数: 0
发布时间仲裁耗时毫秒: 0
发现时间: "2026-08-13T23:02:21+08:00"
入库时间: "2026-08-13T15:02:27.467Z"
来源平台: "固定入口"
搜索渠道: "fixed_url"
搜索词: "https://developer.nvidia.com/blog/"
匹配关键词:
  - "NVL72"
  - "GPU"
  - "NVLink"
  - "performance"
  - "latency"
  - "throughput"
  - "AI"
相关厂家:
  - "NVIDIA"
相关专家:
  []
内容类型: "网页"
抓取工具: "Free Fetch + Defuddle"
清洗工具: "Defuddle Markdown + Defuddle/Readability 正文提取"
原始附件:
  []
AI优质: "是"
AI打分: 84
AI分档: "高置信优质"
AI质检状态: "通过"
AI打分理由: "NVIDIA官方博客，讨论Qwen3.8模型在GB300 NVL72（rack-scale AI系统）上的部署与性能，包含架构、NVLink、FP8吞吐等技术细节，来源权威，信息完整，主题高度相关。"
AI质检模型: "ali-deepseek-v4-flash"
AI质检时间: "2026-08-13T23:02:43+08:00"
AI主题相关性: 18
AI来源权威性: 15
AI新颖性: 15
AI技术细节: 16
AI商业部署信号: 10
AI完整性: 10
AI摘要: "阿里巴巴发布开源模型Qwen3.8-2.4T-A95B（Qwen3.8-Max），总参数2.4T、每token激活95B，采用细粒度MoE与混合注意力架构，支持百万级上下文和可配置推理深度。"
AI摘要模型: "ali-deepseek-v4-flash"
AI摘要时间: "2026-09-07T03:39:39.643Z"
采集批次: "2026年8月13日23点02分20秒"
采集批次ID: "20260813-230220-895"
去重键: "https://developer.nvidia.com/blog/serve-qwen3-8-2-4t-a95b-a-2-4t-parameter-model-with-configurable-reasoning-on-nvidia-gb300-nvl72"
---

Alibaba released the open weights for Qwen3.8-2.4T-A95B (Qwen3.8-Max), its largest open-weight model, bringing near-frontier capabilities to the open ecosystem. It has 2.4T total parameters with 95B activated per token. It has 2.4T total parameters with 95B activated per token. It’s a fine-grained mixture of experts (MoE) architecture with a hybrid of full and linear attention, a context window of up to one million tokens, and an output length of up to 128K, designed for demanding reasoning and agentic workloads.

Deploying a 2.4T parameter open-weight model requires data-center-scale accelerated compute. Inference at this scale depends on extreme co-design across chips, system architecture, and software. NVIDIA is working with the open-source ecosystem to bring the model to multinode deployments through optimized kernels, inference runtimes, and distributed serving recipes.

Without additional model tuning, the model achieves a throughput of over 4K tokens per second per GPU and over 350 tokens per second per user on NVIDIA GB300 NVL72 in FP8 precision on Day 0. Further optimizations, including NVFP4 precision, are expected to deliver enhanced performance gains over time.

## Architectural innovations for long-context inference

Qwen3.8-2.4T-A95B is built for the hardest agentic workloads like coding, large-scale document analysis, and long-running multi-step workflows. Unlike chat-first models that send a single prompt and receive a single reply, agentic applications accumulate system instructions, tool outputs, retrieved documents, code, logs, and multi-step reasoning traces across a workflow. As context grows, attention, compute, and KV cache memory become the binding constraints.

The full-attention and linear-attention hybrid architecture addresses this, and the model alternates between the two. In the full-attention layers, every token attends to every other token, and in the linear-attention layers, the growing KV cache is replaced with a bounded recurrent state. Qwen3.8-2.4T-A95B keeps both compute and memory bounded as context scales to up to one million tokens.

Fine-grained MoE makes the 2.4T parameter count practical to serve. Instead of a small number of large experts, capacity is distributed across a larger population of smaller experts, improving specialization and routing efficiency per unit of activated compute. A learned router activates only the experts needed per token, so serving costs track active parameters, not the full 2.4T parameters, delivering frontier-scale capacity at a fraction of the cost of a comparable dense model.

Built-in reasoning controls (low/high/xhigh) enable developers to configure inference depth per request, trading compute for reasoning quality depending on the task: dial up for complex multi-step reasoning or dial down for high-throughput document processing.

![A diagram showing how full attention and linear attention with a mixture-of-experts handle large contexts efficiently with far less memory and compute. ](https://developer-blogs.nvidia.com/wp-content/uploads/2026/08/Blog-Arch-Diagram.webp)

Figure 1. Overview of the Qwen3.8-2.4T-A95B linear gated delta networks plus full attention with fine-grained MoE architecture

## Qwen3.8-2.4T-A95B optimized performance on GB300 NVL72

The GB300 NVL72 features a rack-scale architecture that integrates 72 NVIDIA Blackwell Ultra GPUs into a single platform. Its large, 72-GPU NVIDIA NVLink domain enables efficient all-to-all communication at 130 TB/s, eliminating bottlenecks that appear when expert traffic must cross traditional off-the-shelf networks.

Out of the box, Qwen3.82.4T-A95B running on NVIDIA Blackwell GB300 NVL72 delivers over **4K tokens per second per GPU and over 350 tokens per second per user**, enabling AI factories to run large-parameter models in production at high throughput and low latency.

![Qwen3.8-2.4T-A95B FP8 performance on NVIDIA GB300 NVL72  throughput vs. interactivity using TensorRT-LLM.](https://developer-blogs.nvidia.com/wp-content/uploads/2026/08/Qwen3.8-NVL72-Performance.webp)

Figure 2. A Pareto curve showing Qwen3.8-2.4T-A95B achieving over 4K tokens per second per GPU at peak throughput on NVIDIA GB300 NVL72

## Post-train Qwen3.8-2.4T-A95B and choose a serving path

NVIDIA supports multiple inference stacks to meet a variety of developer needs. [SGLang](https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8), [vLLM](https://recipes.vllm.ai/Qwen/Qwen3.8-2.4T-A95B), and [NVIDIA Dynamo](https://docs.nvidia.com/dynamo/dev/recipes/qwen-3-8-2-4t-a95b) provide open-source inference recipes for developers who require greater control over performance on the NVIDIA-accelerated platform.

It’s also available to deploy via a [model-free NVIDIA NIM](https://catalog.ngc.nvidia.com/orgs/nim/nvidia/containers/sglang-model-free-nim/-), a single inference container that serves any supported model. Download the model weights and deploy on Day-0 to serve fine-tuned checkpoints, and scale to production.

Developers can post-train the model for domain-specific use cases using [NVIDIA NeMo AutoModel](https://github.com/NVIDIA-NeMo/Automodel/tree/main/docs/model-coverage/llm/qwen/qwen3-8-max.mdx), a PyTorch-native fine-tuning library with Day-0 Hugging Face checkpoint support. Train directly on existing checkpoints without model conversion, with support for full SFT or memory-efficient LoRA fine-tuning.