---
格式版本: 2
标题: "AWS, Google, Microsoft and OCI Boost AI Inference Performance for Cloud Customers With NVIDIA Dynamo | NVIDIA Blog"
原文链接: "https://blogs.nvidia.com/blog/think-smart-dynamo-ai-inference-data-center/"
发布日期: "2026-06-30"
发布时间校准状态: "found"
发布时间来源: "llm:strict_original_body"
发布时间证据: "time class=related-news-date nvidia-article-date datetime=2026-06-30T08:00:57-07:00: Jun 30, 2026"
发布时间校准原因: "该日期来自HTML中的time标签，class包含nvidia-article-date，且位于标题附近，符合文章发布时间的特征。"
发布时间校准置信度: "1"
发布时间候选数量: 38
发布时间严格候选数量: 12
发布时间原页读取状态: "原页面来自已抓取 HTML"
发布时间未找到原因: "候选日期无效或 LLM 未确认"
发布时间校准时间: "2026-07-20T12:26:54+08:00"
发现时间: "2026-07-20T09:26:26+08:00"
入库时间: "2026-07-20T04:30:38.899Z"
来源平台: "NVIDIA Blog 搜索"
搜索渠道: "source_template"
搜索词: "https://blogs.nvidia.com/?s=AWS"
匹配关键词:
  - "GPU"
  - "NVL72"
相关厂家:
  - "AWS"
  - "NVIDIA"
  - "Microsoft"
  - "Google"
  - "Oracle"
相关专家:
  []
内容类型: "网页"
抓取工具: "AgentKey Scrape"
清洗工具: "AgentKey Markdown + LLM 正文裁剪"
原始附件:
  []
AI优质: "否"
AI打分: 72
AI分档: "召回候选"
AI质检状态: "不通过"
AI打分理由: "NVIDIA官方发布，涉及GB200/GB300 NVL72机柜级系统与多云Kubernetes集成部署，但核心聚焦软件推理平台Dynamo/Grove与推理性能优化，硬件架构、供电散热互连等超节点物理层细节较少，商业落地信号偏软件集成。"
AI质检模型: "qwen3.6-plus"
AI质检时间: "2026-07-20T12:30:38+08:00"
AI主题相关性: 12
AI来源权威性: 14
AI新颖性: 16
AI技术细节: 10
AI商业部署信号: 10
AI完整性: 10
图片摘要:
  - "★ ./assets/img-c0485e50.jpg | photo | Jensen Huang展示Blackwell NVL72相比Hopper的10倍性能提升，含Token吞吐量与交互性对比曲线。"
  - "✗ ./assets/img-83d2f103.jpg | photo | 相关文章推荐配图，非正文内容"
  - "✗ ./assets/img-1f6575f1.png | diagram | 相关文章推荐配图（Vera CPU），与正文无关"
  - "✗ ./assets/img-3511966c.jpg | photo | 相关文章推荐配图/品牌宣传，非正文内容"
  - "✗ ./assets/img-e47e99dc.png | infographic | 相关文章推荐配图，非正文内容"
采集批次: "2026年7月20日9点23分34秒"
采集批次ID: "20260720-092334-062"
去重键: "https://blogs.nvidia.com/blog/think-smart-dynamo-ai-inference-data-center"
---

*Editor’s note: This post is part of* [*Think SMART*](https://blogs.nvidia.com/blog/tag/think-smart)*, a series focused on how leading AI service providers, developers and enterprises can boost their* [*inference performance*](https://developer.nvidia.com/deep-learning-performance-training-inference/ai-inference) *and return on investment with the latest advancements from NVIDIA’s full-stack* [*inference platform*](https://www.nvidia.com/en-us/solutions/ai/inference/)*.*

NVIDIA Blackwell delivers the highest performance and efficiency, and lowest total cost of ownership across every tested model and use case in the recent independent [SemiAnalysis InferenceMAX v1 benchmark](https://blogs.nvidia.com/blog/blackwell-inferencemax-benchmark-results/).

![图片](./assets/img-c0485e50.jpg)

NVIDIA CEO Jensen Huang highlighted at NVIDIA GTC Washington, D.C., how Blackwell delivers 10x the performance of NVIDIA Hopper, enabling 10x the revenue.

Achieving this industry-leading performance for today’s most complex [AI models](https://developer.nvidia.com/ai-models), such as large-scale mixture-of-experts (MoE) models, requires distributing (or disaggregating) [inference](https://www.nvidia.com/en-us/glossary/ai-inference/) across multiple servers (nodes) to serve millions of concurrent users and deliver faster responses.

The [NVIDIA Dynamo](https://www.nvidia.com/en-us/ai/dynamo/) software platform unlocks these powerful multi-node capabilities for production, enabling enterprises to achieve this same benchmark-winning performance and efficiency across their existing cloud environments. Read on to learn how the shift to multi-node inference is driving performance, as well as how cloud platforms are putting this technology to work.

## Tapping Disaggregated Inference for Optimized Performance

For AI models that fit on a single GPU or server, developers often run many identical replicas of the model in parallel across multiple nodes to deliver high throughput. In a [recent paper](https://signal65.com/research/ai/azure-gb300-performance/), Russ Fellows, principal analyst at Signal65, showed that this approach achieved an industry-first record aggregate throughput of 1.1 million [tokens](https://blogs.nvidia.com/blog/ai-tokens-explained/) per second with 72 NVIDIA Blackwell Ultra GPUs.

When scaling AI models to serve many concurrent users in real time, or when managing demanding workloads with long input sequences, using a technique called [disaggregated serving](https://www.nvidia.com/en-us/glossary/disaggregated-serving/) unlocks further performance and efficiency gains.

Serving AI models involves two phases: processing the input prompt (prefill) and generating the output (decode). Traditionally, both phases run on the same GPUs, which can create inefficiencies and resource bottlenecks.

Disaggregated serving solves this by intelligently distributing these tasks to independently optimized GPUs. This approach ensures that each part of the workload runs with the optimization techniques best suited for it, maximizing overall performance. For today’s large [AI reasoning](https://www.nvidia.com/en-us/glossary/ai-reasoning/) and MoE models, such as DeepSeek-R1, disaggregated serving is essential.

NVIDIA Dynamo easily brings features like disaggregated serving to production scale across GPU clusters.

It’s already delivering value.

[Baseten](https://www.baseten.co/blog/how-baseten-achieved-2x-faster-inference-with-nvidia-dynamo/?utm_source=partner&utm_medium=Nvidia&utm_campaign=fall_into_inference_2025&utm_term=&utm_content=blog), for example, used NVIDIA Dynamo to speed up inference serving for long-context code generation by 2x and increase throughput by 1.6x, all without incremental hardware costs. Such software-driven performance boosts enable AI providers to significantly reduce the costs to manufacture intelligence.

## Scaling Disaggregated Inference in the Cloud

Much like it did for large-scale AI training, Kubernetes — the industry standard for containerized application management — is well-positioned to scale disaggregated serving across dozens or even hundreds of nodes for enterprise-scale AI deployments.

With NVIDIA Dynamo now integrated into managed Kubernetes services from all major cloud providers, customers can scale multi-node inference across NVIDIA Blackwell systems, including [GB200](https://developer.nvidia.com/blog/how-nvidia-gb200-nvl72-and-nvidia-dynamo-boost-inference-performance-for-moe-models/) and [GB300 NVL72](https://www.nvidia.com/en-us/data-center/gb300-nvl72/), with the performance, flexibility and reliability that enterprise AI deployments demand.

- [Amazon Web Services](https://aws.amazon.com/blogs/machine-learning/accelerate-generative-ai-inference-with-nvidia-dynamo-and-amazon-eks/) is accelerating generative AI inference for its customers with NVIDIA Dynamo and integrated with Amazon EKS.
- [Google Cloud](https://cloud.google.com/blog/products/compute/ai-inference-recipe-using-nvidia-dynamo-with-ai-hypercomputer) is providing Dynamo recipe to optimize large language model ([LLM](https://www.nvidia.com/en-us/glossary/large-language-models/)) inference at enterprise scale on its AI Hypercomputer.
- [Microsoft Azure](https://blog.aks.azure.com/2025/10/24/dynamo-on-aks) is enabling multi-node LLM inference with NVIDIA Dynamo and ND GB200-v6 GPUs on Azure Kubernetes Service.
- [Oracle Cloud Infrastructure](https://www.oracle.com/cloud/cloud-native/kubernetes-engine/) (OCI) is enabling multi-node LLM inferencing with OCI Superclusters and NVIDIA Dynamo.

The push towards enabling large-scale, multi-node inference extends beyond hyperscalers.

[Nebius](https://app.frame.io/reviews/cfcca4af-f3c9-4298-929f-ae8c4f3b490e/7b3701b8-75b8-404a-a337-903f089e497b?version=9a46c1c4-4f78-4ff8-be58-e3641c33d032), for example, is designing its cloud to serve inference workloads at scale, [built on NVIDIA accelerated computing infrastructure](https://nebius.com/newsroom/nebius-ai-cloud-arrives-in-uk-with-one-of-the-country-s-first-advanced-nvidia-ai-infrastructure-deploymentsbuilt) and working with NVIDIA Dynamo as an ecosystem partner.

Scaling AI Inference Performance in the Cloud with Nebius - YouTube

NVIDIA 2.23M subscribers

## Simplifying Inference on Kubernetes With NVIDIA Grove in NVIDIA Dynamo

Disaggregated AI inference requires coordinating a team of specialized components — prefill, decode, routing and more — each with different needs. The challenge for Kubernetes is no longer about running more parallel copies of a model, but rather about masterfully conducting these distinct components as one cohesive, high-performance system.

[NVIDIA Grove](https://developer.nvidia.com/grove), an application programming interface now available within NVIDIA Dynamo, allows users to provide a single, high-level specification that describes their entire inference system.

For example, in that single specification, a user could simply declare their requirements: “I need three GPU nodes for prefill and six GPU nodes for decode, and I require all nodes for a single model replica to be placed on the same high-speed interconnect for the quickest possible response.”

From that specification, Grove automatically handles all the intricate coordination: scaling related components together while maintaining correct ratios and dependencies, starting them in the right order and placing them strategically across the cluster for fast, efficient communication. Learn more about how to get started with NVIDIA Grove in this [technical deep dive](https://developer.nvidia.com/blog/streamline-complex-ai-inference-on-kubernetes-with-nvidia-grove/).

As AI inference becomes increasingly distributed, the combination of Kubernetes and NVIDIA Dynamo with NVIDIA Grove simplifies how developers build and scale intelligent applications.

*Try* [*NVIDIA’s AI-at-scale simulation*](https://www.nvidia.com/en-us/solutions/ai/inference/#:~:text=Explore%20the%20Story%20Behind%20AI%20at%20Scale) *to see how hardware and deployment choices affect performance, efficiency and user experience.* *To dive deeper on disaggregated serving and learn how Dynamo and NVIDIA GB200 NVL72 systems work together to boost inference performance, read this* [*technical blog*](https://developer.nvidia.com/blog/how-nvidia-gb200-nvl72-and-nvidia-dynamo-boost-inference-performance-for-moe-models/)*.*

*For [monthly updates](https://info.nvidia.com/rs/156-OFN-742/images/NVIDIA-Think-SMART-Inference-Signals-October-2025-Recap.html?version=0), sign up for the [NVIDIA Think SMART newsletter](https://www.nvidia.com/en-us/solutions/ai/inference/?modal=sign-up-form).*

![图片](./assets/img-c0485e50.jpg)NVIDIA CEO Jensen Huang highlighted at NVIDIA GTC Washington, D.C., how Blackwell delivers 10x the performance of NVIDIA Hopper, enabling 10x the revenue.