Deploying Kimi K3 on AWS

Kimi K3，一个拥有2.8万亿参数的超级AI模型，正式登陆AWS平台，成为首个公开权重、可自主部署的“准万亿级”开源AI。这意味着，任何组织都能像搭积木一样，在自己的服务器上运行这个目前最强大的AI之一。通俗点说，这个模型就像一个由896位“专家”组成的智囊团，每次只调用其中16位专家处理问题，所以它虽然体量巨大（总参数2.8万亿），但实际每次计算只动用了1040亿参数，效率比前代提升了2.5倍。而且它能一次记住并处理长达100万token的上下文——相当于连续读完整部《三体》三部曲还不会忘记前面内容。AWS提供了两种部署方式：一种是“开箱即用”的托管方案，另一种是高度自由的云原生集群，就像给你两把钥匙：一把是傻瓜式自动门禁，一把是工程师的万能工具箱。这项技术对普通人意味着什么？未来你用的智能客服、代码助手、甚至个性化家教可能都会“换脑”，它们能处理更复杂的任务，比如帮你规划跨国旅行的全流程——从签证材料到航班延误自动改签。企业也能更低价地部署顶尖AI，因为不用依赖第三方的在线API，数据完全留在自己手里。一句话：AI的“智商天花板”被再次抬高，而且亲民了。

<p>Open weight models have become powerful enough to handle complex tasks such as multi-step agentic workflows, advanced reasoning, and long-horizon coding. However, as these models grow in capability, they also grow in size and hosting multi-trillion parameter architectures requires purpose-built infrastructure, high-end GPU compute, and optimized serving frameworks. On July 27, 2026, Moonshot AI released Kimi K3, a 2.8 trillion parameter Mixture of Experts (MoE) model that represents the <a href="https://platform.kimi.ai/docs/guide/kimi-k3-quickstart" target="_blank" rel="noopener noreferrer">first open-weight system</a> to reach the 3 trillion parameter class. Kimi K3 delivers <a href="https://artificialanalysis.ai/models/kimi-k2" target="_blank" rel="noopener noreferrer">frontier-level intelligence</a> while making its weights publicly available, so that organizations can self-host one of the most capable models in existence on their own infrastructure.</p> <p>This post walks through deploying Kimi K3 on AWS using two approaches: Amazon SageMaker HyperPod, and Amazon Elastic Kubernetes Service (Amazon EKS) cluster.</p> <h1><strong>About Kimi K3</strong></h1> <p>Kimi K3 is built on a differentiated architecture featuring Kimi Delta Attention (KDA), Gated Multi Head Latent Attention (MLA), and a Stable LatentMoE framework. The model distributes its 2.8 trillion parameters across 896 specialist experts, activating only 16 per token. This means approximately 104 billion parameters are active during any single forward pass, yielding a <a href="https://platform.kimi.ai/docs/guide/kimi-k3-quickstart" target="_blank" rel="noopener noreferrer">2.5x improvement</a> in scaling efficiency over its predecessor, Kimi K2.</p> <table class="styled-table" border="1px" cellpadding="10px"> <tbody> <tr> <td style="padding: 10px;border: 1px solid #dddddd"><strong>Attribute</strong></td> <td style="padding: 10px;border: 1px solid #dddddd"><strong>Value</strong></td> </tr> <tr> <td style="padding: 10px;border: 1px solid #dddddd">Total Parameters</td> <td style="padding: 10px;border: 1px solid #dddddd">2.8 Trillion</td> </tr> <tr> <td style="padding: 10px;border: 1px solid #dddddd">Active Parameters per Token</td> <td style="padding: 10px;border: 1px solid #dddddd">104 Billion</td> </tr> <tr> <td style="padding: 10px;border: 1px solid #dddddd">Architecture</td> <td style="padding: 10px;border: 1px solid #dddddd">Mixture of Experts (MoE)</td> </tr> <tr> <td style="padding: 10px;border: 1px solid #dddddd">Expert Count</td> <td style="padding: 10px;border: 1px solid #dddddd">896 (16 activated per token)</td> </tr> <tr> <td style="padding: 10px;border: 1px solid #dddddd">Context Window</td> <td style="padding: 10px;border: 1px solid #dddddd">1 Million Tokens</td> </tr> <tr> <td style="padding: 10px;border: 1px solid #dddddd">Modality</td> <td style="padding: 10px;border: 1px solid #dddddd">Native Multimodal (Text + Vision)</td> </tr> <tr> <td style="padding: 10px;border: 1px solid #dddddd">Release Date</td> <td style="padding: 10px;border: 1px solid #dddddd">July 27, 2026</td> </tr> </tbody> </table> <p>Kimi K3 excels at long-horizon coding tasks, agentic workflows, and complex reasoning. It supports native tool calling, structured output, and an always-on thinking mode for multi-step problem solving.</p> <h1><strong>Model availability and format</strong></h1> <p>The open weights for Kimi K3 are available on Hugging Face under the model identifier moonshotai/Kimi-K3. The weights are distributed in <strong>MXFP4</strong> (Microscaling Floating Point 4-bit) format, which provides an effective balance between model quality and memory efficiency for large-scale inference deployments.</p> <p>Given the model’s architecture and size, serving Kimi K3 requires a vLLM day-0 inference container for Kimi K3. At the time of writing, vllm co