accelerate scientific research could result in cures for disease and solutions for climate change and resource shortages. As Demis Hassabis, CEO of Google DeepMind, has suggested: ‘First we solve AI, then use years80 AI User + Usage + CapEx Growth = UnprecedentedAI Usage – ChatGPT = Rising Rapidly Across Age Groups in USA, per Pew & Elon University Note: 7/23 data per Pew Research study on ChatGPT use, n=10,133 more tokens per task. The appetite for AI isn't slowing down. It’s growing into every available resource – just like software did in the age of desktop and cloud. But infrastructure is not just standing

mechanisms in Table 1. MLA requires only a small amount of KV cache, equal to GQA with only 2.25 groups, but can achieve stronger performance than MHA. 8 Attention Mechanism KV Cache per Token (# Element) denotes the dimension per attention head, ? denotes the number of layers, ?? denotes the number of groups in GQA, and ?? and ?? ℎ denote the KV compression dimension and the per-head dimension of the decoupled DeepSeek-V2, ?? is set to 4?ℎ and ?? ℎ is set to ?ℎ 2 . So, its KV cache is equal to GQA with only 2.25 groups, but its performance is stronger than MHA. 2.2. DeepSeekMoE: Training Strong Models at Economical

PVR TOPI Alibaba ALLabs 阿里巴巴人工智能实验室 Blocking Splits the workload into thread blocks (work groups) and individual threads (work items) Processing Element batch 二

Elaboration（细化）：深入探讨每个子任务的细节 • Connection（连接）：建立子任务之间的逻辑关联 • Temporal Arrangement（时序安排）：考虑任务的时 间维度 • Resource Allocation（资源分配）：为每个子任务分配 适当的注意力资源 • Adaptation（适应）：根据AI反馈动态调整任务结构 为了更有效地进行任务分解，可以采用SPECTRA模型（Systematic

Elaboration（细化）：深入探讨每个子任务的细节 • Connection（连接）：建立子任务之间的逻辑关联 • Temporal Arrangement（时序安排）：考虑任务的时 间维度 • Resource Allocation（资源分配）：为每个子任务分配 适当的注意力资源 • Adaptation（适应）：根据AI反馈动态调整任务结构 为了更有效地进行任务分解，可以采用SPECTRA模型（Systematic