Behind The Momentum Performance, 16-bit FLOP/s +150% / Year Enabled by 1.6x annual growth in chips per cluster and 1.6x annual growth in performance per chip Performance of Leading AI Supercomputers MM Revenue ($B) Revenue, $B $0 $2 $4 2022 2023 2024Time to 365B Annual Searches = ChatGPT 5.5x Faster vs. Google Note: Dashed-line bars are for years where Google did not disclose annual search Models AI Development Trending = Unprecedented38 AI Developer Growth (NVIDIA Ecosystem as Proxy) = +6x to 6MM Developers Over Seven Years Number of Developers, MM 0 3 6 2005 2007 2009 2011 2013 2015

1 65B LLaMA 2 13B LLaMA 2 34B LLaMA 2 70B LLaMA 3 8B LLaMA 3 70B Mistral 7B Mixtral 8x7B Mixtral 8x22B Command R Command R+ Grok-1 DBRX Qwen1.5 32B Qwen1.5 72B LLaMA 1 Family LLaMA 2 Family 2024) (our previous release), Qwen1.5 72B (Bai et al., 2023), LLaMA3 70B (AI@Meta, 2024), and Mixtral 8x22B (Mistral, 2024). We evaluate all these models with our internal evaluation framework, and ensure performance on 14 Benchmark (Metric) # Shots DeepSeek Qwen1.5 Mixtral LLaMA 3 DeepSeek-V2 67B 72B 8x22B 70B Architecture - Dense Dense MoE Dense MoE # Activated Params - 67B 72B 39B 70B 21B # Total Params

µdeG÷øÕf$2°,67ËþæacCFWghXFPŸ R³Œjk Clm<ÑG]nopmqr>st2022E,FPŸ R<100'u#xÆS)÷ø,vw60+3C,ôK40[+cC%ã,xŸcCyz 7700[+FW{ã,|/5nFW}$~•> • L€Monica•‚,9€Œ"ƒ<„…Muv,ƒ5†D‡[ˆ%GD‡5†IJÞ--‰Š!ƒD‡5†[ˆGfigma> *+¦%G§¨CWOá²'¶> • *˜5GooglecDuckDuckGocSerpercExa> • 5⃣ ()+•©žx5ª« AI *+3z,¬-•®xC•¯°x> • *˜5ArizecLangSmithcLangfusecHelicone> • 6⃣ ()+I±5š›x²'# AI *+Ðd³,KfJK’3)€> • *˜5LangGraphcAutogencHaystackcSwarmcMulti-agent ne«Æ¥]^¬ÇMongoDBc PostgreSQLcWeaviatecNeo4j«Å,/È]^¬ • 1⃣1⃣ ¡¹ÉÊ«Infra/Base¬5AI *+GË3ÌÍ,¬-•ßàxC•®x> • *˜5DockercKubernetes«ÎÞÆCI±¬cAuto Scale VMs|4ßàÏÐû • 1⃣2⃣ ÑÒr35š›áâÑÒ)€Ì() AI ÓÔC#+>12 !"#$%Bloomberg*&'()

the CUDA WUMAA4 4AP1 FP16 or FP32 FP16 or FP32 Background 和 CUDA C half a[16x8] wmma::fragment a[1] a=Afindex] IR Passes *。 Need to satisfy warp tile requirements (16x16x16 .…) | TensorCore Intrinsics "。Kind of Auto Tensorization Wi/7TTa:oaaSto/e 。threadIdx.x -> 0 "threadIdxy 让 | Sr -> threadIdx.y/warpDim.y*warpDim.y badGimy -8 y warpDim.y = 32/warpDim.x = 32/blockDim.x Loop scaling

cuDNN Frameworks New operator introduced by operator fusion optimization potential benefit: 1.5x speedup Engineering intensiveMachine Learning based Program Optimizer TVM: Learning-based Learning 1.44 Large MatMul BatchConv Small MatMul BatchMatMul CuDNN w/ TensorCores tvm w/ TensorCores 1.4x better on emerging workloads Transformer related workloads Credit: Siyuan FengWhere are we goingUnified emerging tensor instructionsTensorization Challenge C = tvm.compute((m, n), 
 lambda y, x: tvm.sum(A[k, y] * B[k, x], axis=k)) Computation Specification (Tensor Expression) A = tvm.placeholder((8, 8))

Graph Target-independent Relay passes Target-optimized graph Target-dependent Relay passes Intel x86 ARM CPU Nvidia GPU ARM GPU Schedule templates written in TVM Tensor IR .. More targets AutoTVM QNN Dialect QNN passes Target-independent Relay passes Target-optimized Int8 Relay Graph Intel x86 schedule ARM CPU schedule Nvidia GPU schedule ARM GPU schedule Relay Int8 Graph Target-dependent QNN Optimization passes • Some optimizations are easier at QNN level • Intel x86 VNNI requires conv input dtypes to uint8 x int8© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved

1.61X MobilenetVl TFlite TV Hexagon DSP | 1.27X MobilenetV1 upport 1.12X Mobilenet V1 TFlite AN 2X MobilenetV2 MobilenetV2 TFLite 1.34X MobilenetV2 QNNPACK AliOs @ Roewe RX5 MAX OpenVINO @ Intel GPU AliDS AR-Nav Product @ SUV Release and adopt TVM (Apollo Lake Gold) Ready OO 2019.8 AiOS 1驱动万物智能 @ 和 Yunqi Conf AR-Nav Product Show Lanenet Model 1.6X Intel AliOs TVM Arch Model 。 Facelandmark Pedestrian & Vehicle Detection Voice-GUI Gesture Lanenet

strategy.register_specialized_implement(wrap_compute_conv2d(topi.x86.conv2d_winograd), topi.x86.conv2d_winograd, strategy.register_default_implement(wrap_compute_conv2d(topi.x86.conv2d_nchw), topi.x86.schedule_conv2d_nchw) elif layout == "NHWC": strategy. topi.x86.schedule_conv2d_nhwc) elif layout == "NCHWc": strategy.register_default_implement(wrap_compute_conv2d(topi.nn.conv2d_nchwc), topi.x86.schedule_conv2d_nchwc)

int16 * int16 erflow-aware int16 = int8 xint8 ent pl 1=int8 int8 * int8 int32 = int16 1 + int16 x int8 Alibaba Al.Labs 阿里巴巴人工智能实验室 CPU : MTK8167S (ARM32 A35 1.5GHz) Model : MobileNetV2_ 1.0_ 224 400 [和| Alibaba AL.Labs 阿里巴巴人工智能实验室 HIFI 4 Alibaba Al.Labs 阿里巴巴人工智能实验室 Resolution 1. GEMM Tensorize (10x speed up) 2. HIFI4 Program (don't need dlopen) Serial Communication HIFI4 DSP HIFI4 和 | (Work group) 名 | | | Apady+m in_channel x+p -一一 人 下| [lm 加| ilw |太 5| | 各 Eee actor Wo

deepseek.com Z u N e P 6 7 K w S v L C q Y 4 Y V 1 T 8 0 u m B k k m O x d k C i y K r j i 6 n p Y d O w t v B 4 G 0 G p y 8 U I q e T 9 M “学习太难？DeepSeek带你‘开挂’逆袭！ 场景1：课堂上突然跟不上了，怎么办 场景：数学课上，老师正在讲解“隐函数求导”，步骤写到第三行时突然跳过了中间推导，直接给出结果：“所 以这里的dy/dx=(-2x-y)/(x+3y²)”。你盯着白板上的公式一脸懵——前两步的链式法则展开去哪了？为什么分 母突然多了3y²？ 周围同学纷纷点头，老师已经翻到下一页讲应用题了。你手心冒汗，想举手提问又怕被说“这 么简单还不会”，不提问又担心后面全听不懂…… 课堂当下（隐蔽求助） p 适用场景：课堂上随时快速跟进 p 操作技巧： Ø 在笔记软件中快速标注困惑点（如：“疑问：第二 步到第三步如何展开？”） Ø 输入精准问题： “隐函数求导例题：从方程x² + xy + y³ = 0推导 dy/dx，请展示完整的链式法则展开步骤，特别是分母 3y²的来源。” Ø 秒速获取步骤解析： 立即对照补全笔记，跟上老师进度。 2. 课间5分钟（深度追问）