Admin Costs Margins Marketing Spend Effectivity ROIC Revenues Sales Productivity Customer Service Production / Output Revenue-Focused Cost-Focused ‘Traditional’ Enterprise AI Adoption = Rising Erica Virtual Assistant (6/18) Note: We assume a start at zero users from Erica’s launch in 6/18. Pilot users excluded. Source: Bank of America (2/21, 4/24, 2/25) Bank of America Erica Virtual Assistant 10/21 3/22 8/22 1/23 6/23 11/23 4/24 9/24 2/25 Cumulative Client Interactions with Erica Virtual Assistant (MM) Note: Erica is a conversational AI built into Bank of America’s mobile app that helps

platform, introduced a new AI assistant to streamline customer service. Within a few months, the assistant was handling two-thirds of all service chats—doing the work of hundreds of agents and cutting average invested heavily in our API to make it easier to customize and fine-tune models—whether as a self-service approach or using our tools and support. We worked closely with Lowe’s, the Fortune 50 home improvement team Uses it to answer 40,000 questions a year on policies, compliance, and more. The Customer Service team Automates the sentiment analysis of NPS surveys. 16 AI in the EnterpriseAnd the wins continue

model in TVM ● Support Any-dim in typing ● Use shape function to compute the type at runtime ● Virtual machine as a new runtime for Relay ● Dynamic codegen (WIP) ○ Kernel dispatch for a single op ○ Data dependent© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Relay virtual machine Relay Executable relay.vm.compile Relay Object (hardware independent) Code segment VM a data type using the entries from a register. AllocClosure Allocates a closure with a lowered virtual machine function. If Jumps to the true or false offset depending on the condition. Goto Unconditionally

areas: o Core Infrastructure Improvements to TVM o_uTVM: support for microcontrollers in TVM o_ Virtual Machine and dynamic NNs support (w/ AWS folks) o_ Improved NLP support, with focus on transformers o QQ octoML BERT has many reshape operations, which are currently implemented using copy, 10 Virtual Machine e Many improvements from contributors at UW, AWS, and OctoML. e Initial implementation

remote_mod = remote.load_module(“mylib.so") func = remote_mod[“npufunction0"] func(remote_a, remote_b)Virtual Machine: Supporting Dynamic Workload Dynamic shape workloads More runtime objects: Arrays, Tuples

buffer to hide memory load latency 。 storage align to reduce bank conflicts of shared memory 。 Virtual threads for data reuse (on going) Performance on V100 (FP16) 计算平台事业部 COMPUTING PLATFORM 512, 16

sequence of steps that must be executed to meet the user’s goal, whether that's resolving a customer service issue, booking a restaurant reservation, committing a code change, 
 or generating a report. Applications judgment, exceptions, or 
 context-sensitive decisions, for example refund approval 
 in customer service workflows. 02 Difficult-to-maintain rules: Systems that have become unwieldy due to extensive and records, or sending messages. Send emails and texts, update a CRM record, hand-off a customer service ticket to a human. Orchestration Agents themselves can serve as tools for other agents—see the

compared with dense DeepSeek 67B. Inference Efficiency. In order to efficiently deploy DeepSeek-V2 for service, we first convert its parameters into the precision of FP8. In addition, we also perform KV cache DeepSeek-V2 based on the prompt and generation length distribution from the actually deployed DeepSeek 67B service. On a single node with 8 H800 GPUs, DeepSeek-V2 achieves a generation throughput exceeding 50K tokens based on the overall score. Models marked with * represent that we evaluate them through their API service or open-weighted model, instead of referring to the results reported in their original papers. Suffixes

2 _ _ 10 9.86 。， Online Service 8 8 6.952 。 C++

这些先进的 AI 模型， 快速完成从模型到应用的跨越，如 、 等。 ： 大模型聚合平台主要用于整合和管理多个大型机器学习模型，在聚合平台之上，衍生出 MaaS（Model-as-a- Service，大模型即服务）的服务模式——通过提供统一的接口和框架，以更高效地部署、运行和优化这些模型， 。 ：其它开发相关的 LLM 工具，如云原生构建多模态AI应用的工具 Jina，嵌入式数据库 txtai