model in TVM ● Support Any-dim in typing ● Use shape function to compute the type at runtime ● Virtual machine as a new runtime for Relay ● Dynamic codegen (WIP) ○ Kernel dispatch for a single op ○ Data dependent© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Relay virtual machine Relay Executable relay.vm.compile Relay Object (hardware independent) Code segment VM a data type using the entries from a register. AllocClosure Allocates a closure with a lowered virtual machine function. If Jumps to the true or false offset depending on the condition. Goto Unconditionally

areas: o Core Infrastructure Improvements to TVM o_uTVM: support for microcontrollers in TVM o_ Virtual Machine and dynamic NNs support (w/ AWS folks) o_ Improved NLP support, with focus on transformers o QQ octoML BERT has many reshape operations, which are currently implemented using copy, 10 Virtual Machine e Many improvements from contributors at UW, AWS, and OctoML. e Initial implementation

Erica Virtual Assistant (6/18) Note: We assume a start at zero users from Erica’s launch in 6/18. Pilot users excluded. Source: Bank of America (2/21, 4/24, 2/25) Bank of America Erica Virtual Assistant 10/21 3/22 8/22 1/23 6/23 11/23 4/24 9/24 2/25 Cumulative Client Interactions with Erica Virtual Assistant (MM) Note: Erica is a conversational AI built into Bank of America’s mobile app that helps

remote_mod = remote.load_module(“mylib.so") func = remote_mod[“npufunction0"] func(remote_a, remote_b)Virtual Machine: Supporting Dynamic Workload Dynamic shape workloads More runtime objects: Arrays, Tuples

buffer to hide memory load latency 。 storage align to reduce bank conflicts of shared memory 。 Virtual threads for data reuse (on going) Performance on V100 (FP16) 计算平台事业部 COMPUTING PLATFORM 512, 16

Product Note: Operator Operator is an example of OpenAI’s agentic approach. Leveraging its own virtual browser, Operator can navigate the web, click on buttons, fill in forms, and gather data just like