2.2.1 Basic Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.2.2 Device-Limited Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.2.3 Auxiliary Loss the affinity scores calculated for the ?-th token and all routed experts. 2.2.2. Device-Limited Routing We design a device-limited routing mechanism to bound MoE-related communication costs. When expert perform top-K selection among experts on these ? devices. In practice, we find that when ? ⩾ 3, the device-limited routing can achieve a good performance roughly aligned with the unrestricted top-K routing

Serialized Subgraph Library Relay Runtime (VM, Graph Runtime, Interpreter) Your Dispatcher Target Device General Devices (CPU/GPU/FPGA) Mark supported operators or subgraphs 1. Implement an operator-level Serialized Subgraph Library Relay Runtime (VM, Graph Runtime, Interpreter) Your Dispatcher Target Device General Devices (CPU/GPU/FPGA) Mark supported operators or subgraphs 1. Implement extern operator Serialized Subgraph Library Relay Runtime (VM, Graph Runtime, Interpreter) Your Dispatcher Target Device General Devices (CPU/GPU/FPGA) Mark supported operators or subgraphs 1. Implement extern operator

Overview *。 Plug directly into TVYM as a backend *，Target C to emit code for microcontrollers that is device- agnostic AuroTYM QQ octoML AutoTVM on HTVM DTYM Runtime send program 较 ,we 人 Interace (Self-Hosted Models) Host Device High-Level 人 ORGREEE Te Conv2D mized RE -一 一 QQ octoML Transformer

Optimization Search Space LLVM, CUDA, Metal VTA Edge FPGA Cloud FPGA ASIC Optimization AutoTVM Device FleetExisting Deep Learning Frameworks High-level data flow graph Hardware Primitive Tensor Credit: Siyuan FengWhere are we goingUnified Runtime For Heterogeneous Devices CUDA Driver NPU Driver Device Drivers External Runtimes NPUModule CUDAModule TFModule tvm::runtime::Module GetFunction(string)

so to support parallel. 。 Could run end-to-end TFLite Mobilenet V2 quantized model on Simulator / Device. /NiiOS ! 驱动万物智能 Alios TVM @ Hexagon DSP 。， Performance is our focus next. We tvm.caLL_pure_intrin

Tencent, & Baidu for ten years ending 2022. ‘Tens of billions of units’ refers to the potential device & user base that could end up using AI technology; this includes smartphones, IOT devices, robotics