function to compute the type at runtime ● Virtual machine as a new runtime for Relay ● Dynamic codegen (WIP) ○ Kernel dispatch for a single op ○ Graph dispatch for a (sub-)graph In collaboration with Amazon Web Services, Inc. or its Affiliates. All rights reserved. Dynamic codegen: op dispatch (proposal) ● Goal: support codegen for dynamic shape ● Challenges ○ Single kernel performs poor across different coupled together© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Dynamic codegen: kernel dispatch (proposal) Relay op: conv2d Default function FTVMStrategy A generic function

schedule Se 一人一 了9 。 Normal schedule: the schedule for CUDA 本 codegen IR Passes *。 Need to satisfy TensorCore Intrinsics "。Kind of Auto Tensorization 下 CUDA CodeGen *。IR passes to automatically transform sub-tree to TensorCore Intrinsics Pattern Matching compute_locallo族 了 了 Performance Optimization 计划了全事业部 “Same as non-TensorCore CUDA codegen 。Auto tune tiling sizes 。 Vectorized load/store for higher bandwidth utilization 。Double buffer

MATLAB Coder ○ ONNX RuntimeArm platform support in TVM upstream IPs Target Hardware/Model Options Codegen CPU arm_cpu pixel2 (snapdragon 835), mate10/mate10pro (kirin 970), p20/p20pro (kirin 970) -tar Implement Arm NN generic backend in TVM for more flexibility with the runtime plugins? ○ Integrate TVM codegen into Arm NN? ● CI and benchmark testing for TVM on member hardware platforms ○ Shall we maintain

ExternFunc, …) runtime::Module High-level optimizations (Auto) Schedules
 Low-level optimizations Codegen Import LowerMixed Function Variants in the Same Module def @relay_add_one(%x : Tensor((10,), f32))