np from tvm import relay 2. Load a pretrained network mod, params = relay.testing.mobilenet.get_workload(batch_size=1) 3. Partition and build the network with an external codegen mod = relay.build_extern(mod ib//graph_annotator.py ● Apply the annotator to a workload: mod, params = relay.testing.mobilenet.get_workload(batch_size=1) mod[‘main’] = MyAnnotator().visit(mod[‘main’]) mod = relay

让 1 splits the workload into thread <| | Apaday+my blocks Bly zx) https://docstvm ai/ PVR TOPI Alibaba ALLabs 阿里巴巴人工智能实验室 Blocking Splits the workload into thread blocks (work groups) and individual threads (work items) Processing Element

45 .31让工 1.31 -35 1 129 中131 124有23152136 2 1.14 am omo oo Convolution Workload Performance AiOS 1驱动万物智能 Alios TVM @ ARM CPU INT8 Depthwise Convolution 。， NHWC layout 。 Using 33. 1.15 116 111 09工08 工区 0.77 0.77 | | | Depthwise Convolution Workload Performance Alios TVM @ ARM CPU INT8 Performance Comparison @ rasp 3b+ AARCH64 aoo0 8.87 sm ao

PerformanceNVIDIA GPU Performance = +225x Over Eight Years 106 1 GPT-MoE Inference Workload = A type of workload where a GPT-style model with a Mixture-of-Experts (MoE) architecture is used for inference over eight years while requiring 4x fewer GPUs… $1B Data Center Comparison GPT-MoE Inference Workload1 …Inference token capacity +27,500x over eight years, implying +30,000x higher theoretical

func = remote_mod[“npufunction0"] func(remote_a, remote_b)Virtual Machine: Supporting Dynamic Workload Dynamic shape workloads More runtime objects: Arrays, Tuples, Trees, ADTs Minimum runtime for