Impressive61 NVIDIA AI Ecosystem Tells Over Four Years = >100% Growth in Developers / Startups / Apps Note: GPU = Graphics Processing Unit. Source: NVIDIA (2021 & 2025) NVIDIA Computing Ecosystem – 2021-2025, per Cloud vs. AI Patterns105 Tech CapEx Spend Partial Instigator = Material Improvements in GPU PerformanceNVIDIA GPU Performance = +225x Over Eight Years 106 1 GPT-MoE Inference Workload = A type of workload Source: NVIDIA (5/25) Performance of NVIDIA GPU Series Over Time – 2016-2024, per NVIDIA Tech CapEx Spend Partial Instigator = Material Improvements in GPU Performance Pascal Volta Ampere Hopper Blackwell

passes Target-optimized graph Target-dependent Relay passes Intel x86 ARM CPU Nvidia GPU ARM GPU Schedule templates written in TVM Tensor IR .. More targets AutoTVM – Tuning the kernels Relay passes Target-optimized Int8 Relay Graph Intel x86 schedule ARM CPU schedule Nvidia GPU schedule ARM GPU schedule Relay Int8 Graph Target-dependent Relay layout opt© 2019, Amazon Web Services Relay passes Target-optimized Int8 Relay Graph Intel x86 schedule ARM CPU schedule Nvidia GPU schedule ARM GPU schedule Relay Int8 Graph Target-dependent Relay layout opt© 2019, Amazon Web Services

人 e 人 e@ TVM Q@ AliOs Overview TVM @ AliOs ARM CPU TVM @ AliOos Hexagon DSP TVM @ Alios Intel GPU Misc /NiiOS ! 驱动万物智能 PART ONE TVM Q@ AliOs Overview AiOS 1驱动万物智能 AliOs overview 。 AliOs (www AN 2X MobilenetV2 TFLite 1.34X MobilenetV2 QNNPACK AliOs @ Roewe RX5 MAX OpenVINO @ Intel GPU AliDS AR-Nav Product @ SUV Release and adopt TVM (Apollo Lake Gold) Vmem( rO++#1) = V31.new 上 r0 = #0; jumpr r31 } PART FOUR Alios TVM @ Intel GPU AiOS 1驱动万物智能 Alios TVM @ Intel GPU 。 Implement the schedule from scratch Subgroups 。 Leverage Intel

Relay Runtime (VM, Graph Runtime, Interpreter) Your Dispatcher Target Device General Devices (CPU/GPU/FPGA) Mark supported operators or subgraphs 1. Implement an operator-level annotator, OR 2. Implement Relay Runtime (VM, Graph Runtime, Interpreter) Your Dispatcher Target Device General Devices (CPU/GPU/FPGA) Mark supported operators or subgraphs 1. Implement extern operator functions, OR 2. Implement Relay Runtime (VM, Graph Runtime, Interpreter) Your Dispatcher Target Device General Devices (CPU/GPU/FPGA) Mark supported operators or subgraphs 1. Implement extern operator functions, OR 2. Implement

(bcm2837) -target=armv7l-linux-gnueabihf -mattr=+neon pynq -target=armv7a-linux-eabi -mattr=+neon GPU mali (midgard) firefly rk3399, rock960 (mali t860) N/A opencl bifrost hikey960 (mali g71) N/A FPGA closely in an organized way ○ Arm - Cortex-A/Cortex-M/Neoverse CPU, Mali GPU, Ethos NPU ○ Qualcomm - Hexagon DSP, Adreno GPU ○ Hisilicon, Xilinx, NXP, TI, ST, Fujitsu, Riken, and etc ● Collaborations

阿里巴巴人工智能实验室 AiILabs & TVM PART 1 : ARM32 CPU CONTENT PART 2 : HIFI4 DSP PART 3 : _ PowervVR GPU [和| Alibaba AL.Labs 阿里巴巴人工智能实验室 ARM 32 CPU Resolution Quantization Orize Kernel ALIOS TVM Alibaba DSP HIFI4 DSP HIFI4 DSP [和| Alibaba AL.Labs 阿里巴巴人工智能实验室 PowerVR GPU Alibaba Al.Labs 阿里巴巴人工智能实验室 PowerVR support by TVM NNVM Compiler -Execution graph -Model layers

strategy func GPU strategy func OpStrategy OpStrategy OpStrategy Default implement Specialized implement 1 Specialized implement 2 (e.g., winograd) kernel_size <= 3 b < 8 “cpu” “gpu”© 2019, Amazon

gain by autotuning 3. TVM can support many kinds of hardware platform: Intel/arm CPU, Nividia/arm GPU, VTA…5 �������������� 1. Get a .log file from the autotvm on Ubuntu 2. Use the .log from step1

level: the less the better 。 The requirement of familiarity with WMMA API “Unified matmul schedule for GPU 。 Maintainability & Common Optimization Sharing 。 Search across the entire space (TensorCore + non-TensorCore)

MPI.jl and Elemental.jl provide access to the existing MPI ecosystem of libraries. 4. GPU computing: The Julia GPU compiler provides the ability to run Julia code natively on GPUs. There is a rich ecosys- array operations distributed across workers, as outlined above. A mention must be made of Julia's GPU programming ecosystem, which includes: 1. CUDA.jl wraps the various CUDA libraries and supports compiling option, often significantly outperforming MKLSparse. 2. CUDA.jl exposes the CUSPARSE library for GPU sparse matrix operations. 3. SparseMatricesCSR.jl provides a Julia native implementation of the Compressed