Partitioning Your Codegen LLVM, CUDA, Metal, VTA Serialized Subgraph Library Relay Runtime (VM, Graph Runtime, Interpreter) Your Dispatcher Target Device General Devices (CPU/GPU/FPGA) Mark supported Partitioning Your Codegen LLVM, CUDA, Metal, VTA Serialized Subgraph Library Relay Runtime (VM, Graph Runtime, Interpreter) Your Dispatcher Target Device General Devices (CPU/GPU/FPGA) Mark supported Partitioning Your Codegen LLVM, CUDA, Metal, VTA Serialized Subgraph Library Relay Runtime (VM, Graph Runtime, Interpreter) Your Dispatcher Target Device General Devices (CPU/GPU/FPGA) Mark supported

goingUnified Runtime For Heterogeneous Devices CUDA Driver NPU Driver Device Drivers External Runtimes NPUModule CUDAModule TFModule tvm::runtime::Module GetFunction(string) -> tvm::runtime::PackedFunc nc SaveToBinary/LoadFromBinary Runtime Module Interface SubclassesUnified Runtime Benefit mod.export_library("mylib.so") Unified library packaging Free API (Py/Java/Go) lib = tvm.module.load("mylib remote_b)Virtual Machine: Supporting Dynamic Workload Dynamic shape workloads More runtime objects: Arrays, Tuples, Trees, ADTs Minimum runtime for dynamic models Credit: Jared Roesch, Haichen Shen et.aluTVM: TVM

Documentation 1806 105 Documentation of Julia's Internals 1807 105.1 Initialization of the Julia runtime . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1807 105.2 Julia ASTs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1869 105.17 printf() and stdio in the Julia runtime . . . . . . . . . . . . . . . . . . . . . . . . . . 1877 105.18 Bounds checking . . . . . . . . changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1986 108.3 Compiler/Runtime improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . 1987 108.4 Command-line option

Documentation 1806 105 Documentation of Julia's Internals 1807 105.1 Initialization of the Julia runtime . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1807 105.2 Julia ASTs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1869 105.17 printf() and stdio in the Julia runtime . . . . . . . . . . . . . . . . . . . . . . . . . . 1877 105.18 Bounds checking . . . . . . . . changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1986 108.3 Compiler/Runtime improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . 1987 108.4 Command-line option

Documentation 1806 105 Documentation of Julia's Internals 1807 105.1 Initialization of the Julia runtime . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1807 105.2 Julia ASTs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1869 105.17 printf() and stdio in the Julia runtime . . . . . . . . . . . . . . . . . . . . . . . . . . 1877 105.18 Bounds checking . . . . . . . . changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1986 108.3 Compiler/Runtime improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . 1987 108.4 Command-line option

Documentation 1845 105 Documentation of Julia's Internals 1846 105.1 Initialization of the Julia runtime . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1846 105.2 Julia ASTs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1911 105.17 printf() and stdio in the Julia runtime . . . . . . . . . . . . . . . . . . . . . . . . . . 1919 105.18 Bounds checking . . . . . . . . changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2029 108.3 Compiler/Runtime improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . 2029 108.4 Command-line option

Documentation 1855 106 Documentation of Julia's Internals 1856 106.1 Initialization of the Julia runtime . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1856 106.2 Julia ASTs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1920 106.17 printf() and stdio in the Julia runtime . . . . . . . . . . . . . . . . . . . . . . . . . . 1928 106.18 Bounds checking . . . . . . . . changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2038 109.3 Compiler/Runtime improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . 2038 109.4 Command-line option

Documentation 1854 106 Documentation of Julia's Internals 1855 106.1 Initialization of the Julia runtime . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1855 106.2 Julia ASTs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1920 106.17 printf() and stdio in the Julia runtime . . . . . . . . . . . . . . . . . . . . . . . . . . 1928 106.18 Bounds checking . . . . . . . . changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2038 109.3 Compiler/Runtime improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . 2038 109.4 Command-line option

Documentation 1854 106 Documentation of Julia's Internals 1855 106.1 Initialization of the Julia runtime . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1855 106.2 Julia ASTs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1920 106.17 printf() and stdio in the Julia runtime . . . . . . . . . . . . . . . . . . . . . . . . . . 1928 106.18 Bounds checking . . . . . . . . changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2038 109.3 Compiler/Runtime improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . 2038 109.4 Command-line option

Documentation 1857 106 Documentation of Julia's Internals 1858 106.1 Initialization of the Julia runtime . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1858 106.2 Julia ASTs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1922 106.17 printf() and stdio in the Julia runtime . . . . . . . . . . . . . . . . . . . . . . . . . . 1931 106.18 Bounds checking . . . . . . . . changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2041 109.3 Compiler/Runtime improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . 2041 109.4 Command-line option