Serialized Subgraph Library Relay Runtime (VM, Graph Runtime, Interpreter) Your Dispatcher Target Device General Devices (CPU/GPU/FPGA) Mark supported operators or subgraphs 1. Implement an operator-level Serialized Subgraph Library Relay Runtime (VM, Graph Runtime, Interpreter) Your Dispatcher Target Device General Devices (CPU/GPU/FPGA) Mark supported operators or subgraphs 1. Implement extern operator Serialized Subgraph Library Relay Runtime (VM, Graph Runtime, Interpreter) Your Dispatcher Target Device General Devices (CPU/GPU/FPGA) Mark supported operators or subgraphs 1. Implement extern operator

Overview *。 Plug directly into TVYM as a backend *，Target C to emit code for microcontrollers that is device- agnostic AuroTYM QQ octoML AutoTVM on HTVM DTYM Runtime send program 较 ,we 人 Interace (Self-Hosted Models) Host Device High-Level 人 ORGREEE Te Conv2D mized RE -一 一 QQ octoML Transformer

Optimization Search Space LLVM, CUDA, Metal VTA Edge FPGA Cloud FPGA ASIC Optimization AutoTVM Device FleetExisting Deep Learning Frameworks High-level data flow graph Hardware Primitive Tensor Credit: Siyuan FengWhere are we goingUnified Runtime For Heterogeneous Devices CUDA Driver NPU Driver Device Drivers External Runtimes NPUModule CUDAModule TFModule tvm::runtime::Module GetFunction(string)

so to support parallel. 。 Could run end-to-end TFLite Mobilenet V2 quantized model on Simulator / Device. /NiiOS ! 驱动万物智能 Alios TVM @ Hexagon DSP 。， Performance is our focus next. We tvm.caLL_pure_intrin

rand(Float32,1000) * 1.f-9 Avoid string interpolation for I/O When writing data to a file (or other I/O device), forming extra intermediate strings is a source of overhead. Instead of: println(file, "$a $b") Vector{Tuple{UInt64, Symbol, Type}}: (0x0000000000000000, :desc, Union{RawFD, String}) (0x0000000000000008, :device, UInt64) (0x0000000000000010, :inode, UInt64) (0x0000000000000018, :mode, UInt64) (0x0000000000000020 OS_HANDLE} The path or OS file descriptor size Int64 The size (in bytes) of the file device UInt ID of the device that contains the file inode UInt The inode number of the file mode UInt The protection

rand(Float32,1000) * 1.f-9 Avoid string interpolation for I/O When writing data to a file (or other I/O device), forming extra intermediate strings is a source of overhead. Instead of: println(file, "$a $b") Vector{Tuple{UInt64, Symbol, Type}}: (0x0000000000000000, :desc, Union{RawFD, String}) (0x0000000000000008, :device, UInt64) (0x0000000000000010, :inode, UInt64) (0x0000000000000018, :mode, UInt64) (0x0000000000000020 OS_HANDLE} The path or OS file descriptor size Int64 The size (in bytes) of the file device UInt ID of the device that contains the file inode UInt The inode number of the file mode UInt The protection

rand(Float32,1000) * 1.f-9 Avoid string interpolation for I/O When writing data to a file (or other I/O device), forming extra intermediate strings is a source of overhead. Instead of: println(file, "$a $b") Vector{Tuple{UInt64, Symbol, Type}}: (0x0000000000000000, :desc, Union{RawFD, String}) (0x0000000000000008, :device, UInt64) (0x0000000000000010, :inode, UInt64) (0x0000000000000018, :mode, UInt64) (0x0000000000000020 OS_HANDLE} The path or OS file descriptor size Int64 The size (in bytes) of the file device UInt ID of the device that contains the file inode UInt The inode number of the file mode UInt The protection

rand(Float32,1000) * 1.f-9 Avoid string interpolation for I/O When writing data to a file (or other I/O device), forming extra intermediate strings is a source of overhead. Instead of: println(file, "$a $b") Vector{Tuple{UInt64, Symbol, Type}}: (0x0000000000000000, :desc, Union{RawFD, String}) (0x0000000000000008, :device, UInt64) (0x0000000000000010, :inode, UInt64) (0x0000000000000018, :mode, UInt64) (0x0000000000000020 OS_HANDLE} The path or OS file descriptor size Int64 The size (in bytes) of the file device UInt ID of the device that contains the file inode UInt The inode number of the file mode UInt The protection

rand(Float32,1000) * 1.f-9 Avoid string interpolation for I/O When writing data to a file (or other I/O device), forming extra intermediate strings is a source of overhead. Instead of: println(file, "$a $b") Vector{Tuple{UInt64, Symbol, Type}}: (0x0000000000000000, :desc, Union{RawFD, String}) (0x0000000000000008, :device, UInt64) (0x0000000000000010, :inode, UInt64) (0x0000000000000018, :mode, UInt64) (0x0000000000000020 OS_HANDLE} The path or OS file descriptor size Int64 The size (in bytes) of the file device UInt ID of the device that contains the file inode UInt The inode number of the file mode UInt The protection

3v[2]-w[2]). 34.19 Avoid string interpolation for I/O When writing data to a file (or other I/O device), forming extra intermediate strings is a source of overhead. Instead of: println(file, "$a $b") Vector{Tuple{UInt64, Symbol, Type}}: (0x0000000000000000, :desc, Union{RawFD, String}) (0x0000000000000008, :device, UInt64) (0x0000000000000010, :inode, UInt64) (0x0000000000000018, :mode, UInt64) (0x0000000000000020 Description desc The path or OS file descriptor size The size (in bytes) of the file device ID of the device that contains the file inode The inode number of the file mode The protection mode of the