scratch • New Relay passes and TVM schedules required • AlterOpLayout, Graph Fusion etc require work/operator • No reuse of existing Relay and TVM infrastructure. Option 2 – Lower to a sequence of existing 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Lowering of QNN Quantize Operator fn (%input_data: Tensor[(2, 5), float32]) { qnn.quantize(%input_data, out_dtype="uint8", output_zero_point=127 Affiliates. All rights reserved. QNN Conv2D Operator • Calculations are different from FP32 Conv2D https://discuss.tvm.ai/t/tf-lite-quantized-conv2d-operator-conversion/2651/8 𝑟𝑒𝑎𝑙_𝑣𝑎𝑙𝑢𝑒 = 𝒔𝒄𝒂𝒍𝒆

Models with dynamism ● Control flow (if, loop, etc) ● Dynamic shapes ○ Dynamic inputs: batch size, image size, sequence length, etc. ○ Output shape of some ops are data dependent: arange, nms, etc. ○ Control function ● Relax type inference/checking for Any at compilation time ● Register a shape function for operator to check the type and compute the output shape© 2019, Amazon Web Services, Inc. or its Affiliates function ● Relax type inference/checking for Any at compilation time ● Register a shape function for operator to check the type and compute the output shape ● Shape function has two modes (op_attrs, input_tensors

sampling net runtime Image from LPCNetExit, Pursued By A Bear - 3400us (baseline), 40us (target) - 85x speedup - Uh ohEnter, TVM and model co-design - PyTorch operator overhead makes interpreter general technique, allows clean vectorization - Related work in Gibiansky (2017), Gray (2019), et al. Image from OpenAI- Add relay.nn.sparse_dense for block-sparse matrix multiplication (~50 lines of TVM IR)

carried out in 3/25 using GPT-4.5. During the test, participants incorrectly identified the left image (Witness A) as human with 87% certainty, saying ‘A had human vibes. B had human imitation vibes B was human. AI Development Trending = Unprecedented44 AI Performance = Increasingly Realistic Image Generation… Notes: Dates shown are the release dates of each Midjourney model. Source: Midjourney Penguin, ‘How Midjourney Evolved Over Time (Comparing V1 to V6.1 Outputs)’ (9/24) AI-Generated Image: ‘Women’s Necklace with a Sunflower Pendant’ – 2/22-4/25, per Midjourney / Gold Penguin Model v1

language model input and output, a text prompt (sometimes accompanied by other modalities such as image prompts) is the input the model uses to predict a specific output. You don’t need to be a data scientist `f` string syntax for string interpolation is more readable and concise than the traditional `+` operator. 4. The code doesn’t handle errors that might occur during the renaming process. It would be better

subgraphs 1. Implement an operator-level annotator, OR 2. Implement a graph-level annotator© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Option 1: Operator-Level Annotation ● Implement Boolean functions in the template def conv2d(attrs, args): return is_float32(args) Relay operator name Operator attributes and args (inputs) can be checked as well Return True/False for this op After Device General Devices (CPU/GPU/FPGA) Mark supported operators or subgraphs 1. Implement extern operator functions, OR 2. Implement a graph annotator© 2019, Amazon Web Services, Inc. or its Affiliates

in the same thread block (work group) cooperatively fetch dependent data https/www khronos.org/registry/DpenCLspecs/opencl-1.2.pdf Alibaba A.|.Lab: 多国生ER Thanks

cuDNN Offload to heavily optimized DNN operator library FrameworksLimitations of Existing Approach cuDNN Frameworks New operator introduced by operator fusion optimization potential benefit: System High-level data flow graph and optimizations Directly generate optimized program for new operator workloads and hardware Hardware FrameworksWhy Automation is the Future Clear winner on

help guide your own thinking. Product Note: Operator Operator is an example of OpenAI’s agentic approach. Leveraging its own virtual browser, Operator can navigate the web, click on buttons, fill that previously required human intervention, such as: Automating software testing and QA using Operator to interact with web apps 
 like a real user, flagging any UI issues. Updating systems of record

Although training an MoE model will introduce additional commu- nication overheads, through our operator and communication optimizations, the training for DeepSeek-V2 can attain a relatively high Model