Amazon Trademark Deploying GluonCV models using TVM© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Deploy GluonCV Models GluonCV Models MXNet Computational Graph 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Deploy GluonCV Models https://arxiv.org/pdf/1907.02154.pdf© 2019, Amazon Web Services, Inc. or its Affiliates. All Nano© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Like GluonCV? Go build! https://gluon-cv.mxnet.io https://github.com/dmlc/gluon-cv© 2019, Amazon Web Services

digital datasets that have been in the making for over three decades; breakthrough large language models (LLMs) that – in effect – found freedom with the November 2022 launch of OpenAI’s ChatGPT with computers are ingesting massive datasets to get smarter and more competitive. Breakthroughs in large models, cost-per-token declines, open-source proliferation and chip performance improvements are making infrastructure: agentic interfaces, enterprise copilots, real-world autonomous systems, and sovereign models. Rapid advances in artificial intelligence, compute infrastructure, and global connectivity are

evals 6 Embed AI into your products 9 Start now and invest early 11 Customize and fine-tune your models 13 Get AI in the hands of experts 16 Unblock your developers 18 Set bold automation goals 21 research and deployment company, OpenAI prioritizes partnering with global companies because our models will increasingly do their best work with sophisticated, complex, interconnected workflows and systems teams. Our Research Team advances the foundations of AI, developing new models and capabilities. Our Applied Team turns those models into products, like ChatGPT Enterprise and our API. And our Deployment

its Affiliates. All rights reserved. Animesh Jain Amazon SageMaker Neo Compilation of Quantized Models in TVM AWS AI© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Quantization dataset • Finds suitable quantization scale • Produces a quantized graph • Compiling Pre-quantized models – QNN Dialect • TVM ingests a pre-quantized graph in TFLite or MxNet • Use high-level wrapper ops Frontend Parsers • TFLite Pre-quantized Models • In good shape • Supports all Image Classification PreQuantized hosted models • MXNet Pre-quantized Models • Tested internally with MxNet + MKLDNN

remumn dming data AutoTYM 二 QQ octoML Coming Soon to HTVM (Self-Hosted Models) Host Device mized RE -一 一 QQ octoML Transformer Improvements Transformer based models such as BERT have recently become very Popular and require first class support in TVML. ee What relay ONNX frontend to support all opset versions of BERT. 里This enables importing of native ONNX models and those converted from Tensorflow. 5 ， Improve scheduling of batch matrix multiplies. 时”Early autotuning

Unstructured Sparsity - Lots of 'free' wins from exploring sparsity in modern ML models - Can often prune models to 80%+ sparsity(with retraining) - Massive speedups combined with specialized code-generation

workloads and hardware Hardware FrameworksWhy Automation is the Future Clear winner on emerging models in product Competitive on benchmarking type model Quickly enables other optimizations: fusion Dynamic shape workloads More runtime objects: Arrays, Tuples, Trees, ADTs Minimum runtime for dynamic models Credit: Jared Roesch, Haichen Shen et.aluTVM: TVM on bare-metal Devices Support bare-metal J-TAG

Xilinx Elliott Delaye FPGA CNN Accelerator and TVM© Copyright 2018 Xilinx TVM Target devices and models >> 2 HW Platforms ZCU102 ZCU104 Ultra96 PYNQ Face detection Pose estimation Video Video analytics Lane detection Object detection Segmentation Models© Copyright 2018 Xilinx Xilinx Cloud DPU Processor (xDNNv3) >> 3 ˃ Configurable Overlay Processor

Dynamic Model in TVM AWS AI© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Models with dynamism ● Control flow (if, loop, etc) ● Dynamic shapes ○ Dynamic inputs: batch size, image concatenate within a while loop Limitation of TVM/graph runtime ● Cannot compile and run dynamic models© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Support dynamic model in

this lack of well-documented research and of historical examples that other programs could use as models, we sought the views of experts representing diverse acquisition disciplines on how to appropriately organized, and managed, as well as whether the business processes, governance reviews, and funding models that support an acquisition are structured to support Agile. To succeed, the Agile model depends leverage to meet the DoDI 5000.02 requirements. Figure 8 provides a potential framework that adapts Models 2, 3, and Hybrid B from the Interim DoDI 5000.02 for an Agile program acquisition. Figure 8: