[和| Alibaba AL.Labs 阿里巴巴人工智能实验室 AiILabs & TVM PART 1 : ARM32 CPU CONTENT PART 2 : HIFI4 DSP PART 3 : _ PowervVR GPU [和| Alibaba AL.Labs 阿里巴巴人工智能实验室 ARM 32 CPU Resolution Quantization Orize Orize Kernel ALIOS TVM Alibaba Al.Labs 阿里巴巴人工智能实验室 7= 590一I) rm 一 下Er (mm) =肪+2mxaM0 [5 (全-2)+o| current plan 1 = int16 * int16 erflow-aware int16 = int8 xint8 ent pl 1=int8 int8 * int8 int32 int32 = int16 1 + int16 x int8 Alibaba Al.Labs 阿里巴巴人工智能实验室 CPU : MTK8167S (ARM32 A35 1.5GHz) Model : MobileNetV2_ 1.0_ 224 400 336 350 3丈 300 250

10/22 4/25 800MM Big Six* USA Technology Company CapEx *Apple, NVIDIA, Microsoft, Alphabet, Amazon (AWS only), & Meta Platforms Source: Capital IQ (3/25), Morgan Stanley 2014 2024 CapEx, $B +63% the installed based of smartphones & tablets in 2020. Cloud & data center capex includes Google, Amazon, Microsoft, Meta, Alibaba, Apple, IBM, Oracle, Tencent, & Baidu for ten years ending 2022. ‘Tens the Future of Music’ (5/2/25); Spotify earnings releases; eMarketer, ‘Spotify dominates Apple and Amazon in digital audio’ (4/25) AI-Powered Audio Translation – 5/25, per Spotify Imagine if you’re a creator

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon/Intel Confidentia Presenter: Zhi Chen, Cody Yu Amazon SageMaker Neo, Deep Engine Science Bring Your Own Codegen to TVM TVM AWS AI© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Considering You... Design and manufacture a deep learning chip which achieves amazing performance on widely-used operators Maximum Suppression (NMS) is too new to be supported by your chip But NMS is supported by TVM!© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Let TVM Be the Compiler of Your Chip

available throughout the organization.” They are self-healing ii. Netflix – April 21, 2011; entire Amazon AWS US-EAST availability zone went down; Netflix was unaffected. Since 2008 they were building and defining failure modes and then testing to ensure they operate as expected iv. Great Amazon Reboot of 2014 – 10% of Amazon EC2 servers had to reboot for Xen emergency security patch. At Netflix, zero downtime ii. Examine dependencies for known vulnerabilities and consolidate multiple versions of the same library iii. 2014 Verizon PCI Data Breach Investigation Report – studies over 85K cardholder breaches.

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Deploying GluonCV models using TVM© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Amazon Trademark Deploy GluonCV Models GluonCV Models MXNet Computational Graph Json Acyclic Graph Export As-is Optimize with TVM© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved reserved. Amazon Trademark Deploy GluonCV Models https://arxiv.org/pdf/1907.02154.pdf© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Overall Performance AWS DeepLens

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Presenter: Haichen Shen, Yao Wang Amazon SageMaker Neo, Deep Engine Science Dynamic Model in TVM AWS AI© 2019, Amazon Web Services within a while loop Limitation of TVM/graph runtime ● Cannot compile and run dynamic models© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Support dynamic model in TVM ● Support ○ Graph dispatch for a (sub-)graph In collaboration with Jared Roesch, Zhi Chen, Wei Chen© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. “Any” in Relay typing Any: represent

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Animesh Jain Amazon SageMaker Neo Compilation of Quantized Models in TVM AWS AI© 2019, Amazon Web Services, Inc. or its Affiliates 𝑟𝑒𝑎𝑙_𝑣𝑎𝑙𝑢𝑒 = 𝑠𝑐𝑎𝑙𝑒 ∗ (𝑞𝑢𝑎𝑛𝑡𝑖𝑧𝑒𝑑_𝑣𝑎𝑙𝑢𝑒 − 𝑧𝑒𝑟𝑜_𝑝𝑜𝑖𝑛𝑡)© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Quantization in TVM • Quantization within ingests a pre-quantized graph in TFLite or MxNet • Use high-level wrapper ops of QNN dialect© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. TVM Overview Framework Graph Mxnet

an Enterprise Strategist at Amazon Web Services and the author of The Art of Business Value and A Seat at the Table: IT Leadership in the Age of Agility. Before joining AWS he was the CIO of US Citizenship Computer Science from Yale, and an MA in Philosophy from Yale. Mark and Mike first met at the 2019 AWS Public Sector Summit in Washington, DC. They’ve been best friends ever since.

& found | Grab coffee/beer break 3. Archiving Ultimate DevEx with DevPod – Przemek Białoń (Loft Labs) 4. After: nearbyOur HostSponsorsSponsors CommunityWhere to find us? Slack: #poland / #poland-warsaw