Amazon Web Services, Inc. or its Affiliates. All rights reserved. Presenter: Haichen Shen, Yao Wang Amazon SageMaker Neo, Deep Engine Science Dynamic Model in TVM AWS AI© 2019, Amazon Web Services, Inc. loop Limitation of TVM/graph runtime ● Cannot compile and run dynamic models© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Support dynamic model in TVM ● Support Any-dim in dispatch for a (sub-)graph In collaboration with Jared Roesch, Zhi Chen, Wei Chen© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. “Any” in Relay typing Any: represent an unknown

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon/Intel Confidentia Presenter: Zhi Chen, Cody Yu Amazon SageMaker Neo, Deep Engine Science Bring Your Own Codegen to TVM TVM AWS AI© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Considering You... Design and manufacture a deep learning chip which achieves amazing performance on widely-used operators Suppression (NMS) is too new to be supported by your chip But NMS is supported by TVM!© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Let TVM Be the Compiler of Your Chip Your chip can

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Animesh Jain Amazon SageMaker Neo Compilation of Quantized Models in TVM AWS AI© 2019, Amazon Web Services, Inc. or its Affiliates 𝑟𝑒𝑎𝑙_𝑣𝑎𝑙𝑢𝑒 = 𝑠𝑐𝑎𝑙𝑒 ∗ (𝑞𝑢𝑎𝑛𝑡𝑖𝑧𝑒𝑑_𝑣𝑎𝑙𝑢𝑒 − 𝑧𝑒𝑟𝑜_𝑝𝑜𝑖𝑛𝑡)© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Quantization in TVM • Quantization within TVM - pre-quantized graph in TFLite or MxNet • Use high-level wrapper ops of QNN dialect© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. TVM Overview Framework Graph Mxnet TF …. parsers

on internet infrastructure, which allows for wicked-fast adoption of easy-to-use broad-interest services. AI Technology Compounding = Numbers Behind The Momentum22 Knowledge Distribution Evolution = free humanity from menial repetitive work and to dramatically increase the production of goods and services could presage an era of peace and plenty. The capacity to accelerate scientific research could personal assistants, primary care, cancer and drug research, biology, robotics, space, financial services, neighborhood networks – everything. - Amazon CEO Andy Jassy in 2024 Amazon Shareholder Letter

scope and/or Capability Drop (CD) documents for smaller items such as applications (see Figure 13). Services and requirements oversight organizations have the flexibility to identify alternative names for enterprise platforms, IT programs will deliver capabilities via a series of applications or web services. IT systems must therefore be designed, developed, and maintained in concert with enterprise architectures requirements managed and tracked in the product backlog. For example, the contract requirements for a services contract may refer to the expertise required for the development team (e.g., 6 full-time-equivalent

© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Deploying GluonCV models using TVM© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon TVM© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Deploy GluonCV Models https://arxiv.org/pdf/1907.02154.pdf© 2019, Amazon Web Services, Inc. or its Affiliates Trademark Overall Performance AWS DeepLens Acer aiSage NVIDIA Jetson Nano© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Effects of Vision-specific Optimizations

TO START 47 7. Ch. 5 Selecting Which Value Stream to Start With 51 a. GREENFIELD vs BROWNFIELD SERVICES i. DevOps is not just for Greenfield ii. Important Predictor – Is the application architected TECHNICAL DEBT i. Don’t just make the interest payments, pay down the principle ii. If not managed, services become so fragile, delivery grinds to a halt g. INCREASE THE VISIBILITY OF WORK i. Keep the current i. I-shaped – specialist ii. T-shaped – generalists iii. E-shaped h. FUND NOT PROJECTS, BUT SERVICES AND PRODUCTS i. Create stable service teams ii. Enable them to execute strategy and roadmap.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 6.6 Integration with other services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 6.7 Utilities Coroutines and concurrency 135Tornado Documentation, Release 6.5.1 6.6 Integration with other services 6.6.1 tornado.auth — Third-party login with OpenID and OAuth This module contains implementations authen- tication tokens to make requests to the respective services. They all take slightly different arguments due to the fact all these services implement authentication and authorization slightly differently

INITIALLY SELF-MANAGE THEIR PRODUCTION SERVICE i. Google has development groups self-manage their services in production before they become eligible for Ops group to manage. 1. Define launch requirements to help make production-ready services (Hand-Off Readiness Review, HRR) ii. Operations has a service handback mechanism to the development team when production services become fragile 5. Ch. 17 – Integrate

responsible for unsupported platforms ii. Systematically review production infrastructure and services for items that are causing disproportionate amounts of failure and unplanned work; plan for elimination CODE REPOSITORIES AND SHARED SERVICES i. Add mechanisms & tools ii. Add security’s pre-blessed libraries, implementations, etc. iii. Collaborate with any shared services to provide prebuilt, secured