rights reserved. Presenter: Haichen Shen, Yao Wang Amazon SageMaker Neo, Deep Engine Science Dynamic Model in TVM AWS AI© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Models with shapes ○ Dynamic inputs: batch size, image size, sequence length, etc. ○ Output shape of some ops are data dependent: arange, nms, etc. ○ Control flow: concatenate within a while loop Limitation of TVM/graph models© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Support dynamic model in TVM ● Support Any-dim in typing ● Use shape function to compute the type at runtime ● Virtual

Unified Data Layers: A New Era for Scalable Analytics, Search, and AI v 1.1Table of Contents Introduction 1. The Interconnection of Analytics, Search, and AI 2. What is a Real-Time Unified Data Layer Layer? 3. Why Do You Need a Real-Time Unified Data Layer? 4. 5.CrateDB: A Modern Real-Time Unified Data Layer1. Introduction Data teams are facing more challenges than ever. As applications generate and and consume unprecedented volumes of data across a growing number of sources and formats, data engineering and architecture teams must design systems that not only scale but also deliver real-time access

datapoints turned into this beast. As soon as we updated one chart, we often had to update another – a data game of whack-a-mole… a pattern that shows no sign of stopping…and will grow more complex as competition related to the artificial intelligence technology evolution is indeed unprecedented, as supported by the data. This document is filled with user, usage and revenue charts that go up-and-to-the-right… often supported Change Happening Faster Than Ever? Yes, It Is • AI User + Usage + CapEx Growth = Unprecedented • AI Model Compute Costs High / Rising + Inference Costs Per Token Falling = Performance Converging + Developer

They started with three model evals: 01 Language translation Measuring the accuracy and quality of translations produced 
 by a model. 02 Summarization Evaluating how a model condenses information, using resilient to change. 
 Evals are built around tasks that measure 
 the quality of the output of a model against 
 a benchmark—is it more accurate? More compliant? Safer? Your key metrics will depend on employees can focus on 
 the things only people can do. And because AI can process huge amounts of data from many sources, it can create customer experiences that feel more human because they’re more relevant

VGG16 ResNet-50 GoogleNet-V3 Aristotle on 7020 FPGA Iphone8plus Kirin 970 CPU MEM CONTROLLER BUS Data Mover IMG WR SCHEDULER WEIGHTS WR SCHEDULER SMART MEM FABRIC IMG RD SCHEDULER WEIGHTS RD >> 4© Copyright 2018 Xilinx Inference Flow >> 5 MxNet CPU Layers FPGA Layers Runtime Image Model Weights Calibration Set Quantizer Compiler Tensor Graph Optimization Framework Tensor Graph to attrs['output_layout'], attrs['model_name'], outs[0], *ins ), name=name) return out >> 10© Copyright 2018 Xilinx Example of FPGA node in TVM graph { "nodes": [ { "op": "null", "name": "data", "inputs": [] }

small-medium-large as units for assigning story points. Over time, as the teams accumulate performance data, this iterative and incremental4 process improves accuracy in allocating points. Point values are culture often run counter to those in the long-established defense acquisition enterprise. The Agile model represents a change in the way DoD conducts business, and programs must rethink how they are staffed funding models that support an acquisition are structured to support Agile. To succeed, the Agile model depends on strong commitments at all levels of the acquisition process. First, Agile requires dedicated

accelerated NLU model @ 2018.10 OO 2019.4 OO 2019.8 AiOS 1驱动万物智能 @ 和 Yunqi Conf AR-Nav Product Show Lanenet Model 1.6X Intel AliOs TVM Arch Model 。 Facelandmark Upstream Master ) 。， Optimize on INT8 & FP32 AiiOS ! 驱动万物智能 Alios TVM @ ARM CPU INT8 * Cache 芍四 Data FO Data FOData … QNNPACK Convolution 。，NHWC layout Cach， 浆百 FeU Cach- 区下 。， re 。 Tensorize GEMM Cache 大站 Fe Data FO Data … FOData QNNPACK /NiiOS ! 驱动万物智能 P Cache 浆加 Data FO Data FOData … NHWC L2 da … FL2 da Alios TVM @ ARM CPU INT8 TVM /QNNPACK

Quantize Operator fn (%input_data: Tensor[(2, 5), float32]) { qnn.quantize(%input_data, out_dtype="uint8", output_zero_point=127, output_scale=0.5f) } def @main(%input_data: Tensor[(2, 5), float32]) -> -> Tensor[(2, 5), uint8] { %0 = divide(%input_data, 0.5f /* ty=float32 */) /* ty=Tensor[(2, 5), float32] */; %1 = round(%0) /* ty=Tensor[(2, 5), float32] */; %2 = cast(%1, dtype="int32") /* ty=Tensor[(2 conv2d fn (%data: Tensor[(1, 3, 2, 3), uint8], %weight: Tensor[(3, 3, 2, 2), uint8]) { qnn.conv2d(%data, %weight, … , out_dtype="int32", input_zero_point=1, kernel_zero_point=1)} def @main(%data: Tensor[(1

int8 int32 = int16 1 + int16 x int8 Alibaba Al.Labs 阿里巴巴人工智能实验室 CPU : MTK8167S (ARM32 A35 1.5GHz) Model : MobileNetV2_ 1.0_ 224 400 336 350 3丈 300 250 PowerVR GPU Alibaba Al.Labs 阿里巴巴人工智能实验室 PowerVR support by TVM NNVM Compiler -Execution graph -Model layers functions Computation Graph Optimizations -Param TvM Tensor Operators Algorithm &Schedule CUDA TOPI Backends Machine Learning Automated Optimizer Schedule explorer Cost model Mali TOPI ROCM TOPI PVRTOPI Alibaba Al.Labs 阿里巴巴人工智能实验室 PVR TOPI > TOPI for PVR,including what

decisions under uncertainty, and then have the courage to face the consequences. In the plan-driven model, quality was easier to understand.  We specified what the system should do, and then measured quality that, either.  We are constantly making quality decisions, especially in a Continuous Delivery model, as we decide whether the quality of each individual feature is adequate for the feature to be deployed that we have not yet learned to take advantage of, caught up as we are in the contractor-control model of IT. Shadow IT is what happens when the IT organization is unable to meet the needs of a part of