rights reserved. Presenter: Haichen Shen, Yao Wang Amazon SageMaker Neo, Deep Engine Science Dynamic Model in TVM AWS AI© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Models with models© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Support dynamic model in TVM ● Support Any-dim in typing ● Use shape function to compute the type at runtime ● Virtual input_name = "data" input_shape = [tvm.relay.Any(), 3, 224, 224] dtype = "float32" block = get_model('resnet50_v1', pretrained=True) mod, params = relay.frontend.from_mxnet(block, shape={input_name:

Change Happening Faster Than Ever? Yes, It Is • AI User + Usage + CapEx Growth = Unprecedented • AI Model Compute Costs High / Rising + Inference Costs Per Token Falling = Performance Converging + Developer 2/24 2/25 4/25 75% 60% 10% 21% 15% 0% Details on Page 293 USA – LLM #1 China USA – LLM #2 AI Model Compute Costs High / Rising + Inference Costs Per Token Falling = Performance Converging + Developer Change Happening Faster Than Ever? Yes, It Is • AI User + Usage + CapEx Growth = Unprecedented • AI Model Compute Costs High / Rising + Inference Costs Per Token Falling = Performance Converging + Developer

They started with three model evals: 01 Language translation Measuring the accuracy and quality of translations produced 
 by a model. 02 Summarization Evaluating how a model condenses information, using resilient to change. 
 Evals are built around tasks that measure 
 the quality of the output of a model against 
 a benchmark—is it more accurate? More compliant? Safer? Your key metrics will depend on more tokens. To increase efficiency, OpenAI and Indeed 
 worked together to fine-tune a smaller GPT model that was able to deliver similar results 
 with 60% fewer tokens. Helping job seekers find the

>> 4© Copyright 2018 Xilinx Inference Flow >> 5 MxNet CPU Layers FPGA Layers Runtime Image Model Weights Calibration Set Quantizer Compiler Tensor Graph Optimization Framework Tensor Graph to ins, outs: tvm.call_packed('tvm.accel.accel_fused', attrs['path'], attrs['output_layout'], attrs['model_name'], outs[0], *ins ), name=name) return out >> 10© Copyright 2018 Xilinx Example of FPGA node Xilinx Performance Pipelines ˃ References to our latest results: https://github.com/Xilinx/AI-Model-Zoo (embedded i.e. ZC104/Ultra96) https://github.com/Xilinx/ml-suite/blob/master/examples/caffe/Benchmark_README

methods not delivering generalized performance 2 Why TVM? XTVM for Speech Synthesis - WaveRNN-style model architecture - Autoregressive sampling net running at faster than real-time - Compute split between - First PyTorch model used a 3,400us sampling net runtime Image from LPCNetExit, Pursued By A Bear - 3400us (baseline), 40us (target) - 85x speedup - Uh ohEnter, TVM and model co-design - PyTorch WaveRNN, Sparse Transformers, etc - Reduce precision with int8/float16 - very helpful to maintain model in core-private L1 dcaches - Use rational approximations for transcendentals (exp, tanh, erf, etc)

culture often run counter to those in the long-established defense acquisition enterprise. The Agile model represents a change in the way DoD conducts business, and programs must rethink how they are staffed funding models that support an acquisition are structured to support Agile. To succeed, the Agile model depends on strong commitments at all levels of the acquisition process. First, Agile requires dedicated team in addition to the development contactors. Close, dedicated acquisition teams facilitate this model, but it must be further reinforced at the top. Leadership can signal that trust by empowering team

int8 int32 = int16 1 + int16 x int8 Alibaba Al.Labs 阿里巴巴人工智能实验室 CPU : MTK8167S (ARM32 A35 1.5GHz) Model : MobileNetV2_ 1.0_ 224 400 336 350 3丈 300 250 PowerVR GPU Alibaba Al.Labs 阿里巴巴人工智能实验室 PowerVR support by TVM NNVM Compiler -Execution graph -Model layers functions Computation Graph Optimizations -Param TvM Tensor Operators Algorithm &Schedule CUDA TOPI Backends Machine Learning Automated Optimizer Schedule explorer Cost model Mali TOPI ROCM TOPI PVRTOPI Alibaba Al.Labs 阿里巴巴人工智能实验室 PVR TOPI > TOPI for PVR,including what

accelerated NLU model @ 2018.10 OO 2019.4 OO 2019.8 AiOS 1驱动万物智能 @ 和 Yunqi Conf AR-Nav Product Show Lanenet Model 1.6X Intel AliOs TVM Arch Model 。 Facelandmark libtvm_hexagon_runtime.so to support parallel. 。 Could run end-to-end TFLite Mobilenet V2 quantized model on Simulator / Device. /NiiOS ! 驱动万物智能 Alios TVM @ Hexagon DSP 。， Performance is our focus next

appears to offer predictability, control, and efficiency, the key values of the contractor-control model. But it doesn’t. Requirements: Requirements are a way of controlling the development team by constraining Characteristics of an Agile governance and oversight model: Before we dive into an Agile governance and oversight model, let’s think about what characteristics such a model should have in order to both take advantage

decisions under uncertainty, and then have the courage to face the consequences. In the plan-driven model, quality was easier to understand.  We specified what the system should do, and then measured quality that, either.  We are constantly making quality decisions, especially in a Continuous Delivery model, as we decide whether the quality of each individual feature is adequate for the feature to be deployed that we have not yet learned to take advantage of, caught up as we are in the contractor-control model of IT. Shadow IT is what happens when the IT organization is unable to meet the needs of a part of