Unstructured Sparsity - Lots of 'free' wins from exploring sparsity in modern ML models - Can often prune models to 80%+ sparsity(with retraining) - Massive speedups combined with specialized code-generation

workloads and hardware Hardware FrameworksWhy Automation is the Future Clear winner on emerging models in product Competitive on benchmarking type model Quickly enables other optimizations: fusion Dynamic shape workloads More runtime objects: Arrays, Tuples, Trees, ADTs Minimum runtime for dynamic models Credit: Jared Roesch, Haichen Shen et.aluTVM: TVM on bare-metal Devices Support bare-metal J-TAG

Dynamic Model in TVM AWS AI© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Models with dynamism ● Control flow (if, loop, etc) ● Dynamic shapes ○ Dynamic inputs: batch size, image concatenate within a while loop Limitation of TVM/graph runtime ● Cannot compile and run dynamic models© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Support dynamic model in