rights reserved. Presenter: Haichen Shen, Yao Wang Amazon SageMaker Neo, Deep Engine Science Dynamic Model in TVM AWS AI© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Models with models© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Support dynamic model in TVM ● Support Any-dim in typing ● Use shape function to compute the type at runtime ● Virtual or its Affiliates. All rights reserved. API Example input_name = "data" input_shape = [tvm.relay.Any(), 3, 224, 224] dtype = "float32" block = get_model('resnet50_v1', pretrained=True) mod, params

Change Happening Faster Than Ever? Yes, It Is • AI User + Usage + CapEx Growth = Unprecedented • AI Model Compute Costs High / Rising + Inference Costs Per Token Falling = Performance Converging + Developer 2/24 2/25 4/25 75% 60% 10% 21% 15% 0% Details on Page 293 USA – LLM #1 China USA – LLM #2 AI Model Compute Costs High / Rising + Inference Costs Per Token Falling = Performance Converging + Developer Change Happening Faster Than Ever? Yes, It Is • AI User + Usage + CapEx Growth = Unprecedented • AI Model Compute Costs High / Rising + Inference Costs Per Token Falling = Performance Converging + Developer

capabilities. Our Applied Team turns those models into products, like ChatGPT Enterprise and our API. And our Deployment Team takes these products into companies to address their most pressing use cases They started with three model evals: 01 Language translation Measuring the accuracy and quality of translations produced 
 by a model. 02 Summarization Evaluating how a model condenses information, using resilient to change. 
 Evals are built around tasks that measure 
 the quality of the output of a model against 
 a benchmark—is it more accurate? More compliant? Safer? Your key metrics will depend on

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 491 38.2 Public API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 492CONTENTS viii 38 Idioms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1132 50.3 API docs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1133 50.4 Implementation Rounding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1300 67.9 API reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1301 68 Delimited

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 491 38.2 Public API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 492CONTENTS viii 38 Idioms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1132 50.3 API docs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1133 50.4 Implementation Rounding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1300 67.9 API reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1301 68 Delimited

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 491 38.2 Public API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 492CONTENTS viii 38 Idioms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1132 50.3 API docs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1133 50.4 Implementation Rounding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1300 67.9 API reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1301 68 Delimited

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457 37.2 Public API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 458 37.3 Sessions Rounding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1112 65.9 API reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1114 66 Delimited seeding) . . . . . . . . . . . . . . . . . . . . . . . . . . . 1428 86.5 Hooking into the Random API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1431 87 Reproducibility 1438 88 SHA 1439

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457 37.2 Public API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 458 37.3 Sessions Rounding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1112 65.9 API reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1114 66 Delimited seeding) . . . . . . . . . . . . . . . . . . . . . . . . . . . 1428 86.5 Hooking into the Random API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1431 87 Reproducibility 1438 88 SHA 1439

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 502 38.2 Public API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503 38.3 Sessions Idioms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1181 51.3 API docs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1182 51.4 Implementation Rounding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1353 68.9 API reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1354 69 Delimited

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503 38.2 Public API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 504 38.3 Sessions Idioms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1179 51.3 API docs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1180 51.4 Implementation Rounding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1351 68.9 API reference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1352 69 Delimited