Amazon Trademark Deploying GluonCV models using TVM© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Deploy GluonCV Models GluonCV Models MXNet Computational Graph 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Deploy GluonCV Models https://arxiv.org/pdf/1907.02154.pdf© 2019, Amazon Web Services, Inc. or its Affiliates. All Nano© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Like GluonCV? Go build! https://gluon-cv.mxnet.io https://github.com/dmlc/gluon-cv© 2019, Amazon Web Services

digital datasets that have been in the making for over three decades; breakthrough large language models (LLMs) that – in effect – found freedom with the November 2022 launch of OpenAI’s ChatGPT with computers are ingesting massive datasets to get smarter and more competitive. Breakthroughs in large models, cost-per-token declines, open-source proliferation and chip performance improvements are making infrastructure: agentic interfaces, enterprise copilots, real-world autonomous systems, and sovereign models. Rapid advances in artificial intelligence, compute infrastructure, and global connectivity are

compressing the Key-Value (KV) cache into a latent vector, while DeepSeekMoE enables training strong models at an economical cost through sparse computation. Compared with DeepSeek 67B, DeepSeek-V2 achieves parameters, DeepSeek-V2 and its chat versions still achieve top-tier performance among open-source models. The model checkpoints are available at h t t p s : / / g i t h u b . c o m / d e e p s e e k - a (b) Figure 1 | (a) MMLU accuracy vs. activated parameters, among different open-source models. (b) Training costs and inference efficiency of DeepSeek 67B (Dense) and DeepSeek-V2. Contents

evals 6 Embed AI into your products 9 Start now and invest early 11 Customize and fine-tune your models 13 Get AI in the hands of experts 16 Unblock your developers 18 Set bold automation goals 21 research and deployment company, OpenAI prioritizes partnering with global companies because our models will increasingly do their best work with sophisticated, complex, interconnected workflows and systems teams. Our Research Team advances the foundations of AI, developing new models and capabilities. Our Applied Team turns those models into products, like ChatGPT Enterprise and our API. And our Deployment

foundations 7 Guardrails 24 Conclusion 32 2 Practical guide to building agents Introduction Large language models are becoming increasingly capable of handling complex, multi-step tasks. Advances in reasoning, to users about the weather.", 7 A practical guide to building agents Selecting your models Different models have different strengths and tradeoffs related to task complexity, latency, and cost. As As we’ll see in the next section on Orchestration, you might want to consider using a variety 
 of models for different tasks in the workflow. Not every task requires the smartest model—a simple retrieval

might need to be optimized for your specific model, regardless of whether you use Gemini language models in Vertex AI, GPT, Claude, or an open source model like Gemma or LLaMA. Besides the prompt, you words? This is also known as the "repetition loop bug", which is a common issue in Large Language Models where the model gets stuck in a cycle, repeatedly generating the same (filler) word, phrase, or “few-shot” prompting. General prompting / zero shot One-shot & few-shot When creating prompts for AI models, it is helpful to provide examples. These examples can help the model understand what you are asking

its Affiliates. All rights reserved. Animesh Jain Amazon SageMaker Neo Compilation of Quantized Models in TVM AWS AI© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Quantization dataset • Finds suitable quantization scale • Produces a quantized graph • Compiling Pre-quantized models – QNN Dialect • TVM ingests a pre-quantized graph in TFLite or MxNet • Use high-level wrapper ops Frontend Parsers • TFLite Pre-quantized Models • In good shape • Supports all Image Classification PreQuantized hosted models • MXNet Pre-quantized Models • Tested internally with MxNet + MKLDNN

remumn dming data AutoTYM 二 QQ octoML Coming Soon to HTVM (Self-Hosted Models) Host Device mized RE -一 一 QQ octoML Transformer Improvements Transformer based models such as BERT have recently become very Popular and require first class support in TVML. ee What relay ONNX frontend to support all opset versions of BERT. 里This enables importing of native ONNX models and those converted from Tensorflow. 5 ， Improve scheduling of batch matrix multiplies. 时”Early autotuning

Unstructured Sparsity - Lots of 'free' wins from exploring sparsity in modern ML models - Can often prune models to 80%+ sparsity(with retraining) - Massive speedups combined with specialized code-generation

workloads and hardware Hardware FrameworksWhy Automation is the Future Clear winner on emerging models in product Competitive on benchmarking type model Quickly enables other optimizations: fusion Dynamic shape workloads More runtime objects: Arrays, Tuples, Trees, ADTs Minimum runtime for dynamic models Credit: Jared Roesch, Haichen Shen et.aluTVM: TVM on bare-metal Devices Support bare-metal J-TAG