users, see NVIDIA ® GPU Cloud ™ (NGC) container registry installation documentation based on your platform. ‣ Ensure that you have access and can log in to the NGC container registry. Refer to NGC Getting C++ APIs. ‣ Starting with the 22.05 release, the PyTorch container is available for the Arm SBSA platform. ‣ Deep learning framework containers 19.11 and later include experimental support for Singularity uses modified model architecture hyperparameters, our modifications were made to achieve better hardware usage and to take advantage of Tensor Cores. This model script is available on GitHub. ‣ Jasper

on efficient deep learning to be categorized in roughly four core areas, with infrastructure and hardware forming the foundation (see Figure 1-7). 7 Lossy compression techniques allow you to compress data which comprises the core areas and relevant techniques as well as the foundation of infrastructure, hardware and tools.) Let us go over each of these areas individually. Compression Techniques These are another example, to get size and latency improvements with quantized models, we need the inference platform to support common neural net layers in quantized mode. TFLite supports quantized models, by allowing

What is Bloomberg? The Bloomberg Terminal delivers a diverse array of information on a single platform to facilitate financial decision- making. 4 © 2018 Bloomberg Finance L.P. All rights reserved 0/deed.en K80 © 2018 Bloomberg Finance L.P. All rights reserved. Back to 2018 Heterogeneous Hardware Modified from https://upload.wikimedia.org/wikipedia/commons/6/67/Kubernetes_logo.svg and https://commons

here. To get improvement in latency (inference or training), we also need to make sure that the hardware can exploit the sparsity in our network and skip some computation. Most of our computation in deep improvements by pruning connections such that there is a certain structure to the sparsity. This helps hardware implementations leverage that structure for faster inference. For instance, NVIDIA GPUs rely on is 50% sparse, the matrix multiplication can be performed in half the time. With the advent of hardware support for sparsity and many industrial and academic use cases reporting significant improvements

complexity. And since they are such a fundamental block of models, they have been optimized both in hardware and software. Let’s take a look at how we can optimize a slightly easier version of this operation require support from the underlying hardware. Moreover, multiplications and divisions are cheaper at lower precisions like b=8 and are well supported by the hardware technologies like the fixed-point SIMD

(Tensor Processing Units) Google Cloud TPU. https://cloud.google.com/tpu NVIDIA V100 TPU v2 TPU v3 Hardware Architecture NVIDIA Volta GPU Google Cloud TPU Google Cloud TPU Memory 16GB / 32GB 64GB 128GB

receiving upgrades to your computer, except that these new upgrades don’t make it slower because your hardware is older, but actually make it perform better than it used to. How nifty! In some cases you can

多个环节 • 多种模型 ✗ 海量参数 ✗ 海量数据 从FM到DeepFM rt 增 加了10倍怎么优化？ 2.模型效果优 化困难 1.方案复杂 Data Model Compute Platform 要求:  准确: 低噪声  全面: 同分布 模型选型:  容量大  计算量小 训练推理:  高qps, 低rt  支持超大模型  性价比 流程长、环节多:  推荐场景: 测、语音识别 • 数据集管理 • 主动学习 • 智能标注 itags AI SaaS服务（OCR、语音识别、推荐系统、金融风控、疾病预测等） Infrastructure PAI平台（Platform of Artificial Intelligence） • 一键部署、弹性扩缩 • 多框架、多语言 • 推理优化Blade • 多维度监控+报警 • 自定义镜像 • 全托管+半托管 可视化建模(Designer) 分布式训练(DLC) 在线服务(EAS) 生态市场 开发者工具 • CLI • PAIFlow • OpenAPI AI能力 体验中心 开源 PAI平台（Platform of Artificial Intelligence） Deep Learning Container 数据量大而全 先进的模型结构 业务场景复杂 计算力强、性价比高 提供 支撑

TensorFlow ���������� Baylor, Denis, et al. "Tfx: A tensorflow-based production-scale machine learning platform." Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining TensorFlow ���������� Baylor, Denis, et al. "Tfx: A tensorflow-based production-scale machine learning platform." Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

SkyPilot master branch automatically cloned by running: # NOTE: '--platform linux/amd64' is needed for Apple silicon Macs docker run --platform linux/amd64 \ -td --rm --name sky \ -v "$HOME/.sky:/root/.sky:rw"