compression techniques. Compression techniques aim to reduce the model footprint (size, latency, memory etc.). We can reduce the model footprint by reducing the number of trainable parameters. However requires many trials and evaluations to reach a smaller model, if it is at all possible. Second, such an approach doesn’t generalize well because the model designs are subjective to the specific problem. In In this chapter, we introduce Quantization, a model compression technique that addresses both these issues. We’ll start with a gentle introduction to the idea of compression. Details of quantization and

(continued) • Which one do you think is better? PyTorch! • Easy Interface − easy to use API. The code execution in this framework is quite easy. Also need a fewer lines to code in comparison. • It is easy to ) • optimizers Prepare Input Data •Load data •Iterate over examples Train Model •Train weights Evaluate Model •Visualise Tensor • Tensor? • PyTorch Tensors are just like numpy arrays, but …... Model • In PyTorch, a model is represented by a regular Python class that inherits from the Module class. • Two components • __init__(self): it defines the parts that make up the model —in our

together? We have four options: none, quantization, clustering, and both. We would need to train a model with each of these four options to make an informed decision. Blessed with a large research community multiple parameters. Figure 7-1: The plethora of choices that we face when training a deep learning model in the computer vision domain. A Search Space for n parameters is a n-dimensional region such that understand this using the earlier example for choosing quantization and/or clustering techniques for model optimization. We have a search space which has two boolean valued parameters: quantization and clustering

样本分布不均匀 ✗ 隐私保护 • 多个环节 • 多种模型 ✗ 海量参数 ✗ 海量数据 从FM到DeepFM rt 增 加了10倍怎么优化？ 2.模型效果优 化困难 1.方案复杂 Data Model Compute Platform 要求:  准确: 低噪声  全面: 同分布 模型选型:  容量大  计算量小 训练推理:  高qps, 低rt  支持超大模型  性价比 Tensorflow PyTorch Parameter Server MPI TreeModel SQL MapReduce Blink  场景丰富: 图像/视频/推荐/搜索  大数据+大模型: Model Zoo  跨场景+跨模态  开箱即用: 封装复杂性  白盒化, 可扩展性强  积极对接开源系统+模型 FTRL SGD Adam Solutions Librarys 优势: Graph-Learn: 分布式图算法库 标准化: Standard Solutions Continuous Optimization: Active learning Data Label Model Serving CV / NLP解决方案: EAS Web App Mobile App On-prem System 3 1 2 证件扫描 活体检测 人脸比对 •

in ANALOG magazine (1991) So far, we have discussed generic techniques which are agnostic to the model architecture. These techniques can be applied in NLP, vision, speech or other domains. However, owing challenges. What good is a model that cannot be deployed in practical applications! Efficient Architectures aim to improve model deployability by proposing novel ways to reduce model footprint and improve running on mobile and edge devices. We have also set up a couple of programming projects for a hands-on model optimization experience using these efficient layers and architectures. Let’s start our journey with

you'll go.” ― Dr. Seuss Model quality is an important benchmark to evaluate the performance of a deep learning model. A language translation application that uses a low quality model would struggle with consumer effectively with others who speak different languages. An application that employs a high quality model with a reasonable translation accuracy would garner better consumer support. In this chapter, our picked to benchmark learning techniques. It is followed by a short discussion on exchanging model quality and model footprint. An in-depth discussion of data augmentation and distillation follows right after

--shm-size= in the command line to docker run --gpus all To pull data and model descriptions from locations outside the container for use by PyTorch or save results to locations and 2X reduced memory storage for intermediates (reducing the overall memory consumption of your model). Additionally, GEMMs and convolutions with FP16 inputs can run on Tensor Cores, which provide an NVIDIA Volta™ tensor cores by using the latest deep learning example networks and model scripts for training. Each example model trains with mixed precision Tensor Cores on NVIDIA Volta and NVIDIA Turing™,

keras：分布式和高性能的 Keras • 构建和训练模型的高层次 API • API 完全兼容原生 Keras • 支持保存和加载 TensorFlow SavedModel • 支持 Eager Execution • 支持分布式训练 tf.data：功能强大的数据管理模块 支持多种数据处理 图像解码 Shuffle py_function 重采样 支持多种数据格式 图像文件 文本文件 CSV | 腾讯互娱基于 CPU 环境的分布式 YOLOv3 实现》 魂斗罗游戏中识别角色 K8s Pod K8s Pod K8s Pod Horovod(CPU) on Kubernetes model TensorFlow Serving Keras 模型训练 • DataGenerator • 随机读取 … … 图片训练集 Ceph 数据并行实现 基于 Horoved CPU 平台的分布式模型训练及部署

........................................................................................... 17 Model ................................................................................................. ............................................................................... 58 10. Keras ― Model Compilation ..................................................................................... ..... 61 Compile the model ........................................................................................................................................ 62 Model Training ..............

49 4.3.1 Model 类 API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 4.3.2 Model 的实用属性 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 4.3.3 Model 类模型方法 . . . . . . . . . . . . . . . 239 20.8 plot_model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 20.9 multi_gpu_model . . . . . . . . . . . . . . . . . . . . . . Keras 的核心数据结构是 model，一种组织网络层的方式。最简单的模型是 Sequential 顺 序模型，它是由多个网络层线性堆叠的栈。对于更复杂的结构，你应该使用 Keras 函数式 API， 它允许构建任意的神经网络图。 Sequential 顺序模型如下所示： from keras.models import Sequential model = Sequential()