time could still turn out to be expensive. There is also a very real concern around the carbon footprint of datacenters that are used for training and deploying these large models. Large organizations about. First, we have quality metrics like accuracy, precision, recall, F1, AUC, etc. Then we have footprint metrics like model size, latency, RAM, etc. Empirically, we have seen that larger deep learning train and deploy hence worse footprint. On the other hand, smaller and shallower models might have suboptimal quality. Figure 1-6: Trade-offs between quality metrics and footprint metrics. In case we have

compression techniques. Compression techniques aim to reduce the model footprint (size, latency, memory etc.). We can reduce the model footprint by reducing the number of trainable parameters. However, this approach categories: footprint metrics such as model size, prediction latency, RAM consumption and the quality metrics, such as accuracy, F1, precision and recall as shown in table 2-1. Footprint Metrics Quality ● Accuracy ● Precision ● Recall ● F1 ● AUC Table 2-1: A few examples of footprint and quality metrics. The footprint and the quality metrics are typically at odds with each other. As stated earlier

that enable us to achieve our quality goals. High quality models have an additional benefit in footprint constrained environments like mobile and edge devices where they provide the flexibility to trade learning techniques. It is followed by a short discussion on exchanging model quality and model footprint. An in-depth discussion of data augmentation and distillation follows right after. Following the and/or label efficient training setup, can we exchange some of this to achieve a model with a better footprint? The next subsection elaborates it further. Using learning techniques to build smaller and faster

of fully connected layers. Exercise: Sparsity improves compression Let's import the required libraries to start with. We will use the gzip python module for demonstrating compression. The code for this case of this convolutional layer, we can drop rows, columns, kernels, and even whole channels. Libraries like XNNPACK3,4 can help accelerate networks on a variety of web, mobile, and embedded devices, precision. We will leave the biases untouched since they do not contribute significantly to the layer’s footprint. num_bits = 8 weights_dequantized, weights_reconstruction_error_quant = simulate_quantization(

estimators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432 23 Related Python libraries 433 23.1 la (larry) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433 24 Comparison with R / R libraries 435 24.1 data.frame . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . intended to integrate well within a scientific computing environment with many other 3rd party libraries. Here are just a few of the things that pandas does well: • Easy handling of missing data (represented

turning the knobs (the hyperparameters) until we are satisfied with the sound (model quality and footprint) that each string produces. Unlike the guitar which has a few knobs, the hyperparameter search space hyperparameter values which achieve the minimum loss are the winners. Let's start by importing the relevant libraries and creating a random classification dataset with 20 samples, each one assigned to one of the five

changes to add a couple of windows and a balcony. Similarly, to gain orders of magnitude in terms of footprint or quality, we should consider employing suitable efficient architectures. The progress of deep Efficient Architectures aim to improve model deployability by proposing novel ways to reduce model footprint and improve inference efficiency while preserving the problem solving capabilities of their giant 2. Even after compression, the vocabulary itself is large: Large vocabularies have a tangible footprint by themselves, which excludes the actual embeddings. They are persisted with the model to help with

without increasing the footprint of the model (size, latency, etc). And as we have described earlier, some of these improved quality metrics can be traded off for a smaller footprint as desired. Continuing techniques to help you improve your model’s quality metrics without taking a hit on any of the footprint metrics. These techniques might get superseded by other better methods over time, but again our chapter 3, we found that distillation was a very handy technique to improve our model’s quality v/s footprint tradeoff. The motivation behind Subclass Distillation (Mueller et al.24) comes from the observation

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 564 26 Comparison with R / R libraries 565 26.1 Base R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . intended to integrate well within a scientific computing environment with many other 3rd party libraries. Here are just a few of the things that pandas does well: • Easy handling of missing data (represented a collection of useful recipes in pandas (and that we want contributions!). There are several libraries that are now Recommended Dependencies 1.4.1 Selection Choices Starting in 0.11.0, object selection

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 624 26 Comparison with R / R libraries 625 26.1 Base R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . intended to integrate well within a scientific computing environment with many other 3rd party libraries. Here are just a few of the things that pandas does well: • Easy handling of missing data (represented a collection of useful recipes in pandas (and that we want contributions!). There are several libraries that are now Recommended Dependencies 1.5.1 Selection Choices Starting in 0.11.0, object selection