Train-Val-Test划分 主讲人：龙良曲 Recap How to detect Splitting Train Set Test Set For example 60K 10K test while train train test trade-off Overfitt ing For others judge ▪ Kaggle Train Set Test Set Set Val Set Unavailable train-val-test K-fold cross-validation Train Set Test Set Val Set k-fold cross validation ▪ merge train/val sets ▪ randomly sample 1/k as val set 下一课时 减轻Overfitting Thank

prepared for professionals who are aspiring to make a career in the field of deep learning and neural network framework. This tutorial is intended to make you comfortable in getting started with the Keras framework 12 Convolutional Neural Network (CNN) ........................................................................................................... 13 Recurrent Neural Network (RNN) .................. ........................................................... 71 12. Keras ― Convolution Neural Network ................................................................................................

章 分类问题 2 集共 70000 张图片。其中 60000 张图片作为训练集?train(Training Set)，用来训练模型，剩 下 10000 张图片作为测试集?test(Test Set)，用来预测或者测试，训练集和测试集共同组成 了整个 MNIST 数据集。 考虑到手写数字图片包含的信息比较简单，每张图片均被缩放到28 × 28的大小，同时 只保留了灰度信息，如图 ? (?) − ?? (?)) 2 10 ?=1 ? ?=1 只需要采用梯度下降算法来优化损失函数得到?和?的最优解，然后再利用求得的模型去 预测未知的手写数字图片? ∈ ?test即可。 3.4 真的解决了吗 按照上述方案，手写数字图片识别问题似乎得到较好地解决？事实果真如此吗？深入 研究的话，就会发现，至少存在两大问题： 预览版202112 第 3 章 分类问题 模块的输出??连同它的网络层参数??和??等称为一层网络层。特别地，对于网络中间的 层，也叫作隐藏层，最后一层也叫作输出层。这种由大量神经元模型连接形成的网络结构 称为神经网络(Neural Network)。从这里可以看到，神经网络并不难理解，神经网络每层的 节点数和神经网络的层数或结构等决定了神经网络的复杂度。 预览版202112 第 3 章 分类问题 10 输入层：?

to achieve the desired model quality on our task. 2. Fine-tuning: This step adapts a pre-trained network to a specific task. Fine-tuning is compute efficient since we reuse the same base model for all the They demonstrate that using a network pre-trained in this fashion improves the quality of the final object detection task, as compared to randomly initializing the network. Similarly, another task is to 6-4 (b)). The authors report that the network trained in a self-supervised manner this way can be fine-tuned to perform nearly as well as a fully supervised network. 3 Gidaris, Spyros, et al. "Unsupervised

weight of that connection). Can we do the same with neural networks? Can we optimally prune the network connections, remove extraneous nodes, etc. while retaining the model’s performance? In this chapter depicts two networks. The one on the left is the original network and the one on the right is its pruned version. Note that the pruned network has fewer nodes and some retained nodes have fewer connections Figure 5-1: An illustration of pruning weights (connections) and neurons (nodes) in a neural network consisting of fully connected layers. Exercise: Sparsity improves compression Let's import the

fed to improve the target models. 12 Hendrycks, Dan, and Thomas Dietterich. "Benchmarking neural network robustness to common corruptions and perturbations." arXiv preprint arXiv:1903.12261 (2019). 11 introduce techniques like Synthetic Minority Oversampling Technique16 (SMOTE) and Generative Adversarial Network17 (GAN) which can generate synthetic data for images. While SMOTE leverages statistical models for learning for this purpose. A GAN is composed of two neural networks: a generator network and a discriminator network as shown in figure 3-15. The generator creates synthetic samples from random inputs

Compression techniques are used to achieve an efficient representation of one or more layers in a neural network with a possible quality trade off. The efficiency goals could be the optimization of the model with techniques can help reduce complexity and improve generalization. Let us consider an arbitrary neural network layer. We can abstract it using a function with an input and parameters such that . In the case case of a fully-connected layer, is a 2-D matrix. Further, assume that we can train another network with far fewer parameters ( ) such that the outputs are approximately the same . Such a model is useful

sentence “the quick brown fox jumps over the lazy dog”, we can mask the word “jumps” and let the neural network predict the word it thinks fits in the sentence based on the surrounding words (context). Mathematically step, we train a model which trains the embedding table along with it. We use a single hidden layer network9 with a softmax classification head for this task. The size of the softmax classification head is been initialized with random values, hence most of the predictions will be guesses. As we train the network, it will start to learn meaningful embeddings for the context words, and it will begin to pick up

training process: performance and convergence. Hyperparameters like number of filters in a convolution network or 1 Note that this search space is just choosing if we are applying the techniques. The techniques tensorflow.keras import layers, optimizers train_ds, val_ds, test_ds = tfds.load( 'oxford_flowers102', split=['train', 'validation', 'test'], as_supervised=True, read_config=tfds.ReadConfig(try_autocache=False) return image, label train_ds = train_ds.map(resize_image) val_ds = val_ds.map(resize_image) test_ds = test_ds.map(resize_image) Note that the create_model() function here has two additional parameters:

44 4.2.3.5 train_on_batch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 4.2.3.6 test_on_batch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 4.2.3.7 predict_on_batch 52 4.3.3.5 train_on_batch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 4.3.3.6 test_on_batch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 4.3.3.7 predict_on_batch y_batch) 只需一行代码就能评估模型性能： loss_and_metrics = model.evaluate(x_test, y_test, batch_size=128) 或者对新的数据生成预测： classes = model.predict(x_test, batch_size=128) 构建一个问答系统，一个图像分类模型，一个神经图灵机，或者其他的任何模型，就是这么 的快