ran- domized experiment X : S → R Examples: Discrete random variables (S is discrete) X(s) = True if a randomly drawn person (s) from our class (S) is female X(s) = The hometown X(s) of a randomly drawn drawn person (s) from (S) Examples: Continuous random variables (S is continuous) X(s) = r be the heart rate of a randomly drawn person s in our class S Feng Li (SDU) GDA, NB and EM September 27, 2023 7 a ran- domized experiment X : S → R For continuous random variable X P(a < X < b) = P({s ∈ S : a < X(s) < b}) For discrete random variable X P(X = x) = P({s ∈ S : X(s) = x}) Feng Li (SDU) GDA, NB and

recognition as an example. Our aim is to identify if there is a cat in a given image. We assume X = [X1, X2, · · · , Xn]T is a random variable representing the feature vector of the given image, and Y ∈ Now, given an image x = [x1, x2, · · · , xn]T , out goal is to calculate P(Y = y | X = x) = P(X = x | Y = y)P(Y = y) P(X = x) (2) where y ∈ {0, 1}. In particular, P(Y = y | X = x) is the probability given that the image can be represented by feature vector x, P(X = x | Y = y) is the probability that the image has its feature vector being x given that it is labeled by y, P(Y = y) is the probability

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 516 x 12.4.1 计算机 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 517 Notebook、相关库以及运行本书所需的代码，以快速入门并 获得动手学习经验。 安装 Miniconda 最简单的方法就是安装依赖Python 3.x的Miniconda8。如果已安装conda，则可以跳过以下步骤。访 问Miniconda网站，根据Python3.x版本确定适合的版本。 如果我们使用macOS，假设Python版本是3.9（我们的测试版本），将下载名称包含字符串“MacOSX”的bash脚 sh Miniconda3-py39_4.12.0-MacOSX-x86_64.sh -b 如果我们使用Linux，假设Python版本是3.9（我们的测试版本），将下载名称包含字符串“Linux”的bash脚 本，并执行以下操作： # 文件名可能会更改 sh Miniconda3-py39_4.12.0-Linux-x86_64.sh -b 接下来，初始化终端Shell，以便我们可以直接运行conda。

232 16.4 开发新的正则化器 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232 目录 X 17 约束 Constraints 233 17.1 约束项的使用 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 现在，你可以批量地在训练数据上进行迭代了： # x_train 和 y_train 是 Numpy 数组 -- 就像在 Scikit-Learn API 中一样。 model.fit(x_train, y_train, epochs=5, batch_size=32) 或者，你可以手动地将批次的数据提供给模型： model.train_on_batch(x_batch, y_batch) 只需一行代码就能评估模型性能： 只需一行代码就能评估模型性能： loss_and_metrics = model.evaluate(x_test, y_test, batch_size=128) 或者对新的数据生成预测： classes = model.predict(x_test, batch_size=128) 构建一个问答系统，一个图像分类模型，一个神经图灵机，或者其他的任何模型，就是这么 的快。深度学习背后的思想很简单，那么它们的实现又何必要那么痛苦呢？

by ωT x + b = 0 (1) where ω ∈ Rn is the outward pointing normal vector, and b is the bias term. The n-dimensional space is separated into two half-spaces H+ = {x ∈ Rn | ωT x + b ≥ 0} and H− = {x ∈ Rn | | ωT x + b < 0} by the hyperplane, such that we can classify a given point x0 ∈ Rn according to sign(ωT x + b). Specifically, given a point x0 ∈ Rn, its label y is defined as y0 = sign(ωT x0 + b), i.e e. y0 = � 1, ωT x0 + b ≥ 0 −1, otherwise (2) Given any x0 ∈ Rn, we can calculate the signed distance from x to the hyperplane as d0 = ωT x0 + b ∥ω∥ = � ω ∥ω∥ �T x0 + b ∥ω∥ (3) The sign of the distance

目前普通消费者能够使用的深度学习加速硬件设备主要来自 NVIDIA 的 GPU 显卡， 图 1.12 例举了从 2008 年到 2017 年 NVIDIA GPU 和 x86 CPU 的每秒 10 亿次的浮点运算数 (GFLOPS)的指标变换曲线。可以看到，x86 CPU 的曲线变化相对缓慢，而 NVIDIA GPU 的浮点计算能力指数式增长，这主要是由日益增长的游戏计算量和深度学习计算量等业务 驱动的。 预览版202112 TensorFlow 接口设计频繁变动，功能设计重复冗余， 符号式编程开发和调试非常困难等问题，TensorFlow 1.x 版本一度被业界诟病。2019 年，Google 推出 TensorFlow 2 正式版本，将以动态图优先模式运行，从而能够避免 TensorFlow 1.x 版本的诸多缺陷。 ❑ PyTorch 是 Facebook 基于原 Torch 框架推出的采用 Python 作为主要开发语言的深度学 框架是业界使用最为广泛的两个深度学习框架， TensorFlow 在工业界拥有完备的解决方案和用户基础，但是 TensorFlow 2 和 TensorFlow 1.x 版本并不兼容，导致几乎所有基于 TensorFlow 1.x 开发的算法、框架等都需要修改甚至重 写，因此可以将 TensorFlow 2 视为一个全新的框架。PyTorch 得益于其精简灵活的接口设 计，可以快速搭建和调试网络模型，在学术界获得好评如潮。根据

NVIDIA cuDNN 8.9.3 ‣ NVIDIA NCCL 2.18.3 ‣ NVIDIA RAPIDS™ 23.06 ‣ Apex ‣ rdma-core 39.0 ‣ NVIDIA HPC-X 2.15 ‣ OpenMPI 4.1.4+ ‣ GDRCopy 2.3 ‣ TensorBoard 2.9.0 ‣ Nsight Compute 2023.1.1.4 ‣ Nsight Systems 0a0+b6df043 21.10 NVIDIA CUDA 11.4.2 with cuBLAS 11.6.5.2 1.10.0a0+0aef44c TensorRT 8.0.3.4 for x64 Linux TensorRT 8.0.2.2 for Arm SBSA Linux 21.09 NVIDIA CUDA 11.4.2 TensorRT 8.0.3 21.08 NVIDIA to cast to FP16. FP16 operations require 2X reduced memory bandwidth (resulting in a 2X speedup for bandwidth-bound operations like most pointwise ops) and 2X reduced memory storage for intermediates (reducing

based linear classifier defined by ω and b Prediction rule: y = sign(ωTx + b) Given: Training data {(x(i), y(i))}i=1,··· ,m Goal: Learn ω and b that achieve the maximum margin For now, assume that entire normal vector The margin γ(i) is the signed distance between x(i) and the hyperplane ωT � x(i) − γ(i) ω ∥ω∥ � + b = 0 ⇒ γ(i) = � ω ∥ω∥ �T x(i) + b ∥ω∥ !" !" !"# + % = 0 ! !(#) !(#) = & & ' ((#) (Contd.) Hyperplane: ωTx + b = 0, where ω is the normal vector The margin γ(i) is the distance between x(i) and the hyperplane Now, the margin is signed If y (i) = 1, γ(i) ≥ 0; otherwise, γ(i) < 0 !" !"

Malignant? (Yes) 1 (No) 0 ℎ! ? = ?"? 0.5 The threshold classifier output hθ(x) at 0.5 If hθ(x) ≥ 0.5, predict y = 1 If hθ(x) < 0.5, predict y = 0 Feng Li (SDU) Logistic Regression September 20, 2023 classification problem, but the linear regression model hθ(x) = θTx can be > 1 or < 0 to fit the given training example Logistic regression: 0 ≤ hθ(x) ≤ 1 Feng Li (SDU) Logistic Regression September 20, 2023 defines hθ(x) using the logistic function hθ(x) = g(θTx) = 1 1 + e−θT x Interpretation of the hypothesis output hθ(x): Estimated probability that y = 1 on input x Example: if x = �x0 x1 � = � 1 Tumor

discarded, based on the tolerance for loss in quality. The JPEG and MP3 formats are able to achieve a 10-11x compression without any perceptible loss in quality. However, further compression might lead to degradation sine waves. Let’s dig deeper into its mechanics using an example. Let’s assume we have a variable x which takes a 32-bit floating point value in the range [-10.0, 10.0]. We need to transmit a collection quantization to reduce transmission size and thus save some costs? What if it did not matter to us if x was stored/transmitted with some error (-5.023 v/s -5.0)? If we can tolerate some loss of precision