Lecture Notes on Gaussian Discriminant Analysis, Naive Bayes and EM Algorithm Feng Li fli@sdu.edu.cn Shandong University, China 1 Bayes’ Theorem and Inference Bayes’ theorem is stated mathematically through parameters θ = {P(X = x | Y = y), P(Y = y)}x,y. 2 Gaussian Discriminant Analysis In Gaussian Discriminate Analysis (GDA) model, we have the following as- sumptions: • A1: Y ∼ Bernoulli(ψ): Y follows is given by pX|Y (x | 1) = 1 (2π)n/2|Σ|1/2 exp � −1 2(x − µ1)T Σ−1(x − µ1) � (7) Given m sample data {(x(i), y(i))}i=1,··· ,m, the log-likelihood is defined as ℓ(ψ, µ0, µ1, Σ) = log m � i=1 pX,Y (x(i)

Lecture 5: Gaussian Discriminant Analysis, Naive Bayes and EM Algorithm Feng Li Shandong University fli@sdu.edu.cn September 27, 2023 Feng Li (SDU) GDA, NB and EM September 27, 2023 1 / 122 Outline Outline 1 Probability Theory Review 2 A Warm-Up Case 3 Gaussian Discriminate Analysis 4 Naive Bayes 5 Expectation-Maximization (EM) Algorithm Feng Li (SDU) GDA, NB and EM September 27, 2023 2 / 122 pX(x) , ∀y We calculate pX|Y (x | y) for ∀x, y and pY (y) for ∀y according to the given training data Fortunately, we do not have to calculate pX(x), because arg max y pY |X(y | x) = arg max y pX|Y

of algorithms, inspired from the model of human brain. Deep learning is becoming more popular in data science fields like robotics, artificial intelligence(AI), audio & video recognition and image recognition Keras Keras runs on top of open source machine libraries like TensorFlow, Theano or Cognitive Toolkit (CNTK). Theano is a python library used for fast numerical computation tasks. TensorFlow is the -U scikit-learn Seaborn Seaborn is an amazing library that allows you to easily visualize your data. Use the below command to install: pip install seaborn You could see the message similar as specified

In the first chapter, we briefly introduced learning techniques such as regularization, dropout, data augmentation, and distillation to improve quality. These techniques can boost metrics like accuracy precision, recall, etc. which often are our primary quality concerns. We have chosen two of them, namely data augmentation and distillation, to discuss in this chapter. This is because, firstly, regularization and dropout are fairly straight-forward to enable in any modern deep learning framework. Secondly, data augmentation and distillation can bring significant efficiency gains during the training phase, which

functionality. PyTorch also includes standard defined neural network layers, deep learning optimizers, data loading utilities, and multi-gpu, and multi-node support. Functions are executed immediately instead nvcr.io/nvidia/ pytorch:-py3 Note: If you use multiprocessing for multi-threaded data loaders, the default shared memory segment size with which the container runs might not be enough To pull data and model descriptions from locations outside the container for use by PyTorch or save results to locations outside the container, mount one or more host directories as Docker® data volumes

for weight sharing. However, quantization falls behind in case the data that we are quantizing is not uniformly distributed, i.e. the data is more likely to take values in a certain range than another equally ranges (bins), regardless of the frequency of data. Clustering helps solve that problem by adapting the allocation of precision to match the distribution of the data, which ensures the decoded value deviates general concept behind quantization. However, what happens if our and were outliers, and the real data was clustered in some smaller concentrated ranges? Quantization will still assign an equal number

import Image from torch import nn from torch.nn import functional as F from torch.utils import data from torchvision import transforms 目标受众 本书面向学生（本科生或研究生）、工程师和研究人员，他们希望扎实掌握深度学习的实用技术。因为我们 从头开始解 编写了一个“学习”程序。如果我们用一个巨大的带标签的数 据集，它很可能可以“学习”识别唤醒词。这种“通过用数据集来确定程序行为”的方法可以被看作用数据 编程（programming with data）。比如，我们可以通过向机器学习系统，提供许多猫和狗的图片来设计一个 “猫图检测器”。检测器最终可以学会：如果输入是猫的图片就输出一个非常大的正数，如果输入是狗的图片 就会输出一个非常小的负数 学习的一个主要分支，本节稍后的内容将对其 进行更详细的解析。 1.2 机器学习中的关键组件 首先介绍一些核心组件。无论什么类型的机器学习问题，都会遇到这些组件： 1. 可以用来学习的数据（data）； 2. 如何转换数据的模型（model）； 3. 一个目标函数（objective function），用来量化模型的有效性； 4. 调整模型参数以优化目标函数的算法（algorithm）。

Previous Work 38 Effectiveness How to make good use of multiple frames? Remaining Challenges 39 Data from Vid4 [Ce Liu et al.] Bicubic x4 Misalignment Occlusion Large motion Effectiveness How to Scalable Arbitrary input size Arbitrary scale factor Arbitrary temporal frames Our Method 44 45 Data from Vid4 [Ce Liu et al.] Motion Estimation Our Method 46 ???????????????????????? ?????????? skip connections Decoder Details from multi-frames Analysis 52 3 identical frames Output (identical) Details from multi-frames Analysis 53 3 consecutive frames Output (consecutive) Output

regression, logistic re- gression, regularization, Gaussian discriminant analysis, Naive Bayes, EM algorithm, SVM, K-means, factor analysis, PCA, neural networks etc. 68 hours (4 hours/week × 17 weeks) Labs People’s Posts and Telecommunications Press, 2016 Trevor Hastie, The Elements of Statistical Learning: Data Mining, Inference, and Prediction (2nd Ed.), World Publishing Corporation, 2015 Christopher M. Bishop or mail filter Personalized tutoring Discover new knowledge from large databases (data mining) Market basket analysis (e.g. diapers and beer) Medical information mining (e.g. migraines to calcium channel

bin样本的之和 bin内所有样本的一阶导之和 bin内所有样本的二阶导之和 可能的候选点分裂点个数 等于样本取值个数减一 排序完了之后，我们就选出a * data_num个梯度大的，然后从剩下的那些样本里面选出b*data_num个梯度小的： 这里是8个样本，所以a*8=2，b*8=2，1−? ? = 3。 即先选出2个梯度大的样本，然后从剩下的里面随机选出2个梯度小的样本 这里选 28(2):393-400. [6] FRIEDMAN J H . Stochastic gradient boosting[J]. Computational Statistics & Data Analysis, 2002, 38. 49 参考文献 [7] FRIEDMAN J H. Greedy function approximation: A gradient boosting machine[J] Annals of statistics, 2001: 1189–1232. [8] MACQUEEN J, OTHERS. Some methods for classification and analysis of multivariate observations[C]//Proceedings of the fifth Berkeley symposium on mathematical statistics