Lecture Notes on Gaussian Discriminant Analysis, Naive Bayes and EM Algorithm Feng Li fli@sdu.edu.cn Shandong University, China 1 Bayes’ Theorem and Inference Bayes’ theorem is stated mathematically through parameters θ = {P(X = x | Y = y), P(Y = y)}x,y. 2 Gaussian Discriminant Analysis In Gaussian Discriminate Analysis (GDA) model, we have the following as- sumptions: • A1: Y ∼ Bernoulli(ψ): Y follows ˜y | X = ˜x) = pY |X(˜y | ˜x) = p(˜x | ˜y)p(˜y) p(˜x) where ˜y = 0, 1. 3 Gaussian Discriminant Analysis and Logistic Regression By far, we introduce two classification algorithms, Logistic Regression

Lecture 5: Gaussian Discriminant Analysis, Naive Bayes and EM Algorithm Feng Li Shandong University fli@sdu.edu.cn September 27, 2023 Feng Li (SDU) GDA, NB and EM September 27, 2023 1 / 122 Outline Outline 1 Probability Theory Review 2 A Warm-Up Case 3 Gaussian Discriminate Analysis 4 Naive Bayes 5 Expectation-Maximization (EM) Algorithm Feng Li (SDU) GDA, NB and EM September 27, 2023 2 / 122 = � −1 −1.5 � Feng Li (SDU) GDA, NB and EM September 27, 2023 41 / 122 Gaussian Discriminant Analysis (Contd.) Y ∼ Bernoulli(ψ) P(Y = 1) = ψ P(Y = 0) = 1 − ψ Probability mass function pY (y) = ψy(1

skip connections Decoder Details from multi-frames Analysis 52 3 identical frames Output (identical) Details from multi-frames Analysis 53 3 consecutive frames Output (consecutive) Output Layer v.s. Baseline Analysis 54 Output (baseline) ????????????????????????→0 BW Resize Backward warping + Resize (baseline) Ablation Study: SPMC Layer v.s. Baseline Analysis 55 Output (SPMC) deconv 86 Data from GOPRO dataset Using Different Number of Scales Analysis 87 1 scale Input 2 scales 3 scales Baseline Models Analysis 88 Model SS SC w/o R RNN SR-Flat Param 2.73M 8.19M 2.73M 3.03M

which makes deep learning a very powerful tool. Deep learning algorithms are also useful for the analysis of unstructured data. Let us go through the basics of deep learning in this chapter. Artificial floatx() 'float32' Let us understand some of the significant backend functions used for data analysis in brief: get_uid() It is the identifier for the default graph. It is defined below: >>> It is used to convert positive into dense vectors of fixed size. Its main application is in text analysis. The signature of the Embedding layer function and its arguments with default value is as follows

regression, logistic re- gression, regularization, Gaussian discriminant analysis, Naive Bayes, EM algorithm, SVM, K-means, factor analysis, PCA, neural networks etc. 68 hours (4 hours/week × 17 weeks) Labs Personalized tutoring Discover new knowledge from large databases (data mining) Market basket analysis (e.g. diapers and beer) Medical information mining (e.g. migraines to calcium channel blockers to these with fewer ones, without loss of information. On simple way is to use PCA (Principal Component Analysis) Suppose that all data are in a space, we first find the direction of high- est variance of these

sentiments of the original and the transformed sentences are consistent. This can be used for sentiment analysis. This transformation has two main implications. First, it augments our dataset with additional examples that the agreement between the text and the original label is intact. In the context of sentiment analysis, the transformation must preserve the original sentiment of the text. For a language translation augmentations. It provides a simple 5 Maas, Andrew, et al. "Learning word vectors for sentiment analysis." Proceedings of the 49th annual meeting of the association for computational linguistics: Human

estate purchase predictions FINRA • Anomaly detection, sequence matching, regression analysis, network/tribe analysis Netflix • Recommendation engine Pinterest • Image recognition search Fraud.net •

文本方面的词袋模型、词嵌入模型等 3. 特征提取 18 许永洪,吴林颖.中国各地区人口特征和房价波动的动态关系[J].统计研究,2019,36(01) 1.PCA(Principal Component Analysis，主成分分析) PCA 是降维最经典的方法，它旨在是找到数据中的主成分，并利 用这些主成分来表征原始数据，从而达到降维的目的。 PCA 的思想是通过坐标轴转换，寻找数据分布的最优子空间。 3. 特征提取 降维 19 许永洪,吴林颖.中国各地区人口特征和房价波动的动态关系[J].统计研究,2019,36(01) 2. ICA(Independent Component Analysis，独立成分分析) ICA独立成分分析，获得的是相互独立的属性。ICA算法本质寻找一 个线性变换 ? = ??，使得 ? 的各个特征分量之间的独立性最大。 PCA 对数据 进行降维 ICA

PCA(主成分分析) 01 降维概述 02 SVD(奇异值分解) 03 PCA(主成分分析) 31 3.PCA(主成分分析) 主成分分析（Principal Component Analysis,PCA）是一种降维方法， 通过将一个大的特征集转换成一个较小的特征集，这个特征集仍然包含 了原始数据中的大部分信息，从而降低了原始数据的维数。 减少一个数据集的特征数量自然是以牺牲准确性为代价的，但降维的诀 Dimensionality of Data with Neural Networks.[J]. Science, 2006. [3] Jolliffe I T . Principal Component Analysis[J]. Journal of Marketing Research, 2002, 87(4):513. [4] 李航. 统计学习方法[M]. 北京: 清华大学出版社,2019. [5]

28(2):393-400. [6] FRIEDMAN J H . Stochastic gradient boosting[J]. Computational Statistics & Data Analysis, 2002, 38. 49 参考文献 [7] FRIEDMAN J H. Greedy function approximation: A gradient boosting machine[J] Annals of statistics, 2001: 1189–1232. [8] MACQUEEN J, OTHERS. Some methods for classification and analysis of multivariate observations[C]//Proceedings of the fifth Berkeley symposium on mathematical statistics