EM Algorithm Feng Li Shandong University fli@sdu.edu.cn September 27, 2023 Feng Li (SDU) GDA, NB and EM September 27, 2023 1 / 122 Outline 1 Probability Theory Review 2 A Warm-Up Case 3 Gaussian Naive Bayes 5 Expectation-Maximization (EM) Algorithm Feng Li (SDU) GDA, NB and EM September 27, 2023 2 / 122 Probability Theory Review Sample space, events and probability Conditional probability Conditional probability distribution Bayes’ Theorem ... ... Feng Li (SDU) GDA, NB and EM September 27, 2023 3 / 122 Sample Space, Events and Probability A sample space S is the set of all possible outcomes

Lecture 1: Overview Feng Li Shandong University fli@sdu.edu.cn September 6, 2023 Feng Li (SDU) Overview September 6, 2023 1 / 57 Lecture 1: Overview 1 About the Course 2 Machine Learning: What and of Machine Learning 4 Some Basic Concepts of Machine Learning Feng Li (SDU) Overview September 6, 2023 2 / 57 Instructor Prof. Feng Li Web: https://funglee.github.io Office: N3-312-1 Education: 2010-2015 Systems, Wireless Net- works, Mobile Computing, Internet of Things. Feng Li (SDU) Overview September 6, 2023 3 / 57 Course Information We will investigate fundamental concepts, techniques and algorithms in

Regression Feng Li Shandong University fli@sdu.edu.cn September 13, 2023 Feng Li (SDU) Linear Regression September 13, 2023 1 / 31 Lecture 2: Linear Regression 1 Supervised Learning: Regression A Probabilistic Interpretation to Linear Regression Feng Li (SDU) Linear Regression September 13, 2023 2 / 31 Supervised Learning Regression: Predict a continuous value Classification: Predict a discrete 2104 400 1600 330 2400 369 1416 232 3000 540 ... ... Feng Li (SDU) Linear Regression September 13, 2023 3 / 31 Supervised Learning (Contd.) Features: input variables, x; Target: output variable, y;

Regression Feng Li Shandong University fli@sdu.edu.cn September 20, 2023 Feng Li (SDU) Logistic Regression September 20, 2023 1 / 29 Lecture 3: Logistic Regression 1 Classification 2 Logistic Regression Regression 3 Newton’s Method 4 Multiclass Classification Feng Li (SDU) Logistic Regression September 20, 2023 2 / 29 Classification Classification problems Email: Spam / Not Spam? Online Transactions: Fraudulent tumor) 1 : “Positive Class” (e.g., malignant tumor) Feng Li (SDU) Logistic Regression September 20, 2023 3 / 29 Warm-Up What if applying linear regress to classification? Tumor Size Malignant? (Yes)

Feng Li Shandong University fli@sdu.edu.cn September 20, 2023 Feng Li (SDU) Regularization and Bayesian Statistics September 20, 2023 1 / 25 Lecture 4: Regularization and Bayesian Statistics 1 Overfitting Statistics September 20, 2023 2 / 25 Overfitting Problem y = θ0 + θ1x y = θ0 + θ1x + θ2x2 y = θ0 + θ1x + · · · + θ5x5 Feng Li (SDU) Regularization and Bayesian Statistics September 20, 2023 3 / 25 Overfitting generalize well to predict new data Feng Li (SDU) Regularization and Bayesian Statistics September 20, 2023 4 / 25 Addressing The Overfitting Problem Reduce the number of features Manually select which features

RN-08516-001_v23.07 | July 2023 PyTorch Release Notes PyTorch RN-08516-001_v23.07 | ii Table of Contents Chapter 1. PyTorch Overview..................................................... denial of service via SAMPLESPERPIXEL. ‣ The following CVEs were detected in OpenCV: CVE-2023-2618, CVE-2023-2617, which will be addressed in future releases. ‣ Performance regression up to 35% for FastPitch vulnerability. ‣ Known security vulnerabilities: ‣ CVE-2022-32212, CVE-2022-43548, CVE-2023-0286, CVE-2022-32223, CVE-2023-0286, CVE-2022-25881, CVE-2022-35255 for nodejs and openssl PyTorch RN-08516-001_v23

1 2023年05月 深度学习-序列模型 黄海广 副教授 2 03 长短期记忆(LSTM) 04 双向循环神经网络 本章目录 01 序列模型概述 02 循环神经网络(RNN) 05 深层循环神经网络 3 03 长短期记忆(LSTM) 04 双向循环神经网络 1.序列模型概述 01 序列模型概述 02

1 2023年04月 深度学习-优化算法 黄海广 副教授 2 01 小批量梯度下降 本章目录 02 优化算法 03 超参数调整和BatchNorm 04 Softmax 3 01 小批量梯度下降 02 优化算法 03 超参数调整和BatchNorm 04 Softmax 1.小批量梯度下降 4 小批量梯度下降 小批量梯度下降（Mini-Batch

1 2023年03月 深度学习-深层神经网络 黄海广 副教授 2 神经网络的概念 3 神经网络的概念 ? 1 = ?1 1 ?2 1 ?3 1 4 神经网络的概念 我们不将输入层 看作一个标准的 层。 ? 1 = ?1 1 ?2 1 ?3 1 5 ?1 1 ?2 1 ?3 1 ?4 1 = . . . ?1 ሾ1]?. . .

1 2023年03月 深度学习-神经网络的编程基础 黄海广 副教授 2 本章目录 01 二分类与逻辑回归 02 梯度下降 03 计算图 04 向量化 3 1.二分类与逻辑回归 02 梯度下降 01 二分类与逻辑回归 03 计算图 04 向量化 4 符号定义 ?：表示一个??维数据，为输入数 据，维度为(??, 1)；