Lecture 4: Regularization and Bayesian Statistics Feng Li Shandong University fli@sdu.edu.cn September 20, 2023 Feng Li (SDU) Regularization and Bayesian Statistics September 20, 2023 1 / 25 Lecture Regularization and Bayesian Statistics 1 Overfitting Problem 2 Regularized Linear Regression 3 Regularized Logistic Regression 4 MLE and MAP Feng Li (SDU) Regularization and Bayesian Statistics September 20, 2023 θ1x y = θ0 + θ1x + θ2x2 y = θ0 + θ1x + · · · + θ5x5 Feng Li (SDU) Regularization and Bayesian Statistics September 20, 2023 3 / 25 Overfitting Problem (Contd.) Underfitting, or high bias, is when the

"Non-stochastic best arm identification and hyperparameter optimization." Artificial intelligence and statistics. PMLR, 2016. 1 27, 3 9, 9 3, 27 2, 81 2 9, 9 3, 27 1, 81 3 3, 27 1, 81 4 1, 81 Table 7-1: CHILD_PARAMS = dict( epochs=6, batch_size=128, learning_rate=0.001, train_ds=train_ds, val_ds=val_ds, rolling_accuracies_window=20, max_branch_length=2, blocks=5, cells=2, initial_width=1, initial_channels=4 self.vds = CHILD_PARAMS['val_ds'].batch(256) self.past_accuracies = deque( maxlen=CHILD_PARAMS['rolling_accuracies_window'] ) self.past_accuracies.append(DATASET_PARAMS['baseline_accuracy']) self.layers

度，并且窗口的长度始终为10个单位长度， 直至移动到末端。 由此可知，通过滑动窗口统计的指标会更加 平稳一些，数据上下浮动的范围会比较小。 57 数据统计—滑动窗口 Pandas中提供了一个窗口方法rolling()。 rolling(window, min_periods=None, center=False, win_ty pe=None, on=None, axis=0, closed=None) ➢ window

View of Boosting: Discussion[J]. Annals of Statistics, 2000, 28(2):393-400. [6] FRIEDMAN J H . Stochastic gradient boosting[J]. Computational Statistics & Data Analysis, 2002, 38. 49 参考文献 [7] FRIEDMAN FRIEDMAN J H. Greedy function approximation: A gradient boosting machine[J]. Annals of statistics, 2001: 1189–1232. [8] MACQUEEN J, OTHERS. Some methods for classification and analysis of multivariate multivariate observations[C]//Proceedings of the fifth Berkeley symposium on mathematical statistics and probability. Oakland, CA, USA, 1(14): 281–297. [9] CHEN T, GUESTRIN C. XGBoost:A Scalable Tree Boosting

or split https://blog.openai.com/generative-models/ ▪ Cat ▪ Stack ▪ Split ▪ Chunk cat ▪ Statistics about scores ▪ [class1-4, students, scores] ▪ [class5-9, students, scores] Along distinct dim/axis

统计属性 主讲人：龙良曲 statistics https://blog.openai.com/generative-models/ ▪ norm ▪ mean sum ▪ prod ▪ max, min, argmin, argmax ▪ kthvalue, topk norm ▪ v.s. normalize ,e.g. batch_norm ▪ matrix norm v

Overview September 6, 2023 5 / 57 Prerequisite Courses Linear algebra Calculus Probability and Statistics Information theory Convex Optimization Feng Li (SDU) Overview September 6, 2023 6 / 57 Remarks

networks." Proceedings of the fourteenth international conference on artificial intelligence and statistics. JMLR Workshop and Conference Proceedings, 2011. Figure 1-2: Growth of parameters in Computer

(x − µ)2 � where µ is the mean and σ2 is the variance Gaussian distributions are important in statistics and are often used in the natural and social science to represent real-valued random variables

clusters (0.0002) is lower than that using quantization with 5 bits (0.0003). # Compute various statistics related to size when using quantization / # clustering. def get_quantized_size_bytes(num_elements