28, 2021 7 / 82 Margin (Contd.) Geometric margin γ(i) = y(i) �� ω ∥ω∥ �T x(i) + b ∥ω∥ � Scaling (ω, b) does not change γ(i) !" !" !"# + % = 0 ! !(#) !(#) = & # ' ⋅ ) ' ⋅ ) * +(#) + ' ⋅ December 28, 2021 8 / 82 Margin (Contd.) Geometric margin γ(i) = y(i) � (ω/∥ω∥)T x(i) + b/∥ω∥ � Scaling (ω, b) does not change γ(i) With respect to the whole training set, the margin is written as γ ! !"# + % = −( ! Feng Li (SDU) SVM December 28, 2021 12 / 82 Maximizing The Margin (Contd.) Scaling (ω, b) such that mini{y(i)(ωTx(i) + b)} = 1, γ = min i � y(i) �� ω ∥ω∥ �T x(i) + b ∥ω∥ ��

the symbol’s width at all times. 3 Brand Guidelines PyTorch Symbol Sizing When sizing or scaling the symbol, never exceed a minimum of 24 pixels for screen or 10mm for print. This ensures here. Choose the appropriate lockup depending on your specific application. When sizing or scaling the wordmark or lockups, ensure a legible size at all times. It should not appear subordinate

EfficientNet的设计思路来源于模型优化的 两个主要思想： 神经网络结构搜索（Neural Architecture Search，NAS）和模型融合。 其主要贡献在于开创性地提出了通过均匀缩 放（Accurate Scaling）来调整网络深度 、宽度和分辨率的方法。 23 3.其它现代网络 EfficientNet 24 01 经典网络 4.卷积神经网络使用技巧 02 深度残差网络 03 for Mobile Vision Applications (Andrew G. Howard et al., 2017) • EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks (Mingxing Tan and Quoc V. Le, 2019) 32 谢 谢！

the model. width_multiplier=1.0, # Factor for scaling the network size. params={}, # Additional hyper-params. ): # Use the width_multiplier for scaling the network, the larger the value, # the larger

Batch Norm 主讲人：龙良曲 Intuitive explanation Intuitive explanation Feature scaling ▪ Image Normalization ▪ Batch Normalization Batch Norm https://medium.com/syncedreview/facebook-ai-proposes-group-normalization-

212000 1494 3 242500 训练数据： 数据分布： 多变量房价预测问题：特征归一化 房屋面积和卧室数量这两个变量（特征）在数值上差了1000倍。在这种情况下，通常先进 行特征缩放（Scaling），再开始训练，可以加速模型收敛。 平均值 标准差 多变量房价预测问题 面积（平方英尺） 卧室数量（个） 价格（美元） 0.13001 -0.22368 0.475747 -0.50419

Information Processing Systems (NIPS), MIT Press, Cambridge, MA, 841-848, 2002. [6] Kohavi R.,Scaling up the accuracy of naïve Bayes classifiers: A decision-tree hybrid[C]// Proceedings of the 2nd

M. BISHOP. Pattern Recognition and Machine Learning[M]. New York: Springer,2006. [6] Kohavi R.,Scaling up the accuracy of naïve Bayes classifiers: A decision-tree hybrid[C]// Proceedings of the 2nd

a toolbox to make our models pareto-optimal has the following benefits: Sustainable Server-Side Scaling Training and deploying large deep learning models is costly. While training is a one-time cost (or

maximizing γ (see Eq. (6)) as follows max γ,ω,b γ s.t. y(i)(ωT x(i) + b) ≥ γ∥ω∥, ∀i Note that scaling ω and b (e.g., by multiplying both ω and b by the same constant) does not change the hyperplane.