+ ? ≥ 1 Negative class: ?!? + ? ≤ −1 ? = 1 ? Figure 2: Hard-margin SVM. aim of the above optimization problem is to find a hyperplane (parameterized by ω and b) with margin γ = 1/∥ω∥ maximized, while training set. 2.2 Preliminary Knowledge of Convex Optimization 2.2.1 Optimization Problems and Lagrangian Duality We now consider the following optimization problem min ω f(ω) (9) s.t. gi(ω) ≤ 0, i = 1 gk(ω) and the equality constraints h1(ω), · · · , hl(ω). We construct the Lagrangian of the above optimization problem as L(ω, α, β ) = f(ω) + k � i=1 αigi(ω) + l � j=1 β jhj(ω) (12) In fact, L(ω, α

Outline 1 SVM: A Primal Form 2 Convex Optimization Review 3 The Lagrange Dual Problem of SVM 4 SVM with Kernels 5 Soft-Margin SVM 6 Sequential Minimal Optimization (SMO) Algorithm Feng Li (SDU) SVM December 28, 2021 15 / 82 Convex Optimization Review Optimization Problem Lagrangian Duality KKT Conditions Convex Optimization S. Boyd and L. Vandenberghe, 2004. Convex Optimization. Cambridge university press press. Feng Li (SDU) SVM December 28, 2021 16 / 82 Optimization Problems Considering the following optimization problem min ω f (ω) s.t. gi(ω) ≤ 0, i = 1, · · · , k hj(ω) = 0, j = 1, · · · , l with

in a Knative based platform ● Performance bottleneck analysis and tuning ○ Istio scalability optimization during Knative Service provisioning ○ Unleash maximum scalability by fully leveraging Istio features MEM Knative Version Knative 0.16, 0.17, 0.18 Istio Version 1.5, 1.6, 1.7 Istio scalability optimization during Knative Service provisioning • Benchmark: Kperf (https://github.com/knative-sandbox/kperf) resolved this issue. o Istiod MEM bumped with large numbers of Knative Services (#25532) Mem usage optimization of pilot resolved this issue. • Tune CPU/MEM to ensure enough capacity Leveraged Metrics to

this using the earlier example for choosing quantization and/or clustering techniques for model optimization. We have a search space which has two boolean valued parameters: quantization and clustering hyperparameters. Some of the commonly tuned hyperparameters are the learning rate and the momentum of the optimization algorithm and the training batch size. Other aspects of the training pipeline like data augmentation may influence each other. Hence, we need a sophisticated approach to tune them. Hyperparameter Optimization (HPO) is the process of choosing values for hyperparameters that lead to an optimal model. HPO

translation alternatives • Runtime optimizations • load management, scheduling, state management • Optimization semantics, correctness, profitability Topics covered in this lecture ??? Vasiliki Kalavri | different strategies? • before execution or during runtime Query optimization (I) ??? Vasiliki Kalavri | Boston University 2020 10 Optimization strategies • enumerate equivalent execution plans • minimize minimize monetary costs (if running in the cloud) Query optimization (II) ??? Vasiliki Kalavri | Boston University 2020 Cost-based optimization 11 Parsed program representation Optimizer statistics

Kyuubi and nothing more. The Kyuubi server-side or the corresponding engines could do most of the optimization. On the other hand, we don’t wholly restrict end-users to special handling of specific cases to Execution (AQE) in Kyuubi The Basics of AQE Spark Adaptive Query Execution (AQE) is a query re-optimization that occurs during query execution. In terms of technical architecture, the AQE is a framework storage, actually without performing the shuffle across the network. The local shuffle reader optimization consists of avoiding shuffle when the SortMerge Join transforms to BroadcastHash Join after applying

Kyuubi and nothing more. The Kyuubi server-side or the corresponding engines could do most of the optimization. On the other hand, we don’t wholly restrict end-users to special handling of specific cases to Execution (AQE) in Kyuubi The Basics of AQE Spark Adaptive Query Execution (AQE) is a query re-optimization that occurs during query execution. In terms of technical architecture, the AQE is a framework storage, actually without performing the shuffle across the network. The local shuffle reader optimization consists of avoiding shuffle when the SortMerge Join transforms to BroadcastHash Join after applying

Pytorch ● Dataset & Dataloader ● Tensors ● torch.nn: Models, Loss Functions ● torch.optim: Optimization ● Save/load models Prerequisites ● We assume you are already familiar with… 1. Python3 ■ deep neural networks Training Neural Networks Training Define Neural Network Loss Function Optimization Algorithm More info about the training process in last year's lecture video. Training & Testing calculation. Training & Testing Neural Networks – in Pytorch Define Neural Network Loss Function Optimization Algorithm Training Validation Testing Step 2. torch.nn.Module Load Data torch.nn – Network

Kyuubi and nothing more. The Kyuubi server-side or the corresponding engines could do most of the optimization. On the other hand, we don’t wholly restrict end-users to special handling of specific cases to (AQE) in Kyuubi 2.2.1. The Basics of AQE Spark Adaptive Query Execution (AQE) is a query re-optimization that occurs during query execution. In terms of technical architecture, the AQE is a framework storage, actually without performing the shuffle across the network. The local shuffle reader optimization consists of avoiding shuffle when the SortMerge Join transforms to BroadcastHash Join after applying

Kyuubi and nothing more. The Kyuubi server-side or the corresponding engines could do most of the optimization. On the other hand, we don’t wholly restrict end-users to special handling of specific cases to Execution (AQE) in Kyuubi 2.1. The Basics of AQE Spark Adaptive Query Execution (AQE) is a query re-optimization that occurs during query execution. In terms of technical architecture, the AQE is a framework storage, actually without performing the shuffle across the network. The local shuffle reader optimization consists of avoiding shuffle when the SortMerge Join transforms to BroadcastHash Join after applying