computations used to define f, and finally to the output y • The model is associated with a directed acyclic graph describing how the functions are composed together • E.g., we use a chain to represent f(x) =

171 14.2 从一个后端切换到另一个后端 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 14.3 keras.json 详细配置 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 14.4 使用抽象 Keras 保存为 JSON json_string = model.to_json() # 保存为 YAML yaml_string = model.to_yaml() 生成的 JSON/YAML 文件是人类可读的，如果需要还可以手动编辑。 你可以从这些数据建立一个新的模型： # 从 JSON 重建模型： from keras.models import model_from_json model model = model_from_json(json_string) # 从 YAML 重建模型： from keras.models import model_from_yaml model = model_from_yaml(yaml_string) 3.3.6.3 只保存/加载模型的权重 如果您只需要 模型的权重，可以使用下面的代码以 HDF5 格式进行保存。 请注意，我们首先需要安装

see the configuration file is located at your home directory inside and go to .keras/keras.json. keras.json { "image_data_format": "channels_last", "epsilon": 1e-07, "floatx": "float32" steps: > cd home > mkdir .keras > vi keras.json Remember, you should specify .keras as its folder name and add the above configuration inside keras.json file. We can perform some pre-defined operations configuration from TensorFlow to Theano, just change the backend = theano in keras.json file. It is described below: keras.json { "image_data_format": "channels_last", "epsilon": 1e-07, "floatx":

似于社交网 络、通信网络、蛋白质分子结构等一系列的不规则空间拓扑结构的数据，它们往往显得力 不从心。2016 年，Thomas Kipf 等人基于前人在一阶近似的谱卷积算法上提出了图卷积网 络(Graph Convolution Network，GCN)模型。GCN 算法实现简单，从空间一阶邻居信息聚 合的角度也能直观地理解，在半监督任务上取得了不错效果。随后，一系列的网络模型相 继被提出，如 GAT、EdgeConv、DeepGCN 为了防止释放计算图资源，设置 retain_graph=True dy2_dy1 = autograd.grad(y2, [y1], retain_graph=True)[0] dy1_dw1 = autograd.grad(y1, [w1], retain_graph=True)[0] dy2_dw1 = autograd.grad(y2, [w1], retain_graph=True)[0] its, grad_outputs=torch.ones_like(d_interplote_logits), create_graph=True, retain_graph=True, )[0] # 计算每个样本的梯度的范数:[b, h, w, c] => [b, -1] grads = grads

(and advantages) • Preview of Numpy & PyTorch & Tensorflow Numpy Tensorflow PyTorch Computation Graph Advantages (continued) • Which one do you think is better? Advantages (continued) • Which one • Visualization Tools like • TensorboardX (monitor training) • PyTorchViz (visualise computation graph) • Various other functions • loss (MSE,CE etc..) • optimizers Prepare Input Data •Load data •Iterate to run on. Visualization • TensorboardX (visualise training) • PyTorchViz (visualise computation graph) https://github.com/lanpa/tensorboardX/ Visualization (continued) • PyTorchViz https://github

的流行 工具，而MNIST数据集的60000个手写数字的数据集被认为是巨大的。考虑到数据和计算的稀缺性，核方法 （kernel method）、决策树（decision tree）和图模型（graph models）等强大的统计工具（在经验上）证明 是更为优越的。与神经网络不同的是，这些算法不需要数周的训练，而且有很强的理论依据，可以提供可预 测的结果。 1.5 深度学习的发展 大约2 ）。 深度学习框架通过自动计算导数，即自动微分（automatic differentiation）来加快求导。实际中，根据设计 好的模型，系统会构建一个计算图（computational graph），来跟踪计算是哪些数据通过哪些操作组合起来 产生输出。自动微分使系统能够随后反向传播梯度。这里，反向传播（backpropagate）意味着跟踪整个计算 图，填充关于每个参数的偏导数。 40 妙的边界条件很重要，我们很可能是在研究 数学而非工程”，这个观点正好适用于这里。下面我们绘制ReLU函数的导数。 y.backward(torch.ones_like(x), retain_graph=True) d2l.plot(x.detach(), x.grad, 'x', 'grad of relu', figsize=(5, 2.5)) 使用ReLU的原因是，它求导表现得特别好：

分布式存储  分布式查询 功能完备:  GSL/负采样  主流图算法  异构图 (user/item/attribute)  动态图 标准化: Standard Libraries Graph-Learn: 分布式图算法库 标准化: Standard Solutions Continuous Optimization: Active learning Data Label Model [VariationalDropout] 通信优化 [GRPC++] 实时训练 [增量更新] 混合精度 [bf16] 工程优化: 千亿特征优化 模型蒸馏 AVX/SSE优化 Graph优化 [User Graph去重] 内存Allocate优化 ParallelStringOp [split/type conversion] Sequence Feature [side info] com/alibaba/EasyCV 6. EasyNLP: https://github.com/alibaba/EasyNLP 7. AliGraph: https://github.com/alibaba/graph-learn 8. DSW: https://help.aliyun.com/document_detail/194831.html 9. DLC: https://help.aliyun.c

values of θ0 and θ1 that you get, and plot the straight line fit from your algorithm on the same graph as your training data according to θ. The plotting commands will look something like this: hold on figure below. 0 10 20 30 40 50 Number of iterations 0 1 2 3 4 5 6 7 Cost J 1010 If your graph looks very different, especially if your value of J(θ) increases or even blows up, adjust your learning learning rates affect convergence, it’s helpful to plot J for several learning rates on the same graph. In Matlab/Octave, this can be done by performing gradient descent multiple times with a hold on command

Adjustment） • 变量数目非常庞大 • 内存空间需求大 • 计算耗时 • 迭代的局部集束调整 • 大误差难以均匀扩散到整个序列 • 极易陷入局部最优 • 姿态图优化（Pose Graph Optimization） • 只优化相机之间的相对姿态，三维点都消元掉； • 是集束调整的一个近似，不是最优解。 基于自适应分段的集束调整 • 将长序列分成若干段短序列； • 每个短 Recognition Pose Graph Optimization + Traditional BA Street序列结果比较 ENFT-SLAM ORB-SLAM Non-consecutive Track Matching Segment-based BA Bag-of-words Place Recognition Pose Graph Optimization + Traditional

interesting aspects of the data Examples: Discovering clusters Discovering latent factor Discovering graph structure Matrix completion Feng Li (SDU) Overview September 6, 2023 28 / 57 Unsupervised Learning: Discovering Graph Structures Sometimes we measure a set of correlated variables, and we would like to discover which ones are most correlated with which others This can be represented by a graph, in which