提升明显。目前，该成果已经被集成进 OpenBLAS 和 PyTorch 中。 • 本次演讲，将向大家介绍我们在倚天 710 ARM 芯片上开展的 Python + AI 优化工作，以及在 ARM 云平台上部署 Python + AI 任务的最佳实践。 深度学习 • 广泛使用的深度学习框架 • TensorFlow、PyTorch • 结合硬件（ARM 服务端芯片） • 倚天 710 Representations at a Large Scale, Yangqing Jia Convolution • ResNet-50 • PyTorch Profiler Transformers • GPT-2 • PyTorch Profiler GEMM • General Matrix Multiplication: C = αAB + βC • MNK 级别的时间复杂度 ARM Compute Library • OpenBLAS • TensorFlow • oneDNN + ACL • DNNL_DEFAULT_FPMATH_MODE=BF16 • PyTorch • OpenBLAS • oneDNN + ACL • torch.set_float32_fast_math_mode("BF16") 深度学习推理加速 • BF16 gemm 实现

优化算法——知识蒸馏
 3. 优化实现——jit/TensorRT PyTorch jit • 原理理介绍 • 转化为graph截图 TensorRT • NVIDIA推出的inference引擎 • ⾃自家硬件使⽤用到极致 • 与CPU⽐比较：20x faster • 正确的问法：
 与TF/PyTorch⽐比较如何？ BERT runtime • 使⽤用SQuAD任务 size分别取1和32 • 计时代码只包含GPU时间，排除掉前后处理理时间，另包含数据在 CPU和GPU之间copy的时间 异步执⾏ • CPU与GPU异构，所以可以异步 • PyTorch也是异步执⾏行行，所以没有带来提升 cuda优化 • 更更⾼高效的kernel函数实现，替代默认导出的算⼦子 知识蒸馏 • Teacher Student 学的更更快 • Huggingface

ow和Keras 2 启动 iPython 终端 运行 TensorFlow 程序 已切换环境 自由切换环境 - PyTorch 具有 CUDA 10 和 MKL-DNN 的 Python 3 上的 PyTorch 启动 iPython 终端 运行 PyTorch 程序 已切换环境 AWS机器学习、人工智能服务 机器学习框架和基础设施 框架 接口 基础设施 P3 & P3dn

开发工具 à Azure Notebook Azure Machine Learning • 拥有不同运算性能的机器 • 降低成本，按需付费 • 支持不同的开源框架：TenserFlow、PyTorch、MXNet 等 Azure Notebook Jupyter Notebook on Azure • 免费 • 全托管 • 无需安装 • 无需配置 Workflow 需要准备哪些东西？ 上传训练数据 5. 准备 training script 6. 把 training 任务提交到 Azure Machine Learning 1. 创建 experiment 2. 创建 PyTorch estimator 3. 提交 training 任务 Deploy the model as a web service 1. 注册 model 2. 部署 model 注册 model

wildcard. For example, *py* returns all packages that have "py" in their names, such as "numpy", "pytorch", "python", etc. version (Optional) Package version. May include the * wildcard or a version range(s) objects. Internally, conda will translate your command line requests, like numpy>=1.19, python=3.* or pytorch=1.8.*=*cuda*, into in- stances of this class. This query language has its own syntax and rules, detailed here. The most important fields of a MatchSpec object are: – name: the name of the package (e.g. pytorch); always expected. – version: the version constraints (e.g. 1.8.*); can be empty but if build is

SECTION TITLE SECTION TITLE 我们能不能直接用 Rust • 训练 • 推理 • AI 应用相关的工具 • WASI-NN spec • WasmEdge 已经支持 Pytorch 、 TensorFlow Lite • WASI-NN 2.0 比如 Langchain Rewrite it in Rust • 笨重 • 资源占用多 • 大部分时间是在等待 •

Practicing Sparrow Ideas  Today, most of the main stream AI frameworks such like Tensorflow, PyTorch and MXNet embrace Python+C++ for their software layers design.  While Rust is coming

#生成默认配置文件，在~/.condarc # conda search pytorch #从所有channels中查找软件 # conda install anaconda/cloud/conda-forge::pytorch==1.13.1 #从指定 channel中安装包 ★conda设置代理

for the user to be able to leverage your hot path capabilities written in C++ ○ Numpy / Pandas / PyTorch are nice DSLs that provide “hot path” C++ functionsWhat Functions to expose in the API ● Init some

wildcard. For example, *py* returns all packages that have "py" in their names, such as "numpy", "pytorch", "python", etc. version (Optional) Package version. May include the * wildcard or a version range(s) objects. Internally, conda will translate your command line requests, like numpy>=1.19, python=3.* or pytorch=1.8.*=*cuda*, into in- stances of this class. This query language has its own syntax and rules, detailed here. The most important fields of a MatchSpec object are: – name: the name of the package (e.g. pytorch); always expected. – version: the version constraints (e.g. 1.8.*); can be empty but if build is