ARM 芯片的 Python + AI 算力优化 主讲人： 朱宏林 – 阿里云程序语言与编译器团队 简介 • 当今开发者们大量使用 Python 语言编写的 AI 程序。过去这些程序总跑在 GPU 或者 x86 架构的 CPU 上。然而综合考虑到功耗、成本、性能等因素，云厂商们开始建设 ARM 架构的服务平台，如 何整合 Python + AI 的相关软件并使其在该平台上发挥最高的性能成为了工程师们关注的焦点。 矩阵乘法是深度学习计算的重要组成部分，我们利用 ARM 架构新提供的矩阵扩展对 bf16 类型的 矩阵乘法计算进行优化，该优化将纯矩阵乘法的运算速度提升 3 倍以上，对深度学习推理任务性能 提升明显。目前，该成果已经被集成进 OpenBLAS 和 PyTorch 中。 • 本次演讲，将向大家介绍我们在倚天 710 ARM 芯片上开展的 Python + AI 优化工作，以及在 ARM 云平台上部署 Python Python + AI 任务的最佳实践。 深度学习 • 广泛使用的深度学习框架 • TensorFlow、PyTorch • 结合硬件（ARM 服务端芯片） • 倚天 710 • AWS graviton • 矩阵乘法 • 为什么矩阵乘法是深度学习的核心 • Conv、Linear、Transformers 来源: Why GEMM is at the heart of deep learning

compatible. For example, try to obfuscate the scripts with --platform linux.arm.0 in Windows: pyarmor obfuscate --platform linux.arm.0 foo.py Because the default platform is full features windows.x86_64.7 reboot with platform windows.x86_64.0, then obfuscate the script for this low feature platform linux.arm.0. It also could be set the enviornment variable PYARMOR_PLATFORM to same feature platform as target ws.x86_64.0 pyarmor obfuscate --platform linux.arm.0 foo.py # In Windows set PYARMOR_PLATFORM=windows.x86_64.0 pyarmor obfuscate --platform linux.arm.0 foo.py set PYARMOR_PLATFORM= Running Obfuscated

compatible. For example, try to obfuscate the scripts with --platform linux.arm.0 in Windows: pyarmor obfuscate --platform linux.arm.0 foo.py Because the default platform is full features windows.x86_64.7 reboot with platform windows.x86_64.0, then obfuscate the script for this low feature platform linux.arm.0. It also could be set the enviornment variable PYARMOR_PLATFORM to same feature platform as target ws.x86_64.0 pyarmor obfuscate --platform linux.arm.0 foo.py # In Windows set PYARMOR_PLATFORM=windows.x86_64.0 pyarmor obfuscate --platform linux.arm.0 foo.py set PYARMOR_PLATFORM= Running Obfuscated

compatible. For example, try to obfuscate the scripts with --platform linux.arm.0 in Windows: pyarmor obfuscate --platform linux.arm.0 foo.py Because the default platform is full features windows.x86_64.7 reboot with platform windows.x86_64.0, then obfuscate the script for this low feature platform linux.arm.0. It also could be set the enviornment variable PYARMOR_PLATFORM to same feature platform as target ows.x86_64.0 pyarmor obfuscate --platform linux.arm.0 foo.py # In Windows set PYARMOR_PLATFORM=windows.x86_64.0 pyarmor obfuscate --platform linux.arm.0 foo.py set PYARMOR_PLATFORM= 3.4.2 Running Obfuscated

compatible. For example, try to obfuscate the scripts with --platform linux.arm.0 in Windows: pyarmor obfuscate --platform linux.arm.0 foo.py Because the default platform is full features windows.x86_64.7 reboot with platform windows.x86_64.0, then obfuscate the script for this low feature platform linux.arm.0. It also could be set the enviornment variable PYARMOR_PLATFORM to same feature platform as target ows.x86_64.0 pyarmor obfuscate --platform linux.arm.0 foo.py # In Windows set PYARMOR_PLATFORM=windows.x86_64.0 pyarmor obfuscate --platform linux.arm.0 foo.py set PYARMOR_PLATFORM= 3.5.2 Running Obfuscated

x86_64 • linux.arm • linux.armv6 • linux.armv7 • linux.aarch32 • linux.aarch64 • android.aarch64 • android.armv7 (New in 5.9.3) • uclibc.armv7 (New in 5.9.4) • linux.ppc64 • darwin.arm64 • freebsd freebsd.x86_64 • alpine.x86_64 • alpine.arm • poky.x86 64 Chapter 10. Support Platfroms PyArmor Documentation, Release 5.9.0 10.2 Platform Tables Table 1: Table-1. Prebuilt Libraries Distributed with by VS2015 vs2015.x86_64 Windows x64 _pytransform.dll Built by VS2015 linxu.arm Linux armv5 _pytransform.so 32-bit Armv5 (arm926ej-s) linxu.armv6Linux armv6 _pytransform.so 32-bit Armv6 (-marm -march=armv6

Windows x86 64-bit macOS Miniconda installer for: macOS with Apple Silicon 64-bit macOS with Intel CPU 64-bit Linux Miniconda installer for: Linux x86 64-bit Homebrew Run the following Homebrew command: features present on the system that cannot be managed directly by conda, like system driver versions or CPU features. Virtual packages are not real packages and not displayed by conda list. Instead conda runs linux-armv6l, linux-armv7l, linux-ppc64, linux-ppc64le, linux-riscv64, linux-s390x, osx-64, osx-arm64, win-32, win-64, win-arm64, zos- z Use packages built for this platform. The new environment will be configured

features present on the system that cannot be managed directly by conda, like system driver versions or CPU features. Virtual packages are not real packages and not displayed by conda list. Instead conda runs Contributors welcome conda, Release 24.3.1.dev2 linux-riscv64, linux-s390x, osx-64, osx-arm64, win-32, win-64, win-arm64, zos- z Use packages built for this platform. The new environment will be configured linux-armv6l, linux-armv7l, linux-ppc64, linux-ppc64le, linux-riscv64, linux-s390x, osx-64, osx-arm64, win-32, win-64, win-arm64, zos- z Use packages built for this platform. The new environment will be configured

features present on the system that cannot be managed directly by conda, like system driver versions or CPU features. Virtual packages are not real packages and not displayed by conda list. Instead conda runs Contributors welcome conda, Release 24.3.1.dev7 linux-riscv64, linux-s390x, osx-64, osx-arm64, win-32, win-64, win-arm64, zos- z Use packages built for this platform. The new environment will be configured linux-armv6l, linux-armv7l, linux-ppc64, linux-ppc64le, linux-riscv64, linux-s390x, osx-64, osx-arm64, win-32, win-64, win-arm64, zos- z Use packages built for this platform. The new environment will be configured

features present on the system that cannot be managed directly by conda, like system driver versions or CPU features. Virtual packages are not real packages and not displayed by conda list. Instead conda runs linux-armv6l, linux-armv7l, linux-ppc64, linux-ppc64le, linux-riscv64, linux-s390x, osx-64, osx-arm64, win-32, win-64, win-arm64, zos- z Use packages built for this platform. The new environment will be configured linux-armv6l, linux-armv7l, linux-ppc64, linux-ppc64le, linux-riscv64, linux-s390x, osx-64, osx-arm64, win-32, win-64, win-arm64, zos- z Use packages built for this platform. The new environment will be configured