INT8 & FP32 AiiOS ! 驱动万物智能 Alios TVM @ ARM CPU INT8 * Cache 芍四 Data FO Data FOData … QNNPACK Convolution 。，NHWC layout Cach， 浆百 FeU Cach- 区下 。， im2col + pack -35 1 129 中131 124有23152136 2 1.14 am omo oo Convolution Workload Performance AiOS 1驱动万物智能 Alios TVM @ ARM CPU INT8 Depthwise Convolution 。， NHWC layout 。 Using TVM schedule primitive completely 130 1.35 1.33. 1.15 116 111 09工08 工区 0.77 0.77 | | | Depthwise Convolution Workload Performance Alios TVM @ ARM CPU INT8 Performance Comparison @ rasp 3b+ AARCH64

batch size = 1 • 1.7x speedup on Inception asymmetric quantized model • Mobilenet requires depthwise convolution VNNI schedule • Symmetric model improves the speedup to 2.8x© 2019, Amazon Web Services,

����������� �� �������������������� 1. OpenVino a black box, can not deploy our network(with depthwise conv2d, ) 2. TVM can not only deploy our network, but also get a good performance gain by autotuning

fern::Interval (y, out.y_start, out.y_start + out.y_len, l fern::Compute( fern::Producer(Convolution Input Filters Convolution 65 }) )) template void gemm(Matrix A,Matrix B,Matrix fern::Interval void conv(image input, image filter, int StrideArg, image out);Convolution Input Filters Convolution 66 }) )) template void gemm(Matrix A,Matrix B,Matrix fern::Interval void conv(image input, image filter, int StrideArg, image out);Convolution Input Filters Convolution 67 }) )) template void gemm(Matrix A,Matrix B,Matrix fern::Interval

problems • Intra-register sorting • Fast linear median-of-seven filter • Fast small-kernel convolution • Faster (?) UTF-8 to UTF-32 conversion (with AVX2) • No heavy code, but lots of pictures • Small-Kernel Convolution 3 CppCon 2020 - Adventures in SIMD ThinkingCopyright © 2020 Bob Steagall K E W B C O M P U T I N G Convolution • f is a signal • g is a kernel • Output f*g is the convolution • Every CppCon 2020 - Adventures in SIMD Thinking 4Copyright © 2020 Bob Steagall K E W B C O M P U T I N G Convolution CppCon 2020 - Adventures in SIMD Thinking 5 S = s0 s1 s2 s3 s4 s5 s6

Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Effects of Convolution operators using TVM AWS DeepLens Acer aiSage NVIDIA Jetson Nano Speedup 0 2 4 6 8 SSD_MobileNet1

compute. @autotvm.register_ topi_schedule(schedule_conv2d_nchw，pvr, [direct]) convolution def schedule_conv2d_nchw_pvr(cfg, outs):

DPU Processor (xDNNv3) >> 3 ˃ Configurable Overlay Processor ˃ DNN Specific Instruction Set Convolution, Max Pool etc. ˃ Any Network, Any Image Size ˃ High Frequency & High Compute Efficiency ˃

problems • Intra-register sorting • Fast linear median-of-seven filter • Fast small-kernel convolution • Faster (?) UTF-8 to UTF-32 conversion (with AVX2) • No heavy code, but lots of pictures •

batched(starmap(math.sumprod, product(m1, transpose(m2))), n) def convolve(signal, kernel): """Discrete linear convolution of two iterables. Equivalent to polynomial multiplication. Convolutions are mathematically commutative; consumed before the calculations begin. Article: https://betterexplained.com/articles/intuitive-convolution/ Video: https://www.youtube.com/watch?v=KuXjwB4LzSA """ # convolve([1, -1, -20], [1, -3]) → 1 Notwithstanding the foregoing, with regard to derivative works based on Python 1.6.1 that incorporate non-separable material that was previously distributed under the GNU General Public License (GPL), the law of