INT8 & FP32 AiiOS ! 驱动万物智能 Alios TVM @ ARM CPU INT8 * Cache 芍四 Data FO Data FOData … QNNPACK Convolution 。，NHWC layout Cach， 浆百 FeU Cach- 区下 。， im2col + pack -35 1 129 中131 124有23152136 2 1.14 am omo oo Convolution Workload Performance AiOS 1驱动万物智能 Alios TVM @ ARM CPU INT8 Depthwise Convolution 。， NHWC layout 。 Using TVM schedule primitive completely 130 1.35 1.33. 1.15 116 111 09工08 工区 0.77 0.77 | | | Depthwise Convolution Workload Performance Alios TVM @ ARM CPU INT8 Performance Comparison @ rasp 3b+ AARCH64

batch size = 1 • 1.7x speedup on Inception asymmetric quantized model • Mobilenet requires depthwise convolution VNNI schedule • Symmetric model improves the speedup to 2.8x© 2019, Amazon Web Services,

����������� �� �������������������� 1. OpenVino a black box, can not deploy our network(with depthwise conv2d, ) 2. TVM can not only deploy our network, but also get a good performance gain by autotuning

Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Trademark Effects of Convolution operators using TVM AWS DeepLens Acer aiSage NVIDIA Jetson Nano Speedup 0 2 4 6 8 SSD_MobileNet1

compute. @autotvm.register_ topi_schedule(schedule_conv2d_nchw，pvr, [direct]) convolution def schedule_conv2d_nchw_pvr(cfg, outs):

DPU Processor (xDNNv3) >> 3 ˃ Configurable Overlay Processor ˃ DNN Specific Instruction Set Convolution, Max Pool etc. ˃ Any Network, Any Image Size ˃ High Frequency & High Compute Efficiency ˃