Confidentia Presenter: Zhi Chen, Cody Yu Amazon SageMaker Neo, Deep Engine Science Bring Your Own Codegen to TVM AWS AI© 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Considering testing.mobilenet.get_workload(batch_size=1) 3. Partition and build the network with an external codegen mod = relay.build_extern(mod, “dnnl”) 4. Run the inference exe = relay.create_executor(“vm”, mod=mod reserved. System Overview Relay IR Graph Annotation with Your Annotator Graph Partitioning Your Codegen LLVM, CUDA, Metal, VTA Serialized Subgraph Library Relay Runtime (VM, Graph Runtime, Interpreter)

compile=16 set_property(TARGET atarget PROPERTY JOB_POOL_COMPILE compile) 1 2 link=1 3 codegen=16) 4 5 6 7 8 set_property(TARGET atarget 9 PROPERTY JOB_POOL_LINK link) 10 11 add_ add_custom_target(protocgen 12 COMMAND protoc --cpp_out=./out server.proto 13 JOB_POOL codegen 14 SOURCES t ) 15Will it CMake? ... and linking ... set_property(GLOBAL PROPERTY JOB_POOLS link=1 set_property(TARGET set_property(TARGET atarget PROPERTY JOB_POOL_LINK link) 1 compile=16 2 3 codegen=16) 4 5 set_property(TARGET atarget 6 PROPERTY JOB_POOL_COMPILE compile) 7 8 9 10 11 add_custom_target(protocgen

bar_vec, ret); }) .consume ([&t](auto ret) { return getSizeType(t.foo_str, ret); }); } CodeGen: class & struct struct Bar { std:string str; }; struct Foo { int a; int ret); }); } return tail; } template class std::vector {...}; CodeGen: containers/* Generated from Debug Info */ static types::st::Unit getSizeType( const Bar& { return returnArg .consume ([&t](auto ret) { return getSizeType(t.str, ret); }); } CodeGen: class & struct struct Bar { std:string str; }; struct Foo { int a; int

Moonshot: rust-analyzer support (e.g. “▶ Run Test | Debug”). Language Library Tooling Tooling Codegen quality: minimal source code example 1 struct Example(Option); impl Drop for Example { self.0.take(); } } pub fn example() -> u32 { Example(Some(10u32)).0.take().unwrap() } Codegen quality: output example::example: pushq %rbx subq $16, %rsp movabsq $42949672961 movl $10, %eax retq When unwrap_unchecked is used instead. Tooling Tooling Codegen quality: example 2, minimal source code use std::ptr::read_volatile; pub unsafe fn test1(ptr: *const

as our C locks) to prevent recursion when doing certain operations (incremental package loading, codegen, etc.). The combination of a lock and this flag can be used to make finalizers safe. 2. A second Print precompile statements for methods compiled during execution or save to a path --image-codegen Force generate code in imaging mode --permalloc- pkgimg={yes|no*} Copy the data section of package initializes the global jl_root_task struct; and sets jl_current_task to the root task. jl_init_codegen() initializes the LLVM library. jl_init_serializer() initializes 8-bit serialization tags for builtin

as our C locks) to prevent recursion when doing certain operations (incremental package loading, codegen, etc.). The combination of a lock and this flag can be used to make finalizers safe. 2. A second Print precompile statements for methods compiled during execution or save to a path --image-codegen Force generate code in imaging mode --permalloc- pkgimg={yes|no*} Copy the data section of package initializes the global jl_root_task struct; and sets jl_current_task to the root task. jl_init_codegen() initializes the LLVM library. jl_init_serializer() initializes 8-bit serialization tags for builtin

as our C locks) to prevent recursion when doing certain operations (incremental package loading, codegen, etc.). The combination of a lock and this flag can be used to make finalizers safe. 2. A second Print precompile statements for methods compiled during execution or save to a path --image-codegen Force generate code in imaging mode --permalloc- pkgimg={yes|no*} Copy the data section of package initializes the global jl_root_task struct; and sets jl_current_task to the root task. jl_init_codegen() initializes the LLVM library. jl_init_serializer() initializes 8-bit serialization tags for builtin

as our C locks) to prevent recursion when doing certain operations (incremental package loading, codegen, etc.). The combination of a lock and this flag can be used to make finalizers safe. 2. A second Print precompile statements for methods compiled during execution or save to a path --image-codegen Force generate code in imaging mode --permalloc- pkgimg={yes|no*} Copy the data section of package initializes the global jl_root_task struct; and sets jl_current_task to the root task. jl_init_codegen() initializes the LLVM library. jl_init_serializer() initializes 8-bit serialization tags for builtin

as our C locks) to prevent recursion when doing certain operations (incremental package loading, codegen, etc.). The combination of a lock and this flag can be used to make finalizers safe. 2. A second Print precompile statements for methods compiled during execution or save to a path --image-codegen Force generate code in imaging mode --permalloc- pkgimg={yes|no*} Copy the data section of package initializes the global jl_root_task struct; and sets jl_current_task to the root task. jl_init_codegen() initializes the LLVM library. jl_init_serializer() initializes 8-bit serialization tags for builtin

as our C locks) to prevent recursion when doing certain operations (incremental package loading, codegen, etc.). The combination of a lock and this flag can be used to make finalizers safe. 2. A second Print precompile statements for methods compiled during execution or save to a path --image-codegen Force generate code in imaging mode --permalloc- pkgimg={yes|no*} Copy the data section of package initializes the global jl_root_task struct; and sets jl_current_task to the root task. jl_init_codegen() initializes the LLVM library. jl_init_serializer() initializes 8-bit serialization tags for builtin