infrastructure growth. As more developers build AI-native apps, they also create tools, wrappers and libraries that make it easier for others to follow. New front-end frameworks, embedding pipelines, model signals. Multimodal AI models are the result. They embed text, pictures, sound, and video into a shared representation and generate outputs in any of those formats. A single query can reference a paragraph have more benefits than drawbacks – up from 78% in 2022. In contrast, only 39% of USA respondents shared that view, with little change over the two-year period. It also reflects a deeper philosophical

the DeepSeekMoE architecture (Dai et al., 2024), which adopts fine-grained expert segmentation and shared expert isolation for higher potential in expert specialization. The DeepSeekMoE architecture demonstrates ????????? ′ 1 ???????????????????????? 1 2 ????????????????????????-1 ???????????????????????? Shared Expert Routed Expert Top-???????????????????????? Attention Feed-Forward Network … 3 4 RMS propose the decoupled RoPE strategy that uses additional multi-head queries q? ?,? ∈ R?? ℎ and a shared key k? ? ∈ R?? ℎ to carry RoPE, where ?? ℎ denotes the per-head dimension of the decoupled queries

transform sub-tree to TensorCore Intrinsics Pattern Matching 计算平台事业部 shared/global lecal 印16/int8 - fpl6/ints ecal 6;++k inner_ innerf Jorfintjc cshared_ocallk_inner_innerj* B_sharea_locollffk inner_inner* 81+ c少 了 了 Jorfintk_inner_inner=0;kinner_i utilization 。Double buffer to hide memory load latency 。 storage align to reduce bank conflicts of shared memory 。 Virtual threads for data reuse (on going) Performance on V100 (FP16) 计算平台事业部 COMPUTING

rights reserved. Example: Dispatch Codegen Built Shared Library runtime::PackedFunc DNNLModule::GetFunction( const std::string& name, const std::shared_ptr& sptr_to_self) { if (name == "init") reinterpret_cast(arg->data); } (*func_s)(packed_args, out); *rv = out; });}} Load the built shared library Get the corresponding subgraph function Execute the subgraph© 2019, Amazon Web Services

Tianqi ChenCurrent Deep Learning Landscape Frameworks and Inference engines DL Compilers Kenrel Libraries Hardware CuDNN NNPack MKL-DNN Hand optimized Open source, automated end-to- end optimization

a workround from FrozenGene a. python/tvm/contrib/ndk.py options = options if options else [ “-shared”, “-fPIC”, “-m32”] b. python tensorflow_blur.py to get the .log c. Use the .log, with target=“llvm

com/Xilinx/ml-suite/blob/master/examples/deployment_modes/mp_classify.py) Streamlined multi-process pipeline using shared memory Usually need >4 Pre-Process cores running to keep up with FPGA ˃ TVM pipeline needed. CPU/FPGA