TensorBoard 2.9.0 ‣ Nsight Compute 2023.1.1.4 ‣ Nsight Systems 2023.2.3.1001 ‣ NVIDIA TensorRT™ 8.6.1.6 ‣ Torch-TensorRT 1.5.0.dev0 ‣ NVIDIA DALI® 1.27.0 ‣ MAGMA 2.6.2 ‣ JupyterLab 2.3.2 including Jupyter-TensorBoard PyTorch code. ‣ A preview of Torch-TensorRT (1.4.0dev0) is now included. Torch-TRT is the TensorRT integration for PyTorch and brings the capabilities of TensorRT directly to Torch in one line Python PyTorch Container Versions The following table shows what versions of Ubuntu, CUDA, PyTorch, and TensorRT are supported in each of the NVIDIA containers for PyTorch. For earlier container versions, refer

深度学习-分布式模型推理 • 推理性能优化 • 减少计算量： operator fusion/XLA/TVM/prune/float16/quantization • 加快计算速度： batching/TensorRT/MPS/SSE/AVX/Neon • operator fusion • 针对特定场景重写耗时算子 • 重构tensorflow计算引擎 • batching • 批量调度请求到GPU，增大并发和吞吐量