GP102 GV100 Tensor Cores NA NA 640 CUDA核数量 3456 3840 5120 处理器制程 - 16nm FinFET 12nm FinFET Core Clock(<=) 1621MHz 1531MHz 1450MHz GPU显存 显存类型 GDDR5X GDDR5 HBM2 显存位宽 384-bit 384-bit 4096-bit 显存带宽 480

compared to the linear computation complexity of RNNs. However, attention is still faster in wall clock time because it processes entire sequences together. The quadratic complexity of attention is addressed