Inspur Information AI Servers with NVIDIA A100 Tensor Core GPUs Maintain Top Single-Node Performance in MLPerf Training v2.0 Global AI Benchmarks

Industry-leading performance with the BERT model with Transformer architecture was achieved for the third time in a row through enhanced hardware-based optimization capabilities

SAN JOSE, California, July 1, 2022BUSINESS WIRE)–The open engineering consortium MLCommons™ has released the latest MLPerf™ Training v2.0 results, featuring Inspur AI servers leading the way in single-node performance in closed divisions.

This press release contains multimedia. View the full edition here:

BERT Single-Node Training Performance in MLPerf v2.0 (Image: Business Wire)

MLPerf is the world’s most influential benchmark for AI performance. It is operated by MLCommons, with members from more than 50 world-leading AI companies and top academic institutions, including Inspur Information, Google, Facebook, NVIDIA, Intel, Harvard University, Stanford University and the University of California, Berkeley. MLPerf AI Training benchmarks are held twice a year to track improvements in computer performance and provide authoritative data guidance for users.

The latest MLPerf Training v2.0 attracted 21 global manufacturers and research institutions, including Inspur Information, Google, NVIDIA, Baidu, Intel-Habana and Graphcore. There were 264 entries, a 50% increase from the previous round. The eight AI benchmarks cover current general-purpose AI scenarios, including image classification with ResNet, medical image segmentation with 3D U-Net, lightweight object detection with RetinaNet, heavyweight object detection with Mask R-CNN, speech recognition with RNN-T, natural language processing with BERT , recommendation with DLRM and reinforcement learning with MiniGo.

Among the closed division benchmarks for single-node systems, Inspur Information with its high-performance AI servers was the top performer in natural language processing with BERT, recommendation with DLRM and speech recognition with RNN-T. It won the most titles among the single-node system submitters. For mainstream high-end AI servers equipped with eight NVIDIA A100 Tensor Core GPUs, Inspur Information AI servers topped five tasks (BERT, DLRM, RNN-T, ResNet, and Mask R-CNN).

Continue to lead in AI computing performance

Inspur AI servers continue to achieve breakthroughs in AI performance through extensive software and hardware optimization. Compared to the MLPerf v0.5 results in 2018, Inspur AI servers showed significant performance gains of up to 789% for typical 8-GPU server models.

The industry-leading performance of Inspur AI servers in MLPerf is the result of outstanding design innovation and full-stack optimization capabilities for AI. Targeting the bottleneck of intensive I/O transmission in AI training, the PCIe retimer-free design of Inspur AI servers provides fast interconnection between CPUs and GPUs for reduced communication delays. For intensive job scheduling with multiple GPUs, the data transfer between NUMA nodes and GPUs is optimized to ensure the highest performance of data I/O in training jobs. In terms of heat dissipation, Inspur Information leads the way in deploying eight 500W high-end NVIDIA Tensor Core A100 GPUs in a 4U space and supports air cooling and liquid cooling. Meanwhile, Inspur AI servers continue to optimize data processing performance before training and use combined optimization strategies such as hyperparameter and NCCL parameter, as well as the many enhancements provided by the NVIDIA AI software stack, to maximize the training performance of AI models. .

Significantly improve Transformer training performance

Pre-trained massive models based on Transformer’s neural network architecture have led to the development of a new generation of AI algorithms. The BERT model in the MLPerf benchmarks is based on the Transformer architecture. Transformer’s concise and stackable architecture allows for training massive models with huge parameters. This has vastly improved the algorithms for large models, but placed greater demands on processing performance, communication interconnection, I/O performance, parallel expansion, topology and heat dissipation for AI systems.

In the BERT benchmark, Inspur AI servers have further improved BERT training performance by using methods such as optimizing data preprocessing, improving dense parameter communication between NVIDIA GPUs, and automatic optimization of hyper parameters, etc. Inspur Information AI servers can complete the BERT model training of approximately 330 million parameters in just 15,869 minutes with 2,850,176 pieces of data from the Wikipedia dataset, a 309% improvement in performance compared to the peak performance of 49.01 minutes in Training v0.7. So far, Inspur AI servers have won the MLPerf Training BERT benchmark for the third time in a row.

Inspur Information’s two AI servers with top scores in MLPerf Training v2.0 are NF5488A5 and NF5688M6. The NF5488A5 is one of the first servers in the world to support eight NVIDIA A100 Tensor Core GPUs with NVIDIA NVLink technology and two AMD Milan CPUs in a 4U space. It supports both liquid cooling and air cooling. It has won a total of 40 MLPerf titles. NF5688M6 is a scalable AI server designed for large-scale data center optimization. It supports eight NVIDIA A100 Tensor Core GPUs and two Intel Ice Lake CPUs, up to 13 PCIe Gen4 IO, and has won a total of 25 MLPerf titles.

About Inspur Information

Inspur Information is a leading provider of data center infrastructure, cloud computing and AI solutions. It is the second largest server manufacturer in the world. Through engineering and innovation, Inspur Information delivers advanced computing hardware design and comprehensive product offerings to key technology sectors such as open computing, cloud data center, AI, and deep learning. Performance-optimized and purpose-built, our world-class solutions empower customers to address specific workloads and real-world challenges. For more information, visit

Check out the source version at


Fiona Liu
PR manager
Inspur Information
[email protected]

Vivian Kelly
Interprose for Inspur information
+1 703.509.5412
[email protected]

Leave a Comment

Your email address will not be published. Required fields are marked *