In recent years, Intel has been talking about its Cascade Lake servers with DL Boost (also called VNNI, Vector Neural Net Instructions). These new features are a subset of the AVX-512 and are intended to specifically accelerate processor performance in artificial intelligence applications. Historically, many AI applications have favored GPUs over processors. The GPU architecture is much better suited to graphics processors than processors. Processors offer many more thread-based execution resources, but even today's multicore processors are overwhelmed by the parallelism available in a high-end graphics processor core.
Anandtech did you compare the performance of Cascade Lake, the Epyc 7601 (soon outperformed by the AMD Rome 7nm processors, but still today AMD's main server core), and a RTX Titan. The article, written by the excellent Johan De Gelas, discusses different types of neural networks beyond CNN networks (convolutional neural networks), which are generally compared, and explains how a key element of the strategy Intel is competing with Nvidia in workloads where GPUs are not as powerful. or can not yet meet the emerging market needs due to memory capacity constraints (GPUs still can not match CPUs here), the use of 'light' artificial intelligence models does not not requiring long periods of workout, or artificial intelligence models that rely on statistical models of non-neural networks.
Growing revenue from data centers is a critical part of Intel's global strategy for artificial intelligence and machine learning. Nvidia, meanwhile, is keen to protect a market in which it currently has virtually no competition. Intel's Artificial Intelligence strategy is broad and encompasses many products, from Movidius and Nervana to DL Boost on Xeon, to the next GPU Xe range. Nvidia seeks to show that GPUs can be used to handle artificial intelligence calculations in a wider range of workloads. Intel incorporates new artificial intelligence features into its existing products, uses new hardware that, it is hoped, will impact the market, and attempts to create its first serious GPU to challenge the work done by AMD and Nvidia in the consumer market.
Anandtech's benchmarks show, overall, that the gap between Intel and Nvidia remains wide, even with DL Boost. This graph of a recurrent neural network test used a "Long Short Term Memory (LSTM)" network as a neural network. A type of RNN, LSTM "selectively remembers" patterns over a period of time. "Anandtech also used three different configurations to test it – Tensorflow ready to use with conda, a Tensorflow optimized for Intel with PyPi, and optimized version of Tensorflow from the source using Bazel, using the latest version of Tensorflow.
This pair of images captures the relative scale between the processors as well as the comparison with the RTX Titan. Ready-to-use performance was quite poor under AMD, although it improved with the optimized code. Intel's performance skyrocketed like a rocket when the source-optimized version was tested, but even the source-optimized version did not fit the performance of Titan RTX very well. De Gelas notes: "Secondly, we were pretty surprised that our Titan RTX is less than three times faster than our dual Xeon setup," which explains how these comparisons are done in the larger article.
DL Boost is not enough to narrow the gap between Intel and Nvidia, but in all fairness, this has probably never been supposed to be. Intel's goal is to improve AI performance enough on Xeon to make the execution of these plausible workloads on servers that will be mainly used for other purposes, or when creating models of artificial intelligence that do not meet the constraints of D & C A modern graphics processor. The long-term goal of the company is to compete in the AI market with a range of equipment, not just Xeons. Since Xe is not quite ready yet, competing in the HPC space means competing with Xeon for the moment.
For those of you who are wondering about AMD, AMD does not really talk about performing artificial intelligence workloads on Epyc processors, but is focused on its RocM initiative to run CUDA code under OpenCL. AMD does not talk much about this side of its business, but Nvidia dominates the GPU application market for AI and HPC. AMD and Intel both want a piece of space. At the moment, both seem to be fighting for one.