Absolute monster with 18,432 CUDA cores and 64 graphics power TFLOPs

Absolute monster with 18,432 CUDA cores and 64 graphics power TFLOPs

by

We have a very delicious rumor going around today and it comes from a highly reliable source. @ kopite7kimi, the Twitter leaker responsible for virtually all Ampere leaks, revealed a rumor about the upcoming NVIDIA architecture Ada Lovelace (probably called NVIDIA ADA). Our 3DCenter colleagues extrapolated a lot of information about the size of the matrix, which Kopite seems to have more or less confirmed. Although the source is highly reliable, we are still marking this post as a rumor because of the magnitude of the leak.

NVIDIA ADA GPU leaked: Monster 64 TFLOPs GPU with 18432 CUDA cores and 5 nm process architecture

The Ada Lovelace architecture – which will probably only be referred to as NVIDIA ADA by the way – was recently released by Kopite (and confirmed by Videocardz) and it already looks like we have the preliminary specs for the next NVIDIA GPU. As we mentioned in Ada’s original article, Hopper seems to have been delayed for now (and along with it, NVIDIA’s MCM ambitions). Fortunately, it looks like NVIDIA has kept its pedal on the metal and its Ada architecture, powered by the AD102 GPU, will be an absolute beast. The following is the leakage of the original mold size:

The 3DCenter folks quickly extrapolated a ton of details (we revised their TFLOP numbers to be a little more conservative with a 1.75 GHz clock) that Kopite confirmed:

For those who want all the information in one place, here is a table that summarizes everything:

NVIDIA Lovelace AD102 GPU specifications

GPU AD102 GA102 TU102
Architecture Ada Lovelace Ampere Turing
Process 5nm Samsung 8nm TSMC 12nm NFF
Graphics processing clusters (GPC) 12 7 6
Texture processing clusters (TPC) 72 42 36
Streaming multiprocessors (SM) 144 84 72
CUDA Cores 18432 10752 4608
Theoretical TFLOPs 64.5 37.6 16.1
Launch March 2017 September 19 2022 (TBC)

The NVIDIA AD102 “GPU ADA” appears to have 18,432 CUDA cores based on information provided by Kopite. This is almost double the number of cores present in Ampere, which was already a major improvement over Turing. The only way this is possible is because NVIDIA is apparently building this in the 5 nm process, which has a significant matrix area and reduced power. Interestingly, if you assume a clock speed of 1.75 GHz, you will also be able to obtain the unique peak precision performance of the ADA 102 GPU: 64 TFLOPs.

According to Kopite, the ADA architecture will feature a much larger L2 cache (both Turing and Ampere have 6 MB of cache), which means that this could be an important architectural overhaul (as Turing was for Pascal and Pascal was for Fermi / Kepler) instead of just the shrinking of the usual process. It is also unclear at the moment whether NVIDIA will use Samsung’s 5 nm process or TSMC. Although the company has already experienced poor results at Samsung, the fact that TSMC is stifled and will not have access to factory capacity for a long time means that NVIDIA may be more willing to exploit Samsung and have an “unrestricted” chance “Production.

Recap NVIDIA ADA GPU architecture

In many ways, Ada Lovelace can be considered the world’s first computer enthusiast. She was the first person to realize that the Analytical Machine proposed by Charles Babbage had applications beyond pure calculation and also published what was thought to be the first algorithm (becoming the first computer programmer) to be loaded by such a machine. That was almost half a century before Alan Turing finished his job and invented the general-purpose computer during the world war.

NVIDIA is known for basing its architectures on prominent physicists, mathematicians and scientists, and Ada Lovelace is no different. Videocardz managed to find a big tip at NVIDIA’s own merchandise store that seems to confirm this rumor about the Lovelace architecture being the company’s next generation of GPUs. If you look at the heroes presented during the GTC lecture in 2018, you will find not only Ada Lovelace, but what potentially are all future NVIDIA architectural codenames. Jensen may have stealthily left the entire future script (as far as codenames are concerned) at the GTC’18 talk.

There are now several rumors that seem to suggest that the Lovelace architecture will be based on a 5 nm process. As NVIDIA transitioned to the Samsung smelter, it is unclear whether 5 nm refers to a TSMC or Samsung process. Keep in mind, however, that a recent report from Korea also confirmed an order for 6nm from NVIDIA – meaning that there is another generation of NVIDIA before Lovelace or that the 6nm process was for the upgrade line.

Source