Oak Ridge National Laboratory Chooses IBM and NVIDIA for Two Supercomputers, Summit and Sierra

Subject: General Tech, Systems | November 27, 2014 - 08:53 PM |
Tagged: nvidia, IBM, power9, Volta

The Oak Ridge National Laboratory has been interested in a successor for their Titan Supercomputer. Sponsored by the US Department of Energy, the new computer will be based on NVIDIA's Volta (GPU) and IBM's POWER9 (CPU) architectures. Its official name will be “Summit”, and it will have a little sibling, “Sierra”. Sierra, also based on Volta and POWER9, will be installed at the Lawrence Livermore National Laboratory.

View Full Size

Image Credit: NVIDIA

The main feature of these supercomputers is expected to be “NVLink”, which is said to allow unified memory between CPU and GPU. This means that, if you have a workload that alternates rapidly between serial and parallel tasks, that you can save the lag in transferring memory between each switch. One example of this would be a series of for-each loops on a large data set with a bit of logic, checks, and conditional branches between. Memory management is like a lag between each chunk of work, especially across two banks of memory attached by a slow bus.

Summit and Sierra are both built by IBM, while Titan, Oak Ridge's previous supercomputer, was developed by Cray. Not much is known about the specifics of Sierra, but Summit will be about 5x-10x faster (peak computational throughput) than its predecessor at less than a fifth of the nodes. Despite the fewer nodes, it will suck down more total power (~10MW, up from Titan's ~9MW).

These two supercomputers are worth $325 million USD (combined). They are expected to go online in 2017. According to Reuters, an additional $100 million USD will go toward research into "extreme" supercomputing.

Source: Anandtech

November 27, 2014 | 11:46 PM - Posted by Anonymous (not verified)

Power8's support up to 8 memory buffer chips, that can have 4 channels of DDR3 memory each, for up to 410 gigabytes per second peak bandwidth(for Memory), so Nvlink, which is derived from IBM's CAPI(Coherent Accelerator Processor Interface), IBM's coherency protocol which sits over the PCI protocol and allows for coherency with GPU/other accelerators, will allow for unified Memory, and coherence between the CPU and GPU data. So Nvlink will allow the power9's to attach Nvidia's GPUs much like CAPI, but with Nvidia's extra tweaking.

There is very little information on the Power9, but looking at what the power8 can already do, with its 32 total channels of DDR3 memory(4 Channels per centaur controller chip), 14 execution units per core, and the ability to have its VMX units execute standard FP instructions when they are not busy doing SIMD vector work, all while being able to handle 8 processor threads per core, for a total of 96 threads for the 12 core power8, who knows what the power9 will bring as far as raw processing power. It's no wonder that the contract was awarded to IBM/Open Power/Nvidia. Hopefully all that Government funding of the latest supercomputers will result in some more powerful GPUs from the Green team.

With the CAPI technology from IBM, made available through the Open Power Foundation, as well as the Power8 core reference design, as licensable IP, AMD would be very wise not to pass up joining the OpenPower foundation and making some Power8 business for itself, x86 has some serious competition coming in the HPC/Server/supercomputer market, and AMD has just as much ability, as Nvidia when it comes to GPUs, and having competition in the HPC accelerator market will keep Nvidia on its toes. The Third party non IBM Power8s will be arriving in 2015, and Tyan already has a developer power8 system, I hope Phoronix will be reviewing the full systems in 2015, as they will be running Linux on Power, and maybe some gaming benchmarks will be forthcoming. For sure the third party power8s will be more affordable, just look what Licensed ARM IP did for the mobile Tablet/Phone market.

See the article in the Sept 2, 2013 edition of The Microprocessor Report, for a full listing of the Power8's ability Titled: "Power8 Muscles Up for servers" to get the full idea of just what a powerful beast the Power8 is, and that will be a good starting point as to what will Power9, at least, will be able to do, and then some.

November 28, 2014 | 01:52 PM - Posted by Anonymous (not verified)

Am not surprised IBM picked Nvidia over AMD as they are partners (IBM & NVidia). And the government picking IBM isn't a surprise either. But in 2015 Intel and AMD will have superior hardware to IBM and Nvidia. We can already see from the leaks of AMD GPU`s and the CPU`s coming from Intel on 14nm, and AMD CPU`s on 20nm.

These Super computers will not be the fastest.

November 28, 2014 | 06:17 PM - Posted by Anonymous (not verified)

In 2015 Intel will still be trying to catch up to Power8 in raw processing power, and server benchmarks, and there is nothing stopping AMD from offering its GPUs as accelerators for the third party licensed Power8 systems market, OpenPower will be licensing Power8 IP, as well as the CAPI coherent interface that allows PCI to have coherency between Power8's and PCI attached GPU accelerators. AMD has a server business(SeaMicro) and currently SeaMicro is using Intel SKUs, as well as AMD x86 SKUs, and AMD's Arm based Seattle SKUs, so AMD needs to get a Power8 license from OpenPower and add one more feather to their cap. It will be a big mistake for AMD to let Tyan, and others, get too much of a jump on AMD in the third party licensed Non IBM power8 market. The Chinese market for Power8s, licensed from OpemPower is going to be huge, and SeaMicro as a division of AMD will most certainly have to get power8s from another third party supplier, if that's what SeaMicro's customers want, AMD makes money from selling servers with Intel SKUs, via SeaMicro, and business is funny that way, when you own a server business.

IBM's in house Power8s, and services may cost a lot, but those third party Power8 licensees Like Tyan, and others, will be offering licensed Power8 based systems at a much more affordable price structure, and Samsung and GlobalFoundries will be sharing the third party Power8 fabrication business. GlobalFoundries now owns IBM's commercial fabrication facilities, and will also be fabricating for IBM's in house Power8's/Power9's supplies. Expect a market to build up around licensed Power8/Power CPUs, very much similar to the market that is built around the ARM based designs. The x86 market will now have to compete with many Power8 licensees starting in 2015.

November 28, 2014 | 01:54 PM - Posted by Anonymous (not verified)

AMD Has been far superior to Nvidia in Compute for the past two generations (Maybe even third), so Nvidia wining was bcoz IBM being its partner, and not performance.

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Lines and paragraphs break automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd> <blockquote><p><br>
  • Web page addresses and e-mail addresses turn into links automatically.

More information about formatting options

By submitting this form, you accept the Mollom privacy policy.