OS Limitations of Vulkan Multi-GPU Support

Subject: Graphics Cards | March 21, 2017 - 07:47 PM |
Tagged: windows 10, vulkan, sli, multi-gpu, crossfire

Update (March 22nd @ 3:50pm EDT): And the Khronos Group has just responded to my follow-up questions. LDA has existed since Windows Vista, at the time for assisting with SLI and Crossfire support. Its implementation has changed in Windows 10, but that's not really relevant for Vulkan's multi-GPU support. To prove this, they showed LDA referenced in a Windows 8.1 MSDN post.

In short:

Vulkan's multi-GPU extensions can be used on Windows 7 and Windows 8.x. The exact process will vary from OS to OS, but the GPU vendor can implement these extensions if they choose, and LDA mode isn't exclusive to Windows 10.

 

Update (March 21st @ 11:55pm EDT): I came across a Microsoft Support page that discusses issues with LDA in Windows 7, so it seems like that functionality isn't limited to WDDM 2.0 and Windows 10. (Why have a support page otherwise?) Previously, I looked up an MSDN article that had it listed as a WDDM 2.0 feature, so I figured DSOGaming's assertion that it was introduced with WDDM 2.0 was correct.

As such, LDA might not require a GPU vendor's implementation at all. It'll probably be more clear when the Khronos Group responds to my earlier request, though.

That said, we're arguing over how much a GPU vendor needs to implement; either way, it will be possible to use the multi-GPU extensions in Windows 7 and Windows 8.x if the driver supports it.

Update (March 21st @ 7:30pm EDT): The Khronos Group has just released their statement. It's still a bit unclear, and I've submit another request for clarification.

Specifically, the third statement:

If an implementation on Windows does decide to use LDA mode, it is NOT tied to Windows 10. LDA mode has been available on many versions of Windows, including Windows 7 and 8.X.

... doesn't elaborate what is required for LDA mode on Windows outside of 10. (It could be Microsoft-supported, vendor-supported, or something else entirely.) I'll update again when that information is available. For now, it seems like the table, below, should actually look something like this:

  Implicit Multi-GPU
(LDA Implicit)
Explicit Multi-GPU
(LDA Explicit)
Unlinked Multi-GPU
(MDA)
Windows 7 Requires GPU Vendor
LDA Implementation?

(Or Equivalent)
Requires GPU Vendor
LDA Implementation?

(Or Equivalent)
Windows 8.1 Requires GPU Vendor
LDA Implementation?

(Or Equivalent)
Requires GPU Vendor
LDA Implementation?

(Or Equivalent)
Windows 10
macOS Apple doesn't allow the Vulkan API to ship in graphics drivers.
At all.
Linux / etc.

... but we will update, again, should this be inaccurate.

Update (March 20th @ 3:50pm EDT): The Khronos Group has just responded that the other posts are incorrect. They haven't yet confirmed whether this post (which separates "device groups" from the more general "multi-GPU in Vulkan") is correct, though, because they're preparing an official statement. We'll update when we have more info.

Original Post Below (March 19th @ 9:36pm EDT)

A couple of days ago, some sites have noticed a bullet point that claims Windows-based GPU drivers will need WDDM in “linked display adapter” mode for “Native multi-GPU support for NVIDIA SLI and AMD Crossfire platforms” on Vulkan. This note came from an official slide deck by the Khronos Group, which was published during the recent Game Developers Conference, GDC 2017. The concern is that “linked display adapter” mode is a part of WDDM 2.0, which is exclusive to Windows 10.

This is being interpreted as “Vulkan does not support multi-GPU under Windows 7 or 8.x”.

khronos-2017-vulkan-alt-logo.png

I reached out to the Khronos Group for clarification, but I’m fairly sure I know what this does (and doesn’t) mean. Rather than starting with a written out explanation in prose, I will summarize it into a table, below, outlining what is possible on each platform. I will then elaborate below that.

  Implicit Multi-GPU
(LDA Implicit)
Explicit Multi-GPU
(LDA Explicit)
Unlinked Multi-GPU
(MDA)
Windows 7    
Windows 8.1    
Windows 10
macOS Apple doesn't allow the Vulkan API to ship in graphics drivers.
At all.
Linux / etc.

So the good news is that it’s possible for a game developer to support multi-GPU (through what DirectX 12 would call MDA) on Windows 7 and Windows 8.x; the bad news is that no-one might bother with the heavy lifting. Linked display adapters allow the developer to assume that all GPUs are roughly the same performance, have the same amount of usable memory, and can be accessed through a single driver interface. On top of these assumptions, device groups also hide some annoying and tedious work inside the graphics driver, like producing a texture on one graphics card and quickly giving it to another GPU for rendering.

Basically, if the developer will go through the trouble of supporting AMD + NVIDIA or discrete GPU + integrated GPU systems, then they can support Windows 7 / 8.x in multi-GPU as well. Otherwise? Your extra GPUs will be sitting out unless you switch to DirectX 11 or OpenGL (or you use it for video encoding or something else outside the game).

On the other hand, this limitation might pressure some developers to support unlinked multi-GPU configurations. There are some interesting possibilities, including post-processing, GPGPU tasks like AI visibility and physics, and so forth, which might be ignored in titles whose developers were seduced by the simplicity of device groups. On the whole, device groups was apparently a high-priority request by game developers, and its inclusion will lead to more multi-GPU content. Developers who can justify doing it themselves, though, now have another reason to bother re-inventing a few wheels.

Or... you could just use Linux. That works, too.

Again, we are still waiting on the Khronos Group to confirm this story. See the latest update, above.

Serious Sam VR: The Last Hope scales with multi-GPU

Subject: Graphics Cards | November 14, 2016 - 11:22 AM |
Tagged: video, the last hope, serious sam vr, rx 480, radeon, Polaris, multi-gpu, liquidvr, amd, affinity

While VR excitement might have cooled slightly in the enthusiast community, there continues to be innovation and software releases on both the Oculus Rift and HTC Vive that are bringing me back to what I think we believe to be part of the future of PC gaming. Serious Sam VR: The Last Hope was announced at E3 this year and is now available as an early access game on Steam. It is a dual wielding shooter that combines the enemies of the previous games along with the crazy weapons that made the series iconic.

ssvrscreen1.jpg

And hey, there is something awesome about using a missile launcher that takes up half the screen.

One interesting technology addition to the game is use of AMD LiquidVR affinity multi-GPU. A Croteam developer recently posted a blog on the GPUOpen.com site talking about the implementation.

We wanted to add LiquidVR Affinity Multi-GPU rendering support to our engine because two GPUs can render the two eye views in almost half the time compared to a single GPU and this would greatly reduce our GPU bottlenecks. Affinity MGPU can either be done in one pass or with a separate pass for each eye, in which case we reap the GPU side benefits while the CPU workload stays the same.

ssvrscreen2.jpg

We needed about a week to modify all shaders and to make sure that correct data is set for each eye. Single pass rendering with Affinity Multi-GPU gave us a huge speed improvement on both CPU and GPU from our original VR implementation. In the end, it took us less time to do single pass rendering correctly than it took us to fix all the problems caused by multi pass multi-GPU rendering.

After the interest in the Deus Ex multi-GPU scaling video I thought I would see if the Serious Sam implementation was actually beneficial to gamers.

  • Test System
  • Core i7-5960X
  • X99 MB + 16GB DDR4
  • AMD Radeon RX 480 8GB
  • Driver: 16.10.2

The test was simple: I found that a single RX 480 could run the game at Medium settings perfectly well, but could it be playable on High with multi-GPU? By adding in a second Radeon RX 480 I was able to bring the performance up by 55% or so, making the VR experience nearly flawless.

ssvr-1.png

ssvr-2.png

It's not perfect scaling, but the benefits of multi-GPU for VR, when properly implemented, are obvious. As more games and experiences are released that require higher compute capability or have in-game settings that allow for better image quality, the ability to scale across GPUs will be a welcome addition to the ecosystem.

Check out the video here if you haven't seen any Serious Sam VR gameplay yet!

Podcast #423 - Logitech G231, DX12 Multi-GPU, GTX 1070/1080 issues and more!

Subject: General Tech | November 3, 2016 - 10:35 AM |
Tagged: vrm, video, skyrim, qualcomm, prodigy, powercolor, podcast, nxp, multi-gpu, msi, micron, logitech, GTX 1080, gtx 1070, g231, evga, dx12, devil box, deus ex: mankind divided, amd, Alienware 13

PC Perspective Podcast #423 - 11/03/16

Join us this week as we discuss the Logitech Prodigy G231, multi-GPU scaling with DX12, Qualcomm buying NXP, issues with GTX 1070 and 1080 cards and more!

You can subscribe to us through iTunes and you can still access it directly through the RSS page HERE.

The URL for the podcast is: http://pcper.com/podcast - Share with your friends!

Hosts:  Ryan Shrout, Allyn Malventano, Josh Walrath, Jeremy Hellstrom

Program length: 1:10:25

  1. Week in Review:
  2. Today’s episode is brought to you by Harry’s! Use code PCPER at checkout!
  3. News items of interest:
  4. Hardware/Software Picks of the Week
  5. Closing/outro

Subscribe to the PC Perspective YouTube Channel for more videos, reviews and podcasts!!

DX12 Multi-GPU scaling up and running on Deus Ex: Mankind Divided

Subject: Graphics Cards | November 1, 2016 - 11:57 AM |
Tagged: video, rx 480, readon, nvidia, multi-gpu, gtx 1060, geforce, dx12, deus ex: mankind divided, amd

Last week a new update was pushed out to Deus Ex: Mankind Divided that made DX12 a part of the main line build and also integrated early support for multi-GPU support under DX12. I wanted to quickly see what kind of scaling it provided as we still have very few proof points on the benefit of running more than one graphics card with games utilizing the DX12 API.

As it turns out, the current build and driver combination only shows scaling on the AMD side of things. NVIDIA still doesn't have DX12 multi-GPU support enabled at this point for this title.

  • Test System
  • Core i7-5960X
  • X99 MB + 16GB DDR4
  • AMD Radeon RX 480 8GB
    • Driver: 16.10.2
  • NVIDIA GeForce GTX 1060 6GB
    • Driver: 375.63

mankind1080-avg.png

Not only do we see great scaling in terms of average frame rates, but using PresentMon for frame time measurment we also see that the frame pacing is consistent and provides the user with a smooth gaming experience.

mankind1080-plot.png