Feedback

AMD Mantle and NVIDIA 337.50 Scaling Demonstrated with Star Swarm on AM1

Author:
Manufacturer: Various

Competition is a Great Thing

While doing some testing with the AMD Athlon 5350 Kabini APU to determine it's flexibility as a low cost gaming platform, we decided to run a handful of tests to measure something else that is getting a lot of attention right now: AMD Mantle and NVIDIA's 337.50 driver.

Earlier this week I posted a story that looked at performance scaling of NVIDIA's new 337.50 beta driver compared to the previous 335.23 WHQL. The goal was to assess the DX11 efficiency improvements that the company stated it had been working on and implemented into this latest beta driver offering. In the end, we found some instances where games scaled by as much as 35% and 26% but other cases where there was little to no gain with the new driver. We looked at both single GPU and multi-GPU scenarios on mostly high end CPU hardware though.

Earlier in April I posted an article looking at Mantle, AMD's answer to a lower level API that is unique to its ecosystem, and how it scaled on various pieces of hardware on Battlefield 4. This was the first major game to implement Mantle and it remains the biggest name in the field. While we definitely saw some improvements in gaming experiences with Mantle there was work to be done when it comes to multi-GPU scaling and frame pacing. 

Both parties in this debate were showing promise but obviously both were far from perfect.

View Full Size

While we were benchmarking the new AMD Athlon 5350 Kabini based APU, an incredibly low cost processor that Josh reviewed in April, it made sense to test out both Mantle and NVIDIA's 337.50 driver in an interesting side by side.

Continue reading our story on the scaling performance of AMD Mantle and NVIDIA's 337.50 driver with Star Swarm!!

Here is the setup. Using a GeForce GTX 750 Ti and a Radeon R7 260X graphics card, somewhat equivalent in terms of pricing and performance, we ran the Star Swarm stress test benchmark. This application, built originally to demonstrate the performance abilities of AMD's Mantle API, was also used in some of NVIDIA's slides to demonstrate the performance improvement in its latest beta driver stack. To add some interest to the test, we ran these on both the AMD Athlon 5350 APU (considered one of the lowest performing platforms you'll find a x16 PCIe slot) as well as the high-end Core i7-3960X 6-core Sandy Bridge-E platform we use for our normal GPU test bed.

Test System Setup
CPU Intel Core i7-3960X Sandy Bridge-E
AMD Athlon 5350 Kabini APU
Motherboard ASUS P9X79 Deluxe
Gigabyte AM1M-S2H
Memory Corsair Dominator DDR3-1600 16GB
Hard Drive OCZ Agility 4 256GB SSD
Sound Card On-board
Graphics Card AMD Radeon R7 260X 2GB
NVIDIA GeForce GTX 750 Ti 2GB
Graphics Drivers NVIDIA: 335.23 WHQL, 337.50 Beta
AMD: Catalyst 14.3 Beta
Power Supply Corsair AX1200i
Operating System Windows 8 Pro x64

I think you'll find the results are quite interesting. Let's take a look.

Our first set of results looks at a pre-optimized software stack. That means we are looking at the GeForce GTX 750 Ti with the 335.23 driver and the Radeon R7 260X with DirectX on the 14.3 driver.

View Full Size

If you look at the results from the Core i7-3960X platform, the GeForce GTX 750 Ti has the advantage in average frame rate by 52% - a strong margin. On the Athlon 5350, the much slower processor in our set of tests, that performance lead for NVIDIA's GTX 750 Ti shrinks to 18%. Clearly the hardware and driver for the GTX 750 Ti is more able to take advantage of the available CPU headroom with DirectX compared to what AMD is doing with the 260X and its DX11 implementation. 

Moving from the slower CPU to the much faster CPU, NVIDIA's hardware sees improvement by 116%. AMD on the other hand only improves by 68% indicating that either the R7 260X itself, or the driver, isn't able to take advantage over the extra CPU headroom.

Now, let's look at these same sets of results but using a post-optimized software stack that enabled Mantle for the R7 260X with the Catalyst 14.3 driver and moved to the NVIDIA 337.50 driver for the GTX 750 Ti.

View Full Size

Things look a little bit different this time! At the top end of the graph on the results run on the Core i7-3960X processor, NVIDIA's GeForce GTX 750 Ti maintains a lead, though slightly smaller, at 45%. Keep in mind that this is WITH Mantle enabled on the 260X and with the new DX11 changes made in NVIDIA's 337.50 driver. 

On the Athlon 5350 platform, AMD's R7 260X is able to take the lead away from NVIDIA's GTX 750 Ti by 8%. This indicates that the performance advantage of Mantle on the lower end platform is larger for AMD than the DX11 changes in NVIDIA's 337.50 driver are for the GTX 750 Ti.

If we compare the 260X performance on both platforms though, clearly something is holding it back. Moving from the Athlon 5350 to the Core i7-3960X only sees a 12% performance improvement while NVIDIA's GeForce GTX 750 Ti average frame rate increases by more than 75%. 

Finally, let's see all these results on the same graphic for a different view.

View Full Size

On this image you can see how each platform with each graphics card is able to scale with the software changes made with Mantle and with the 337.50 driver. In the red bar we have results from the 335.23 NVIDIA driver and the 14.3 Catalyst driver running in DirectX mode while the blue represents the Mantle and 337.50 scores. 

Impressively, the Radeon R7 260X sees a performance improvement of 91% by enabling the Mantle version of Star Swarm on the lower end Athlon 5350 APU. On that same system, NVIDIA's GeForce GTX 750 Ti is able to achieve a 49% frame rate increase while continuing to use DirectX 11 and improving driver efficiency. On the high end platform with the Core i7-3960X, the R7 260X GPU improves by just 27% from enabling Mantle; the GTX 750 Ti scales by 21%.

Both GPUs and both software changes (Mantle and 337.50) see more performance advantages when running on the Athlon 5350 APU system than on the Core i7-3960X system, which obviously makes a lot of sense. Of these two different solutions we are showing, only on two processors and only with one game, AMD Mantle appears to have the bigger potential performance advantage for games that are CPU bound, or in CPU-bound sections of a game.

But what NVIDIA has done with the 337.50 driver changes are impressive considering they are staying within the bounds of the existing, well entrenched and well known API of DirectX 11. A 49% gain is nothing to sneeze at even though Mantle saw a 91% advantage under the same conditions. 

Many users will argue (and have in our comment sections) that what NVIDIA has done with 337.50 is really just game-specific optimizations and isn't anything different than what these two GPU companies have been doing for the past decade. Even though NVIDIA has clearly stated that is not the case, you are make up your own mind if you choose to believe them, but I would posit that it doesn't really matter. Is not a game-specific Mantle integration the same thing and possibly even more troublesome as it requires the developer, rather than the hardware vendor, to take the brunt of the work?

View Full Size

Clearly we would like to have more data on these graphs with more graphics cards and we might be able to do some of that testing starting next week. I think seeing how a GeForce GTX 780 or Radeon R9 290X scales on these two platforms would be equally as compelling. 

These tests above aren't meant to be conclusive evidence for either vendor's current direction but are simply there to add more data points to our discussion moving forward. AMD Mantle has huge potential upside but requires a lot of commitment from game developers and AMD's own software team to keep up. NVIDIA's continued DirectX improvement seems have a lower peak but can be implemented on a much wider array of games and without the need for developers to commit to something long term.

I'm almost afraid to ask but...please leave me your thoughts on this debate and the future of both Mantle and DX11 in the comments below!

April 18, 2014 | 03:21 PM - Posted by collie (not verified)

And BooM!, Ryan bitch slaps the hatters! You want fair and biased test and reviews, y'all get it, bitches! great read, interesting results. And it answers the question, "Can I throw a gpu at a AM1?" Yes, yes you can.

April 18, 2014 | 04:05 PM - Posted by Ryan Shrout

Hopefully you meant UNbiased, right?

April 18, 2014 | 04:11 PM - Posted by Collie (not verified)

Yea, good catch

April 18, 2014 | 04:30 PM - Posted by Anonymous (not verified)

No temp or clock results yet for 335.23 vs 337.50 ?

http://www.hardocp.com/article/2014/04/08/amd_radeon_r9_295x2_video_card...


On the temps, we have noticed with these latest drivers the 780 Ti is able to go up to 87c now, versus 84c with previous drivers. The observed clock speed also seemed slightly higher than we've experienced before, 1019MHz sustained clock speed on the GTX 780 Ti's while gaming. In the past, we have seen about 1006MHz clock speeds on these cards. This new 337.50 driver may have changed the thermal profile slightly to allow slightly higher clock speeds.

April 19, 2014 | 04:47 PM - Posted by Anonymous (not verified)

hello ryan,
it's your 3rd article and benchs, and you are still avoiding cpu overhead situation, either by putting gpu limited situation or 3960x for no bottleneck.
after all these benchs it just seem really unlikely that your knowledge about how thing works just vanished, or the most likely sponsored article.
as i commented on the last article this became weird for me to even comment on, you seemed very persistant to avoid giving relevant bench to the cpu overhead issue repeatedly, by hiding behind gpu limited situations, tweeking settings to show special effect management, buged games, SLI, very high setting, anyone with half a brain knows that multi-gpu isnt cpu overhead, or gpu bound isn't cpu overhead, or effects resource management optimisation isn't cpu overhead.
but there you go, you seem convinced that most readers are too stupid to notice, and most definitly there are some, just by- reading some of the comments here.
1st article i thought you didnt know and made a mistake.
2nd article i found it weird, to see how persistant you are at avoiding cpu overhead.
3rd article just removed any doubt i might had.

April 18, 2014 | 05:26 PM - Posted by Anonymous (not verified)

You mean Ryan didn't forget to read the 337.50 release notes this time ?

335.23 & prior
NO Star Swarm optimization

337.50
◦Up to 21% in Star Swarm
Star Swarm optimization

Well maybe he did


Many users will argue (and have in our comment sections) that what NVIDIA has done with 337.50 is really just game-specific optimizations and isn't anything different than what these two GPU companies have been doing for the past decade. Even though NVIDIA has clearly stated that is not the case, you are make up your own mind if you choose to believe them, but I would posit that it doesn't really matter. Is not a game-specific Mantle integration the same thing and possibly even more troublesome as it requires the developer, rather than the hardware vendor, to take the brunt of the work?

Some how he is refusing to read release notes and take what ever Nvidia tell him as truth. That would be fine if it were true and not contradicted by the driver release notes.

He goes on to pose that driver optimization is better then game engine optimization?

I want what he is smoking so I can believe the Bengals will have a shot this year too.

April 25, 2014 | 06:30 AM - Posted by Anonymous (not verified)

The patch notes include a list of measured gains that Nvidia observed using 337.50... the fact that they noticed performance gains in Star Swarm is not indicative of game-specific optimization.

If Nvidia released a driver that forced your GPU to run 100Mhz faster, and in the patch notes Nvidia says it will give you a 10% increase in Star Swarm, would you call that a game-specific optimization?

Also, he didn't say that Nvidia's solution was BETTER, nor that Mantle is WORSE. He's saying they both have their pros and cons.

Nvidia's driver optimizations give benefits to the consumer without requiring anything extra from the developers, but (as demonstrated) it doesn't seem to have as great an improvement on performance.

Mantle (as demonstrated) can give a hell of a boost in performance, but it will require a lot more work from the developers if they want to take advantage of its strength.

Did you even bother to read the article you're commenting on?

April 18, 2014 | 03:34 PM - Posted by dragosmp (not verified)

Until Dx12 shows up I think MAntle will gain adoption and why not. It's fast and seems reliable enough, many of us have upgraded the drivers for far less. For one I can't wait the Mantle-enabled Civ5 that uses the CPU better so I'd finally be able to play a decent sized map on the laptop

April 18, 2014 | 05:24 PM - Posted by Anonymous (not verified)

By the time DX12 get released there should be some improvments to mantle, as AMD does have a jump on DX, so maybe there will be a Mantle 1.1, or whatever before DX12 get out the door! It's good to see all the intrested parties turn their attention to the Graphics APIs/Drivers. It makes one wonder why it took so long, on the Driver and Graphics API side, or was there an intentional intent to meter these graphics driver/API improvments out slowly to milk the incremental improvments for profits over these years. Yes let's slow the hardware improvments down a bit, and rebrand some SKUs, and only slightly improve the graphics API/Drivers, just enough to make the customer think that thay are getting some improvment for their money. But the rise of the Mobile Market, and AMD's lack of funds for rapid hardware improvments has forced AMD to make up for the slower hardware advances in its APU roadmap with a more rapid improvment in graphics API/driver development, and Nvidia is now forced to do the same. The rise of the mobile market as a money maker, and the more competition in the mobile market, with players like imagination technologies powerVR GPUs, and others, has pushed Nvidia to merge its mobile GPU microarchitecture with its desktop microarchitecture, that and AMD's HSA gamble beginning to pay off! The mobile market has a GPU(PowerVR wizard) with hardware raytracing, and in the future with the Mobile and Desktop GPU microarchitectures merged(Nvidia,others) between Mobile and desktop SKUs, it will be just a matter of scaling up the GPU, and that will make the only difference between desktop and mobile GPUs, just a matter of adding more execution units. I would not be suprised if one of the GPU IP players, in the mobile market, takes advantage of the modern GPUs ability to be scaled up, and introduces a descrete GPU to the market, to compete with AMD and Nvidia. A third player would help, Intel's intrest in the descrete GPU market appears to be nil, at the current time, but some of the mobile GPU technology innovations in GPUs and SOCs, just may begin to work their way into the descrete market, and descrete GPUs may just all become APU type complete SOCs unto themselves in not too many years. (See AMD's console APUs, and Nvidias talk of putting custom ARM Denver CPUs on some desktop Maxwell or later descrete GPUs)

April 18, 2014 | 05:38 PM - Posted by arbiter

MS won't release half finished beta like AMD did. They will make sure DX12 is ready hence why it won't be out til late next year.

April 18, 2014 | 11:00 PM - Posted by Rick (not verified)

That's a bad point

Min me?
Vista?
Win 8?

April 19, 2014 | 09:10 AM - Posted by Anonymous (not verified)

M$ is known to only pay attention to gaming when it serves their needs( Need to maintain OS domination, or as a carrot to entice gamers into their closed ecosystem, 30% cut of the action from game developers, middleman milking TIFKAM 8 OS crAPP store). Mantel and AMD's version of HSA, is starting to show results, as it will for all the other members of the HSA foundation with their versions of HSA, Arm Holdings, Samsung and Qualcomm, to name a few. The Khronos group will also be improving the OpenGL, OpenCL, etc graphics APIs! Steam OS will begin the transition to an open OS, designed and supported by the gaming industry, and other Open OS groups (Debian/Debian bsed). There is no dirver/API system that is not in a state of constent revision, and getting to the market first, and getting the gaming engines makers to support it will provide AMD and gaming with a definite advantage. M$ by not allowing its latest graphics API on windows 7, is just shooting itself in the foot, beacuse Mantle will not have that restriction for any AMD based GPUs, Nvidia is going to be at a disadvantage in going with a M$ DX based only solution, as the majority of computers at this time are running windows 7, and Nvidia is just attatching itself to an anchor known as 8. Mantle can and does provide an improved gamimg experience, not restricted to M$'s OS whims, or M$'s need to force usere into ots closed ecosystem. Gaming OS competition is good for gaming, and AMD with Mantle, and Khronos with its OpenGL, OpenCL will also benifit, hopefully AMD will make the Mantle API completely open, but Mantle's influnce has resulted in much more innovation, as healthy competition always does. M$ is and always will be a detriment to gaming and OS innovation, as long as it is allowed to maintain an OS monopoly.

April 21, 2014 | 04:03 AM - Posted by Jules (not verified)

Are you talking about the same MS that broke win XP with MS Security essentials?

Or the same MS that release a broken patch that drains the battery life out of Surface and didn't really care after a whole month(December) before releasing a fix?

April 18, 2014 | 03:48 PM - Posted by Ophelos

The only thing i get from this is what AMD has been saying that Mantle works great with low-end CPUs.

With these test being done with beta drivers and a beta game demo, i don't see much of a difference between the two at all.

April 18, 2014 | 03:55 PM - Posted by funandjam

whatever it takes to drive the industry forward to give better performance for no extra money is a win-win for us consumers :)

April 18, 2014 | 04:07 PM - Posted by Edkiefer (not verified)

Huh ?

Many were saying these were normal game specific changes, but if so you would not get better % on lower card and lower CPU's .

What Nvidia is doing "works" on all Dx11 app, sure you may not see massive gains but it should help an smooth it out as normally min frame rate is caused by CPU .
Performance improved 2x times on low end HW here .

April 18, 2014 | 05:41 PM - Posted by arbiter

yea AMD seems to dropped doing anything dx wise in games with mantle, very bad for end users if mantle ends up having a serious bug at some point.

April 19, 2014 | 12:11 PM - Posted by Anonymous (not verified)

No just bad for M$'s wallet, and if Nvidia desires to sink along with the M$ rudderless Tub, then so be it, but I suspect, that Nvidia has some API of their own in development, windows 7 is not going away anytime soon, but it will not get the latest DX, anyway! Steam and OpenGL, OpenCL, etc. will also be bad for Redmond's balence sheet, but good for gaming, and gamers! So spin away you M$ Spinion, but the M$ dog has had its day! Time for that little trip out back, behind the shed, for a permanent cure for the OS/Graphics API mange and foaming at the mouth! No one will cry, at this M$ TIFKAM dog's demise.

April 18, 2014 | 04:08 PM - Posted by JohnGR

I would be one of those that insist that these are specific game optimizations. And as mentioned we have seen them from about forever.

That said thanks for the results using the 5350.

April 18, 2014 | 04:35 PM - Posted by Freesome (not verified)

Using the 260x in the tests was a bad idea. In the Mantle test with the Athlon CPU, the 260x is the bottleneck, not the CPU. Using a faster GPU would've showed the real difference Mantle makes with such a low-end CPU.

April 18, 2014 | 05:45 PM - Posted by arbiter

Well its for people with little money to show them what boosts happen. Game yes is pretty cpu bound but not everyone has 2grand to drop on a decent rig, most people seem to be around 1000$ or less. Probably could tossed in say around 100-130$ i3 part just to compare.

April 18, 2014 | 07:37 PM - Posted by Freesome (not verified)

Well, the test configuration isn't really meant then to show how effective Mantle is in reducing CPU usage. Using faster GPUs with the low-end Athlon would've made for much more reliable benchmark.

April 18, 2014 | 04:36 PM - Posted by Anonymous (not verified)

To be fair one has to say that the 337.50 driver from Nvidia is pretty much optimized for Starswarm only, with the gains in other application cases being pretty minuscule (if one does not include the SLI changes to RTW2).

Things look similar with Mantle atm, but there are about two dozen games planed to support Mantle in the near future.

April 18, 2014 | 05:18 PM - Posted by Rick (not verified)

I could be wrong but is this the first nvidia driver to support star swarm? Yes or no, I must admit that it is a nice boost in performance over the 335 drivers. That said you can really see how well mantle scales. The 260x only drops 3fps going from a 6c/12t monster to a 25w 4c CPU. The 750ti's performance dropped almost 50% by changing the CPU.

How about you see how well matle really scales? Take a 290x and run that test again. Will we still see a 3fps drop in rates?

April 18, 2014 | 05:44 PM - Posted by arbiter

Shows how cpu intensive the game is if it dropped that much. But AMD 5350 is super budget cpu, most people in say US won't buy it less building an HTPC or something want to keep low power.

April 18, 2014 | 07:13 PM - Posted by Hameedo (not verified)

Many users are reporting the new driver gave them 80% more performance in Star Swarm with cards like 660Ti, 670, 680 and 780,and CPUs like i7 3770 and 4770.

Please do a 290X vs 780Ti too.

April 18, 2014 | 07:32 PM - Posted by david (not verified)

Using a GeForce GTX 750 Ti and a Radeon R7 260X graphics card, somewhat equivalent in terms of pricing and performance, we ran the Star Swarm stress test benchmark. This application, built originally to demonstrate the performance abilities of AMD's Mantle API.

The equivalent GTX 750 Ti, is Radeon 265.

All we know that radeon 265 , beats de Gtx 750 Ti . I would like to see the test.

April 18, 2014 | 08:05 PM - Posted by Searching4Sasquatch (not verified)

Sweet baby Jesus, I wish these AMD trolls would leave...or at least READ THE F* ARTICLE!

"To be fair one has to say that the 337.50 driver from Nvidia is pretty much optimized for Starswarm only, with the gains in other application cases being pretty minuscule (if one does not include the SLI changes to RTW2)."

DId you read Ryan's previous article? Clearly not as you would have seen 36% gains in Hitman , 31% gains in Thief, 23% gains for Sleeping Dogs, 22% in CoD: Ghosts, and more for SLI. Single GPU gains were also high, with 26% gains in Hitman and 17% in Skyrim.

We get it. You like AMD and get stickers or some other garbage from AMD's "special forum" program. You don't need to keep trolling with this garbage if you aren't even reading PCPer's articles! Go troll at Anandtech. They have plenty of you trolls there already.

April 18, 2014 | 10:33 PM - Posted by Anonymous (not verified)

LOL!!!

Some of those titles you mention aren't even DirectX 11.

April 19, 2014 | 02:59 AM - Posted by Anonymous (not verified)

DX11 shares functions with DX10 and DX9. It's one of the benefits of having a unified and backwards compatible pipeline instead of taking your ball home and making Mantle.

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Lines and paragraphs break automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd> <blockquote><p><br>
  • Web page addresses and e-mail addresses turn into links automatically.

More information about formatting options

By submitting this form, you accept the Mollom privacy policy.