Review Index:
Feedback

Battlefield 4 Mantle Early Performance Testing

Author:
Manufacturer: AMD

A quick look at performance results

Late last week, EA and Dice released the long awaited patch for Battlefield 4 that enables support for the Mantle renderer.  This new API technology was introduced by AMD back in September. Unfortunately, AMD wasn't quite ready for its release with their Catalyst 14.1 beta driver.  I wrote a short article that previewed the new driver's features, its expected performance with the Mantle version of BF4, and commentary about the current state of Mantle.  You should definite read that as a primer before continuing if you haven't yet.  

Today, after really just a few short hours with a useable driver, I have only limited results.  Still, I know that you, our readers, clamor for ANY information on the topic.  I thought I would share what we have thus far.

Initial Considerations

As I mentioned in the previous story, the Mantle version of Battlefield 4 has the biggest potential to show advantages in times where the game is more CPU limited.  AMD calls this the "low hanging fruit" for this early release of Mantle and claim that further optimizations will come, especially for GPU-bound scenarios.  Because of that dependency on CPU limitations, that puts some non-standard requirements on our ability to showcase Mantle's performance capabilities.

View Full Size

For example, the level of the game and even the section of that level, in the BF4 single player campaign, can show drastic swings in Mantle's capabilities.  Multiplayer matches will also show more consistent CPU utilization (and thus could be improved by Mantle) though testing those levels in a repeatable, semi-scientific method is much more difficult.  And, as you'll see in our early results, I even found a couple instances in which the Mantle API version of BF4 ran a smidge slower than the DX11 instance.  

For our testing, we compiled two systems that differed in CPU performance in order to simulate the range of processors installed within consumers' PCs.  Our standard GPU test bed includes a Core i7-3960X Sandy Bridge-E processor specifically to remove the CPU as a bottleneck and that has been included here today.  We added in a system based on the AMD A10-7850K Kaveri APU which presents a more processor-limited (especially per-thread) system, overall, and should help showcase Mantle benefits more easily.

Continue reading our early look at the performance advantages of AMD Mantle on Battlefield 4!!

We have several other platforms setup for testing, with various graphics cards and processors (including a Core i3-4330), but testing will take quite a bit more time to get all of the results we want to present.

Test System 1

  • Intel Core i7-3960X
  • 16GB DDR3-1866 memory
  • ASUS R9 290X DirectCU II

Test System 2

  • AMD A10-7850K
  • 16GB DDR3-2133 memory
  • ASUS R9 290X DirectCU II

 

Testing Methodology

As you might be aware, we use a graphics performance testing methodology called Frame Rating for nearly all of our reviews.  This process uses capture hardware to record the output from the test PC that gets post-processed to evaluate real-world end user experience.  Unfortunately, for Mantle, our tools don't quite work yet. Dice helped us out though by integrating a new feature with the latest BF4 patch to enable frame time logging directly through the in-game console.

View Full Size

PerfOverlay.FrameFileLogEnable 1 starts recording frame times to a CSV file.  Setting that variable to 0 will stop the recording.  All log files will find their way into the My Documents/Battlefield 4 folder.  Even better, because this is done at the engine level, we can get some interesting information from it.  Not only is the frame time for each display frame recorded,  we also get the GPU and CPU time for each frame as well.  This is interesting as it allows us to see how much time each individual component spent on each frame to check for bottlenecks on both ends.

THIS. IS. BETA!!

I'm sure you have all used beta Catalyst drivers before.  These are much different.  I might go as far as to consider these drivers, and the Mantle libraries themselves, in an "alpha" state.  There is a pretty sizeable list of known issues which AMD is being very upfront about.  Its includes the following:

  • Intermittent stuttering or stability issues may occur when utilizing Mantle with AMD CrossFire technology in BattleField 4
  • Mantle performance for the AMD Radeon HD 7000/HD 8000 Series GPUs and AMD Radeon R9 280X and R9 270X GPUs will be optimized for BattleField 4 in future AMD Catalyst releases. These products will see limited gains in BattleField 4 and AMD is currently investigating optimizations for them.
  • Notebooks based on AMD Enduro or PowerXpress technologies are currently not supported by the Mantle codepath in Battlefield 4.
  • AMD Eyefinity configurations utilizing portrait display orientations are currently not supported by the Mantle codepath in Battlefield 4.
  • Graphics hardware in the AMD A10-7850K and A10-7700K may override the presence of a discrete GPU under the Mantle codepath in Battlefield 4.
  • AMD testing for the AMD Catalyst 14.1 Beta has been concentrated on the following products: AMD Radeon R9 290X, R9 290, R9 280, R9 270, R7 260X, R7 260, HD 7000 Series, HD 8000 Series, A10-7850K and A10-7700K. Future AMD Catalyst releases will include full test coverage for all AMD products supported by Mantle.

The CrossFire stability issue, at least in my experience, is not a minor thing as I was never able to get it working properly on either an X79 platform or the A88X + Kaveri system.

In several talks with AMD this week I did express concern over this very early beta status of Mantle.  My fear is that with AMD having difficulty getting Mantle working with a single gaming title (Battlefield 4 today) after having a full month of extra time (remember the initial promised December release) does not bode well for Mantle in the long term.  AMD tells me that Mantle has been in the works for more than two years which should be enough to perfect the platform for these limited variables. It is possible that these growing pains of stability and performance are a one time occurrence to go along with a new graphics API.  But, once several or a dozen games are available using Mantle, how will AMD's support and development teams be able to handle it?

A Quick Note on Image Quality

I know that some of you are curious about the image quality comparisons between DX11 and Mantle codepaths.  Though I didn't do any testing in this story I did go to the man himself to ask the question.  Johan Andersson, Technical Director on Frostbite at Electronic Arts, told me quite clearly that "no, there shouldn't be any difference" in image quality between the two options in Battlefield 4.  I don't usually just take people at their word, but he is an exception!

Worst Case Scenario

Our standard level and section used for testing Battlefield 4 is the opening portions of the single player SP_Dam stage.  As it turns out, this section is quite GPU limited (as we would normally want for our GPU testing...) but this doesn't fair well for AMD's Mantle out of the gate.  Here are results with the Core i7-3960X and R9 290X at 2560x1600 with the Ultra preset.

View Full Size

Click to Enlarge

View Full Size

Click to Enlarge - Note the incorrect labeling here, it is NOT 1920x1080 but in fact 2560x1600

Results from both the Mantle version and the DX11 version are essentially the same here, averaging about 43 FPS.  That's a bit disappointing but, after the preparation from AMD about setting expectations, it more or less makes sense.  This is a high-end processor in a section of the game that is very GPU limited and thus we didn't expect to see a dramatic advantage for the Mantle API.  We did however expect to see SOME KIND of advantage...

View Full Size

Click to Enlarge

View Full Size

Click to Enlarge

If we look at the CPU time graph though from these two runs, it is clear that BF4's Mantle implementation is having an effect.  Notice how much more consistent the CPU time is on the Mantle test run without any kind of spikes of CPU usage.  Our Core i7-3960X is definitely being utilized more efficiently although, in this particular combination of hardware and level, it just doesn't result in any performance advantages.

February 1, 2014 | 11:04 PM - Posted by jebo (not verified)

It's always great to get "free" extra performance, and a 30% increase is certainly impressive. But how many other hand developers will take the time to implement mantle when its benefits are limited to so few users/use cases?

I am however very interested to see how mantle impacts the mid/low range. For example I have a laptop with a Radeon 7750 and i7 CPU. Will it see any benefit?

February 1, 2014 | 11:07 PM - Posted by arbiter

But that 30% is if you have a a10 cpu with a r9 290, yea 170$ that is pretty bit bottle neck for a gpu that is 600+$ gpu. Reason this is a problem is only get this in game with mantle, which there won't be many games with it for a while if any other dev's decide to support it.

February 2, 2014 | 04:45 AM - Posted by JohnGR (not verified)

Few users? You serious? Most people have a dual core or quad core processor, and in many cases an old dual/quad core processor. If Mantle does good and doesn't look like an alpha/beta forever, in the future AMD gpus will be the best option for everyone with a low/mid range system except if Maxwell is a low power consumption/high performance beast.

February 1, 2014 | 11:11 PM - Posted by puppetworx (not verified)

Interesting results; mostly contrary to what I have seen elsewhere but also far more detailed. The majority of the testing that I have seen which gives Mantle the advantage has been at 1080p not 1600p, perhaps that is where the difference lies?

February 2, 2014 | 12:57 AM - Posted by Anonymous (not verified)

Yeah really. Every other review of Mantle out there has shown gains from 10% to as high as 40% depending on configuration. Reviews using 4670K, 3970X, 4770K paired with 290 or 290X all showing those 10 to 40% gains.

http://www.computerbase.de/artikel/grafikkarten/2014/erste-eindruecke-zu...
http://www.golem.de/news/amds-mantle-api-im-test-der-prozessor-katalysat...
http://www.legitreviews.com/amd-mantle-api-real-world-bf4-benchmark-perf...
http://www.legitreviews.com/amd-mantle-api-real-world-bf4-benchmark-perf...

here at pcper nothing and no gains. laughable but to be expected from pcper i suppose. maybe the reviewer just suffering from serious PEBKAC.. lol

February 2, 2014 | 12:58 AM - Posted by Anonymous (not verified)

more

http://anandtech.com/show/7728/battlefield-4-mantle-preview

February 2, 2014 | 02:40 PM - Posted by Jacob (not verified)

You're sufferring from serious "can not read" issues, the single player results are inline with most of what i read online, the multiplayer results, with mp being much more cpu intensive show performance gains of 27% and 32% which certainly make sense.

February 2, 2014 | 01:07 AM - Posted by Ryan Shrout

I clearly show performance gains in our multiplayer testing. And I discuss how our specific single player testing location is a worst case scenario. Did you rtfa?

February 2, 2014 | 03:02 AM - Posted by puppetworx (not verified)

I read it but I did miss that about the single player being at the very worst part of the game for GPU, that explains it then. Considering how most other sites are using different sections of the game this is valuable data to have.

For the record I wasn't complaining or criticising, I was curious and I specifically referenced singleplayer, not multiplayer, because that's where I saw the difference between your review and others. Now I understand why.

February 2, 2014 | 03:33 AM - Posted by puppetworx (not verified)

Actually I see I didn't explicitly reference singleplayer, but since you tested SP at 1600p and MP at 1080p that's what I implied, that's probably where the misunderstanding was.

February 2, 2014 | 03:11 AM - Posted by Anonymous (not verified)

Ryan how did you record framerate?
Fraps doesnt work with Mantle nor any other benchmarking software...

February 2, 2014 | 05:37 AM - Posted by snook

ryan said: "PerfOverlay.FrameFileLogEnable 1 starts recording frame times to a CSV file. Setting that variable to 0 will stop the recording. All log files will find their way into the My Documents/Battlefield 4 folder."

it's in the article.

February 2, 2014 | 07:22 AM - Posted by Anonymous (not verified)

Why are you talking about something you are clueless about?
That command is for measuring frametimes not frame rate.
It s in the article. :)

February 2, 2014 | 08:30 AM - Posted by Ryan Shrout

Once you have the frametimes is quite easy to calculate frame rates using basic math.

February 2, 2014 | 10:48 AM - Posted by snook

Mr. clueless to you. :P as ryan says above, it can be calculated. or as a clueless man such as myself would do, i would just hit "~" and type perfoverlay.drawfps 1.

your turn :0

February 2, 2014 | 11:33 AM - Posted by Ryan Shrout

Once we have the individual frame times calculating the frame rate is pretty simple.

Add up all the frame times until they equal around 1000 (for 1000 ms).  Then divide by the number of frames.  OR average all the frame times and then divide that into 1000.  Ta da.

February 2, 2014 | 03:17 AM - Posted by Anonymous (not verified)

Don't get upset with him. Be upset at yourself Ryan for being lazy.

Other sites provided vastly more information and setups then these two you provided.

Our best case scenario resulted in huge performance gains for the AMD hardware when tested on multiplayer maps that have a lot more work for the CPU to do. 27% and 32% average frame rate increases are clearly a very important data point for AMD although you have to wonder, based on our two wildly different result sets, which of these is going to be the norm and which will be the outlier.

More than likely the truth lies somewhere in the middle.

If your just providing very limited data to your readers and other sites provide a great more setup scenarios and data what do you expect when your article is late and extremely lacking.

February 2, 2014 | 08:31 AM - Posted by Ryan Shrout

Sigh. The claim about "being lazy" is just about the WORST thing I could have read here.  In truth we have about 8-10 hours of testing that we DID NOT include for other hardware setups thanks to AMD's last minute changes to the driver, etc, including not recommending multi-GPU configs.  Also, we had set aside many hours on Wednesday for testing, but didn't get a driver that was promised.  Then we set aside time on Thursday, same result.  Same on Friday, but we didn't get a driver until 6pm local time.  

And it may surprise you but I have a family and stuff.

February 2, 2014 | 12:11 PM - Posted by snook

lol, you have the patience of Job, Ryan.

i would have strangled mantle "out of the box".

February 2, 2014 | 03:42 PM - Posted by Anonymous (not verified)

I could have never guessed you had a family from the maturity shown in your live streams shows.

My condolences to your spouse. He/She deserves it if your throwing them under the bus for a lousy article.

Strange that I haven't read any bickering in other articles who've setup multiple hardware scenarios.

Your maturity and professionalism shines through alright. Stay classy.

It's your job after all.

February 2, 2014 | 09:40 PM - Posted by snook

lol, you have lost the argument/debate when personal attacks against his treatment of spouse and children are your only play.

you know less about that than you did about his time spent testing mantle. congrats on being a zero sum man/woman.

February 2, 2014 | 10:20 PM - Posted by Anonymous (not verified)

In fairness the guy was pointing out something ryan already had said himself. Ryan shouldn't have used it as an excuse if he didn't want people pointing it out.

Bad all around.

February 3, 2014 | 05:21 AM - Posted by snook

perhaps, but he spoke from a position that intimated ryan was a poor husband and father because of comments/actions ryan has made. which he couldn't possibly know.

I could, however, point out that the posters mother didn't teach him manners and lacked them herself and that any woman that is around him suffers due to his mothers incompetence, i would be closer to truth than he was.

and it's always best to clean your own house first, prior to pointing out the dirt in someone else's.

your points are valid in that if he puts something out there he could get it back in spades.

February 2, 2014 | 09:48 PM - Posted by pdjblum

Perhaps the only thing juvenile Ryan did was respond to your post, as absurd and malicious as it was. The ironic thing is that I cannot recall one time where Ryan responded to a post from me with a direct question to him, despite those being nice posts. Ryan, maybe you should be more inclined to respond to neutral or positive legitimate posts rather than negative, illegitimate posts.

February 3, 2014 | 05:26 AM - Posted by snook

I have had the opposite experience. both ryan and josh have responded on several occasions to my questions (mine are noobishly stupid inquiries mostly) so they may feel sorry for me. I encourage you to keep asking, they will respond eventually.

February 3, 2014 | 01:28 PM - Posted by pdjblum

ok, thanks snook, but it has been really frustrating.

February 3, 2014 | 01:54 PM - Posted by snook

I hear ya. follow them on twitter if you use it. every question Ive asked there gets a quick response generally.

February 3, 2014 | 11:34 AM - Posted by Trav (not verified)

...you go through the effort of insulting him...but do it anonymously???
That sir...makes you a PUS%Y!!! .....BHAHAHAHAAHAAAHAAAA!!!

February 3, 2014 | 01:55 PM - Posted by snook

were you able to note if mantle was leveraging the iGPU along with the discrete GPU? or am i getting mantle and HSA confused?

February 1, 2014 | 11:14 PM - Posted by DPM (not verified)

I have an FX-8350 and HD7850 so I'm not an enthusiast by any means. I am personally excited about these results because it may extend the length of time between GPU upgrades. Even then, I might get a few GPU upgrades in before I upgrade the CPU since the CPU bottle-necking is something that Mantle seems to handle well. The amount of work that has gone into this must have been daunting. I mean, running a AAA title without DX11 must require thousands of man-hours of development time.

As others have said, free performance is awesome. Although I would like to know more about the developer implementation. Is it painfully complex? Does it require specialists, training, or partnering with AMD developers? In other words, is Mantle something that will be a chip-shot for developers to add in for great performance returns?

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Lines and paragraphs break automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd> <blockquote><p><br>
  • Web page addresses and e-mail addresses turn into links automatically.

More information about formatting options

By submitting this form, you accept the Mollom privacy policy.