Feedback

Frame Rating Part 3: First Results from the New GPU Performance Tools

Author:
Manufacturer: PC Perspective

In case you missed it...

UPDATE: We have now published full details on our Frame Rating capture and analysis system as well as an entire host of benchmark results.  Please check it out!!

In one of the last pages of our recent NVIDIA GeForce GTX TITAN graphics card review we included an update to our Frame Rating graphics performance metric that details the testing method in more detail and showed results for the first time.  Because it was buried so far into the article, I thought it was worth posting this information here as a separate article to solict feedback from readers and help guide the discussion forward without getting lost in the TITAN shuffle.  If you already read that page of our TITAN review, nothing new is included below. 

I am still planning a full article based on these results sooner rather than later; for now, please leave me your thoughts, comments, ideas and criticisms in the comments below!


Why are you not testing CrossFire??

If you haven't been following our sequence of stories that investigates a completely new testing methodology we are calling "frame rating", then you are really missing out.  (Part 1 is here, part 2 is here.)  The basic premise of Frame Rating is that the performance metrics that the industry is gathering using FRAPS are inaccurate in many cases and do not properly reflect the real-world gaming experience the user has.

Because of that, we are working on another method that uses high-end dual-link DVI capture equipment to directly record the raw output from the graphics card with an overlay technology that allows us to measure frame rates as they are presented on the screen, not as they are presented to the FRAPS software sub-system.  With these tools we can measure average frame rates, frame times and stutter, all in a way that reflects exactly what the viewer sees from the game.

We aren't ready to show our full sets of results yet (soon!) but the problems lie in that AMD's CrossFire technology shows severe performance degradations when viewed under the Frame Rating microscope that do not show up nearly as dramatically under FRAPS.  As such, I decided that it was simply irresponsible of me to present data to readers that I would then immediately refute on the final pages of this review (Editor: referencing the GTX TITAN article linked above.) - it would be a waste of time for the reader and people that skip only to the performance graphs wouldn't know our theory on why the results displayed were invalid.

Many other sites will use FRAPS, will use CrossFire, and there is nothing wrong with that at all.  They are simply presenting data that they believe to be true based on the tools at their disposal.  More data is always better. 

Here are these results and our discussion.  I decided to use the most popular game out today, Battlefield 3 and please keep in mind this is NOT the worst case scenario for AMD CrossFire in any way.  I tested the Radeon HD 7970 GHz Edition in single and CrossFire configurations as well as the GeForce GTX 680 and SLI.  To gather results I used two processes:

  1. Run FRAPS while running through a repeatable section and record frame rates and frame times for 60 seconds
  2. Run our Frame Rating capture system with a special overlay that allows us to measure frame rates and frame times with post processing.

Here is an example of what the overlay looks like in Battlefield 3.

View Full Size

Frame Rating capture on GeForce GTX 680s in SLI - Click to Enlarge

The column on the left is actually the visuals of an overlay that is applied to each and every frame of the game early in the rendering process.  A solid color is added to the PRESENT call (more details to come later) for each individual frame.  As you know, when you are playing a game, multiple frames will make it on any single 60 Hz cycle of your monitor and because of that you get a succession of colors on the left hand side.

By measuring the pixel height of those colored columns, and knowing the order in which they should appear beforehand, we can gather the same data that FRAPS does but our results are seen AFTER any driver optimizations and DX changes the game might make.

View Full Size

Frame Rating capture on Radeon HD 7970 CrossFire - Click to Enlarge

Here you see a very similar screenshot running on CrossFire.  Notice the thin silver band between the maroon and purple?  That is a complete frame according to FRAPS and most reviews.  Not to us - we think that frame rendered is almost useless. 

Continue reading our 3rd part in a series of Frame Rating and to see our first performance results!!

 

View Full Size

Data gathered from FRAPS

Here is a typical frame rate over time graph as generated by FRAPS.  Looks good right?  CrossFire and SLI are competitive with the advantage to the HD 7970s.

View Full Size

Data gathered from Frame Rating Capture

This is the same graph with data gathered from our method that omits RUNT frames that only represent pixels under a certain threshold (to be discussed later).  Removing the tiny slivers gives us a "perceived frame rate" that differs quite a bit - CrossFire doesn't look faster than a single card.

 

View Full Size

Data gathered from FRAPS - Click to Enlarge

This is the raw frame times as captured by FRAPS - again we are looking for a narrow band of frametimes to represent a smooth experience.  Both single cards do pretty well, but SLI sees a bit more variance and CrossFire sees a bigger one.  Quite a bit bigger.

View Full Size

Data gathered from Frame Rating Capture - Click to Enlarge

Here is the same data gathered by our new capture system - the CrossFire configuration looks MUCH worse with many frames hitting near 0ms of screen time.  That would be great if they were ALL like that but unfortunately they also scale up to 20ms and higher quite often.  Also notice NVIDIA's is actually MORE uniform indicating that there is some kind of smoothing going on after the frame leaves the game engine's hands. 

 

View Full Size

Data gathered from FRAPS - Click to Enlarge

Let's zoom in a bit - here is 100 frames of the FRAPS frametimes from above.  Notice the see-saw effect that the CrossFire output has...

View Full Size

Data gathered from Frame Rating Capture - Click to Enlarge

And see how much worse it is here in our Frame Rating Capture configuration.  The pattern is actually exaggerated on the CrossFire solution while the SLI configuration is smoother. 

 

Here are a couple more screenshots from our captures.

View Full Size

Frame Rating capture on Radeon HD 7970 CrossFire - Click to Enlarge

 

View Full Size

Frame Rating capture on GeForce GTX 680s in SLI - Click to Enlarge

 

View Full Size

Frame Rating capture on Radeon HD 7970 CrossFire - Click to Enlarge

 

View Full Size

Frame Rating capture on GeForce GTX 680s in SLI - Click to Enlarge

 

This is just a preview of what we have planned for our new Frame Rating Capture performance testing method.  We have gone through many games with this and the results can vary from looking better than FRAPS to looking much, much worse. 

I am eager to get your feedback - please feel free to leave comments below and the follow on to the conclusion of our GeForce GTX TITAN review!

 

February 22, 2013 | 12:14 PM - Posted by Stefen Heiff (not verified)

I like the way you use hardware-based methods to get framerates and frame times. It's seems to be the best setup to evaluate performance instead of software that will use math and averaging to output a number.

March 28, 2013 | 04:33 PM - Posted by Anonymous (not verified)

Oh good, the pathetic nature of AMD's video cards using the new method is exposed finally - just a litle bit.

LOL - I wonder how many goodies AMD promised to keep this under the radar as long as possible while they SCRAMBLE to find a fix... wow those TOP END meet and discus DINING MEALS must have been just awesome and exquisite...
Def jelly.

Anyway, I'm glad AMD is exposed for the garbage that it is - this just shows to me why there are 100 other driver problems with AMD cards, while nVidia is near perfect 100% of the time.

I don't expect the radeon ragers to ever admit it, though.
AMD will die and they will collapse into a puddle of delicious forever tears and blame nVidia and their evil plots... and the "stupid public" "conned by advertising"... but NEVER the results of the above test and those like it we have had for half a decade now.

I do wonder how AMD gets away with it for years at a time before they are exposed... how shameful it must be when they finally realize they have been crap again for half a decade.

February 22, 2013 | 01:33 PM - Posted by gamerk2 (not verified)

Great review. Keep it comming guys.

February 22, 2013 | 01:51 PM - Posted by Aurhinius (not verified)

Great stuff just want to see AMD's response to their crossfire performance!

February 22, 2013 | 02:04 PM - Posted by I don't have a name. (not verified)

Excellent stuff. Thank you for the real insight. :)

February 22, 2013 | 02:39 PM - Posted by serpinati of the wussu (not verified)

dude, that is crazy! You think the amd cards in crossfire just synchronize poorly to produce this? Or maybe there's a conspiracy going on (cue x-files theme).

March 28, 2013 | 04:45 PM - Posted by Anonymous (not verified)

Yes the conspiracy has been to cover up, forgive, not mention, minimize, and give a big fat break to AMD every time, all the time.

That has produced this pathetic result, and a hundred other unresolved AMD vid car issues that carry on forever, maybe getting fixed a month or two before obsolescence, if then.

So this is very good news - the other single time AMD got jumped was on "crossfire drivers" @ hardocp - they have Terry Maked0n "Catalyst Maker" top dog in and he typed it: " I didn't even know we had a problem".

Yes, he didn't know - or he lied, the CF drivers shortly thereafter TOOK A GIANT LEAP FORWARD AND SURPASSED nVidia for the 1st time ever.

So somehow, AMD insulates all it's engineers ( the few they have left ), all it's PR people, all the disappointed users on their tyranny managed lock and delete "help" website, and all that secret no monkey see hear or taste total blackout leaves them blowing forevermore.

When something finally cracks through the impenetrable blackness of contact with the real world, they actually fix it ! (Well, maybe only if they get a LOT of help from end users replicating problems and sending in their cards and systems and wasting their lives while not getting paid).

Yeah, man amd so blows, hope they die soon.
(maybe their hardware (or engineers) are only meant for the direct run on hardware code that the gaming consoles use, and when it gets extrapolated off direct native hardware into a PC and for gaming, it will never be done correctly).

They need to die, be bought off, a then managed properly, without the lalalallalalala fingers in their ears method.

February 22, 2013 | 02:57 PM - Posted by Anonymous (not verified)

You should try this method on a console!

February 22, 2013 | 03:11 PM - Posted by Ryan Shrout

Only possible if we can get some kind of developer kit and run the overlay application on it as well.

Much more complex.  :/

February 22, 2013 | 03:15 PM - Posted by mateo (not verified)

Ok I'll repost here.

Does turning Vsync ON, fixes CF presenting issues.
If it does, is the performance of Vsycned CF in line when compared to Vsync OFF results.

Also does the Vsynced sysetm exhibits frame skipping like with RadeonPro "smoothing"?

And I'm not asking specifically about CF alone, but SLI and single GPUs also.

February 22, 2013 | 03:17 PM - Posted by Ryan Shrout

Yes, Vsync would usually cause frames to be "dropped" from presentation.  

We are going to be doing Vsync enabled testing for our full article as well.

February 22, 2013 | 03:22 PM - Posted by mateo (not verified)

Awesome.
I won't have to flip back and forth through 10 reviews.
PC Perspective is going to be The Mother Of All Reviews And Benchmarks.

Now the question is... WHEN

February 24, 2013 | 03:28 AM - Posted by Anonymous (not verified)

But dose it also change the numbers, and give a more realistic view of whats happening?

February 22, 2013 | 03:57 PM - Posted by Nilbog

Do you plan on explaining what exactly the overlay colors mean, etc? Maybe i missed something, going to go back and re-read the other articles just in case.

It would also be cool if you also compared last generation cards, like the 580, etc. Or maybe that's just too many cards.

I also wanted to ask why exactly you aren't testing CrossFireX? I understand that these testing methods don't put the cards in the best light. However if that's how they are performing then it should be noted. It is important to me that these kind of things are brought up, in order to encourage better support, and properly inform consumers.

I have the exact same issues with my 6970 CrossFireX setup.
After you posted the first Frame Rating preview, i have started looking more closely at my frame times as well.
This is clearly an AMD driver issue, and has been an issue for a while.
I personally think this should be publicized more until they actually fix these issues for all relevant cards, not just the 7000 series. Smooth gameplay is a big deal IMO.

February 22, 2013 | 05:01 PM - Posted by Brokenstorm (not verified)

If I understand correctly, the color overlay is simply a sequence of colors to indicate if any frame were dropped.

February 22, 2013 | 04:24 PM - Posted by kENNY (not verified)

Can you please change the colors of all your charts to be visible by color blind readers?

February 22, 2013 | 05:54 PM - Posted by Ryan Shrout

Hi Kenny, I completely apologize for that.  I didn't think about it all but I'll try to find out more on the topic and pick some better colors next time.

I had selected red and green to represent AMD and NVIDIA.  :)

February 22, 2013 | 07:32 PM - Posted by Dan (not verified)

Damn Ryan, this rocks.

February 22, 2013 | 10:16 PM - Posted by Anonymous (not verified)

You guys are so full of Sh*t with these benchmarks. Even TR is saying Xfire is fixed, you guys and anther biased site is claiming these absurd results for reasons we know well (Hits & slandering AMD).. I am positive nvidia have purchased you guys from a while back bcoz you guys have consistently bashed AMD and praised nvidia.. you even praised the 1000 card lol

I just hope AMD gives you guys what you deserve... Hopefully before next AMD gen cards (No Cards for you anymore)

February 22, 2013 | 10:56 PM - Posted by Kyle Riste (not verified)

What is wrong with you have you even read the previous articles and how much money and time and programming went into getting this right and making sure that gfxcard companies are actually doing what they advertise and not some b/s. PcPer is calling these companies on their crap and its a service to us the consumers.

February 23, 2013 | 06:00 PM - Posted by crikeyson (not verified)

I agree the poster you are replying to is over the top in his statements.

I do feel though that PCPer is definitely nvidia biased and affiliated to a certain extent. Not that I would completely discount these results. There is obvious nvidia partiality from this website though and I hear these sentiments echoed on many tech forums.

If he were truly 'looking out for consumers' he would of crucified the Titan for being nothing beyond the genuine GTX 680 with its priced doubled, yet he praised it and went so far as to have an nvidia PR man shilling on his website in a live stream.

Yeah, clearly not biased.......

Look forward to more investigations about this frametime business, but I do realize many users have been using crossfire and SLI for years now and it's only a few tech sites that are screaming about this issue, while the end-users have been quite happy with their experiences.

Maybe if this site was not so connected to nvidia it would be more palatable to take these results at face value. But with it having nvidia marketers on the site trying to sell a huge rip-off card like Titan, along with a Titan review not once really laying waste to it being a massively overpriced VGA in relation to history and performance...

well, it gets much more difficult not to take this all as a shill.

March 1, 2013 | 10:10 AM - Posted by DrShuey (not verified)

The funny thing is, he had AMD on a couple of days before he had nVidia on. He has done a ton of live stuff with AMD including his next event on 3/5.

March 6, 2013 | 07:15 PM - Posted by anon (not verified)

Do your homework before making a subjective statement. Accusing pcper of giving more positive reviews for Nvidia cards based on pure preference and not user experience is a grievous error.

Before there was pcper.com there was amdmb.com - they were the same site and it was an entirely AMD based review site. amdmb.com evolved into pcper.com to widen its user base and stay relavant. pcper is still evolving and atte4mpting to remain at the tip of the spear by including reviews of tablets and phones occasionally. There is nothing more cerain than change and hopefully pcper will stay ahead of the curfe.
Many of the pcper old hats are still AMD die hards and prefer AMD because they prefer the underdog - especially in the AMD vs. Intel side of things. When ATI was purchased by AMD, many saw this as a great opportunity for the German based company to continue to develop enthusiasts class chips. The further split of AMD's fabs into a separate company, Global Foundries, gave some hope to many that the underdog would once again be able to develop an x86 chip with performance exceeding that of Intel.

The slanderous statements are entirely incorrect -

HyperMinimalism

March 28, 2013 | 04:51 PM - Posted by Anonymous (not verified)

AMD fans are so used to being given a thousand breaks at every website but two, when AMD is kissed gloriously then finally the "bad data that was held back for AMD's sake" makes it to a webpage, the fanboys go crazy...

If you're coddled forever, slightly less coddling is considered unforgivable. That's the real story with AMD and all the website save 2 of them.
If you know anything, you know who those 2 websites are. They don't coddle AMD and baby it and give it endless breaks.

February 23, 2013 | 01:01 AM - Posted by ThorAxe

Appearing to be fixed on limited data is not the same as actually being fixed.

Have you not seen the contests PC Per have done? The majority have been sponsored by AMD?

Take your head out of the sand.

March 28, 2013 | 04:54 PM - Posted by Anonymous (not verified)

One would think a proper AMD fan reaction ( if said person was sane ) would be along the lines: " So I've been gettig screwed all this time ?!!?? What took so long for this to be exposed! Why is AMD "fixing it" finally while claiming "it's not really a problem" ? "

But no, the amd fan merely hates the facts, and is happy no matter how pathetic the amd video card is. They must drop hundreds of Verdetrol pills a day. ( google it ! LOL )

February 23, 2013 | 01:28 AM - Posted by tackle70 (not verified)

TR is not saying xfire is fixed... not by a long shot. Check out their Titan review and watch how 7970 CF looks awesome on average framerate and not so much on 99th % frametime

February 23, 2013 | 05:39 AM - Posted by Stephen H (not verified)

Hi Anonymous (if that's your real name) - I'm going to guess you disagree with the data. Can you please explain on what basis you disagree, including references where possible to data supporting your argument?

Alternatively, come back in a few years once you've made it past puberty and can hopefully make meaningful and useful comments.

Mr Shrout, I came to this article after hearing you on This Week in Computer Hardware over the last few weeks. You have provided very good descriptions in that podcast, and here. I like that PC Perspective is putting some time and thought into what the end user really sees, rather than just going with "average fps". I haven't had the problem lately, but a few years back graphical stutter was the bane of the PC gamer's life.

Have you got any thoughts on why you get this effect with Crossfire?

March 1, 2013 | 09:14 AM - Posted by cyberwire

Have you even looked at this site before? There is almost always a AMD banner or ad somewhere for their video cards. Stop fanboying behind the anonymous tag and join in the discussion and learn a thing or two.

February 23, 2013 | 01:18 AM - Posted by Circuitfeak (not verified)

Have you guys tried changing the Crossfire rendering method to see if it changes your graphs? I know SLI used AFR style while AMD uses the checkerboard style rendering. Just wondering if that is having any effect.

February 23, 2013 | 03:28 AM - Posted by serpinati of the wussu (not verified)

are you sure about this? I don't have crossfire or SLI, but from what I understand, both default to AFR and you don't really have any option to change it unless the programmers made a specific profile to change it.

This would also make sense since microstutter is really only present in AFR and not in other modes.

February 23, 2013 | 01:55 PM - Posted by Circuitfeak (not verified)

http://techreport.com/review/8826/ati-crossfire-dual-graphics-solution/3

Check this out.

February 23, 2013 | 09:07 PM - Posted by serpinati of the wussu (not verified)

I'm not sure that's relevant anymore, since they are discussing ati x800's, which are pretty old cards.

I dont believe any modern drivers give you the option to switch modes (again, please someone correct me, i dont have SLI or crossfire) and I'm pretty sure they all default to AFR if no profile is found.

This also wouldn't be surprising because AFR usually gives the highest reported FPS, so companies would usually want it to default to this for reviewing and benchmarking purposes.

February 24, 2013 | 12:23 AM - Posted by Circuitfeak (not verified)

You can switch modes in CCC for AMD and Nvidia SLI control panels, most games and benchmarks just have predefined profiles.

February 25, 2013 | 06:28 PM - Posted by kn00tcn

CF uses AFR, it's even listed in the CF best practices documentation pdf as default & what devs should adjust their engine to

plus if you look at the CAP xml, AFR is listed constantly

i have forced the other modes with radeonpro in the past, it did not go so well, from artifacts to a hard lockup that killed my whole GFWL profile

February 23, 2013 | 01:27 AM - Posted by tackle70 (not verified)

You pretty much have to use a framerate limiter with crossfire. It's not perfect, but you can remove stutter with it entirely.

Anytime you have an average fps of your refresh rate or higher, you set the framerate limit to your refresh rate. Otherwise, you do it somewhere around your average fps (I typically find smoother play if I limit the framerate to just under the average fps).

My tool of choice to do this is Afterburner but Radeon Pro can do it too.

February 23, 2013 | 08:53 AM - Posted by Seravia (not verified)

Yes, PCPerspective should have a look, because we've been using this for more than a year and RadeonPro made it easier:
http://www.tomshardware.com/reviews/radeon-hd-7990-devil13-7970-x2,3329-...

February 23, 2013 | 11:43 AM - Posted by tackle70 (not verified)

I've honestly found MSI Afterburner's framerate limiter a much better option than Radeon Pro. It more or less accomplishes the same effect but without having to make a profile for each game you want to play.

Also, Radeon Pro doesn't always work with all games e.g. it's currently not working with Crysis 3.

Occasionally for whatever reason Afterburner won't solve the problem, and then Radeon Pro can be a nice backup option.

February 23, 2013 | 05:45 AM - Posted by BiggieShady

So the FRAPS shows how CrossFire is bad, but the real truth is even worse - FRAPS was actually doing CrossFire a favor. It would almost be funny if it wasn't so sad.

February 24, 2013 | 03:26 AM - Posted by Anonymous (not verified)

Could be that FRAPS with Vsysnc on actually give a more honest representation of whats happening.

Have to wait till the full review, as Ryan is not real clear on that.

March 28, 2013 | 05:00 PM - Posted by Anonymous (not verified)

Yes, and all we heard from AMD was fraps is not showing how good we really are, and "nVidia agrees with us on this".

L O L

AMD is the epitome of the disgusting lying corporate crime structure we all know so well since the '08 crash.

A bailout no doubt is in order, it's "too important" to the fanboys to fail, so continuous failure is preferred.

March 28, 2013 | 05:00 PM - Posted by Anonymous (not verified)

Yes, and all we heard from AMD was fraps is not showing how good we really are, and "nVidia agrees with us on this".

L O L

AMD is the epitome of the disgusting lying corporate crime structure we all know so well since the '08 crash.

A bailout no doubt is in order, it's "too important" to the fanboys to fail, so continuous failure is preferred.

February 23, 2013 | 08:48 AM - Posted by Seravia (not verified)

http://www.tomshardware.com/reviews/radeon-hd-7990-devil13-7970-x2,3329-...

February 23, 2013 | 08:50 AM - Posted by Angry

I understand wanting to show the frame time results, but why not include the raw fraps numbers as well. That would satisfy both crowds at the same time.

February 23, 2013 | 01:46 PM - Posted by dreamer77dd

This is exciting and i wonder if any of the companies will be using this software or helping the community out to test there hardware out even more.
Like if your having a problem they will support you and explain more why their cared is not performing.

In the future will GPU companies be optimizing their cards like they do with other benchmark to sell more or show off the performance?
I wonder, hmm.

March 28, 2013 | 05:17 PM - Posted by Anonymous (not verified)

You can be certain nVidia already had hardware analyzing this very thing, since they have a Smoother, and have talked about it.

AMD on the other hand said they had no idea, and "everyone suffers from frame time differences and stuttering" meaning they had given up before trying, were incapable of even checking and only got on the stick after being thoroughly exposed FIVE YEARS TOO LATE.

Years of AMD suckage under our belts, of course, for now, it's time for the amd fanboys to claim it doesn't matter while simultaneously claiming it has been fixed and also claiming "amd does best in this area".
I have indeed seen all 3 of those all the time now.

February 23, 2013 | 02:30 PM - Posted by RagingCain (not verified)

Hey guys, I am a member of the Overclock.net community, is there any chance you could bring your results, after pcper publication, and discuss with some of us more technically inclined on our forums?

I have been unfortunate myself in that I a seem to suffer the super power of feeling the affects of high latency frame rate and have been feeling it since release of 7970s and the entire 69xx generation.

What I would like to do is share some of the techniques I, and many others use, in overcoming this issue and alleviating it. I would love to see the physical results from your testing methodology and see if perhaps what we do actually alleviates, worsens them, or placebos our gaming experience.

Feel free to PM me any time on that site.

February 26, 2013 | 05:07 PM - Posted by Ryan Shrout

Absolutely.  Send me an email: rshrout ==at== pcper.com

February 25, 2013 | 03:57 AM - Posted by Optimis0r (not verified)

Wow PC per. Just wow... top job keep it up. I think what you guys are doing is superb. A+++

February 25, 2013 | 05:17 AM - Posted by Orthello77 (not verified)

Owning two radeon 7970s i can only feel very dissapointed in seeing these results.

I mean wtf is going on with AMD and the driver team .. the runt frames at first glance look like attempt to falsify the fps readings , but then a deeper look into the single card performance will tell you that AMD don't need the runt frames to beat nvidia in this game. They just need the same scaling without the runt frames and a more even frame delivery.

Hopefully someone at AMD will see this. It would be great to see how this looks with radeon pro and dynamic frame control .. im sure it would almost fix it completely.

Can pcper please include radeon pro or afterburner frame limit in these types of reviews to see if it fixes the issue .. i mean nobody i know games without vsync anyway.

February 25, 2013 | 09:10 AM - Posted by tackle70 (not verified)

Just use a framerate limiter. I use afterburner and limit framerate to 60 and it's fine.

I've got 7970s as well and nearly all of stutter is completely removable if you just play with framerate limiting functionality.

March 28, 2013 | 05:27 PM - Posted by Anonymous (not verified)

Thus the "wins in frame rates" is a total joke.

I can't stand amd or their fans. AMD is better so long as one cripples it and lives with the disability and hacks and endless hassles and total lack of equivalent features.

A stuttering suckobot, it's the greatest...

BTW it's hammered to a pile of humiliated mud when we techs have to work on them, too, another reason I can't stand them.
Endless hours and in the end, the fingers are massively crossed, and for good reason. XML hacking, bios hacking, profile hacking, ATI tool (formerly) Omega drivers, it goes on and on, a bad website, bad information on that bad website, spamming ads in the driver forever (which nVidia has now taken up)...

AMD cannot fold soon enough for me. A restructure and some real leadership could do wonders. How about real communication with the end users and real fixes instead of total oblivion or completely failed promises or retrograde upgrades that rebreak what was broken before and fixed but now arises from the ashes of gehenna once again...

I always need about 3 shots at the bar after "amd video card" tech call day.

February 25, 2013 | 06:06 AM - Posted by Vbs (not verified)

Since you are doing a full article on this, you might just as well add some Lucid Virtu MVP 2.0 testing, as Virtual V-sync and Hyperformance are designed specifically for only showing frames that will be seen. Also, I-mode has been working pretty well on alot of games now.

February 25, 2013 | 12:08 PM - Posted by Anon (not verified)

Hello there, I have a few questions about this testing methodology.

- How's this method any good when you're using the screen tearing to measure it? Isn't screen tearing a bad thing?
- How can faster graphics solutions be any better than slower ones when they produce way more screen tearing?
- How can you claim "The basic premise of Frame Rating is that the performance metrics that the industry is gathering using FRAPS are inaccurate in many cases and do not properly reflect the real-world gaming experience the user has." when no one in his right mind would let his screen tear like yours?
- How can variations under 16.6 ms be any kind of a problem when you can't see them in a 60 Hz display?

February 25, 2013 | 06:39 PM - Posted by kn00tcn

even with tearing, consistency can be seen

the claim about frame rating is talking about how those fps numbers are an average, not realtime (30,60,30,60,30,60 will show up as 45fps in fraps, but you're not actually seeing a steady 45fps image), it has nothing to do with tearing specifically

me i'm looking for 60fps vsync minimum fps

pcper will test with vsync in the full article, so we'll see what happens

February 26, 2013 | 08:23 AM - Posted by Anon (not verified)

If I'm aiming for a smooth gameplay I don't want my screen tearing like crazy so consistency in these tests is far from useful for most ppl out there.

You're wrong about FRAPS. In fact you didn't even understand how this is being tested. FRAPS reports frame times and that's what's being used here.

PCPer will test Vsync? If the screen doesn't tear their capture hardware is totally useless.

February 26, 2013 | 05:09 PM - Posted by Ryan Shrout

We are going to include tests with Vsync, but keep in mind that basically ALL graphics performance testing is done with Vsync disabled.

February 28, 2013 | 10:30 AM - Posted by Anon (not verified)

That's why yours ain't any better than previous or current testings. You're focused measuring little variances no one will notice telling how far ahead one product is from the other. How having 5 spikes is worse than having 3. You're focused calling AMD liars when following your testing the fast your system is the worse the tearing gets.

It's like testing CPUs in a gaming perspective at 1024*768 resolution. Utterly dumb and misleading.

User experience is about real world, not the same crappy graphs even harder to understand than the previous while running unplayable options like having 15 FPS, screen tearing or your physics messed.

Test frame limiters that anyone in his senses will use if he's having stutter or screen tearing issues. And FFS don't dare to publish ever again a graph where every single card is getting less than 30 FPS. You're miles away from measuring the user experience, really.

February 28, 2013 | 10:35 AM - Posted by Anon (not verified)

Oh, forgot to tell you that cranking up every single option to play isn't a thing that a sensible gamer would do.

Stop preaching about user experience when you clearly have no idea.

February 25, 2013 | 08:15 PM - Posted by kukreknecmi (not verified)

How does the capture system distinguishes / timestamps consecutive frames? Does the capture system gets an trigger after the first system sends frame draw data and timetsamps that frame , so it can be saved as new frame? How much delay is getting added?

February 26, 2013 | 07:34 AM - Posted by miahallen (not verified)

Excellent work PCPer :D
You'll be my new go-to resource with this type of investigative work going on ;)

February 26, 2013 | 07:48 AM - Posted by akaronin (not verified)

Did I missunderstood or did Ryan Shrout said that sometimes a Radeon HD 7970 CrossFire renders frames at near 0 ms !?

February 26, 2013 | 05:09 PM - Posted by Ryan Shrout

Its more like they are displayed on the screen for near 0ms of time.

February 26, 2013 | 04:50 PM - Posted by Anonymous (not verified)

Great article, looking forward to the full article.

February 26, 2013 | 05:24 PM - Posted by ToiT (not verified)

Awesome to hear about your initiative and innovation behind this guys, great work - I am very excited to see what comes of it in the future.

This seems to explain a lot of the reports at [H] about SLI just 'feeling' better than x-fire during game-play experience.

February 27, 2013 | 06:24 AM - Posted by Pengo (not verified)

A testing revolution! Keep it going!

March 5, 2013 | 07:21 PM - Posted by SilverSurfer755 (not verified)

This seems to effect multi-gpu more correct?

Should i still go ahead with buying a 7970 ghz or go for a gtx 680 instead?

Good job on the article, interesting stuff.

March 8, 2013 | 11:28 AM - Posted by Goofus Maximus (not verified)

I think we need a better name for those slivers than RUNT frames. ;)

How about "Framelets"? It makes one wonder if this is another round in the ongoing dance between nvidia and amd/ati to inflate the performance of their products compared to the competition...

March 10, 2013 | 11:13 AM - Posted by UltimateBou (not verified)

90% of players, plays with vsync on, triple buffer on.

these settings removes 90-100 % of stutter.

using RadeonPro dynamic vsync removes the remaing 10% of stutter

useless test for most players. Stutter exist without vsync and frame limiter... who plays with tearing? -.-'

March 10, 2013 | 01:01 PM - Posted by Mechromancer (not verified)

I run crossfired HD 4870 512MB gpus. I had horrible, jittery, performance until I ran across this fix:

1) create a 'user.cfg' file in the root of the Battlefield 3 game folder (by renaming a .txt to .cfg).

2) add these two lines-
RenderDevice.TripleBufferingEnable 1
RenderDevice.ForceRenderAheadLimit 1

This DRASTICALLY improved my frame smoothness. It worked even better when I enabled vsync (we all know crossfire likes vsync).

I've found that some games require these kinds of manual configuration file tweaks for AMD GPUs, and some for Nvidia GPUS (Borderlands 2 manual needed tweaks for my 550Ti setup). I'm starting to think games need to be optimized for AMD/Nvidia on both the driver side AND the game side, and not just the driver side nowadays.

March 10, 2013 | 04:37 PM - Posted by UltimateBou (not verified)

It's possible to avoid microstuttering but here they test only in the same way... very different results from user test with same methodologies.. it seems that they want to show only the "dark side" of amd card.. and green fan are really happy.

March 12, 2013 | 10:28 AM - Posted by UltimateBou (not verified)

http://img715.imageshack.us/img715/6445/crysis3exe2013031214463.jpg

frame latency locked to 16ms.

dynamic vsync (radeonpro), crossfire 7970ghz edition on 980x@4.2ghz.

1080p, very high all, fxaa, no blur, af16x.

don't try this at home... lol

March 14, 2013 | 11:08 AM - Posted by Trey Long (not verified)

Here's another article which gives frame latency details which seem to support Pc Per's contention.
http://www.hardwarecanucks.com/forum/hardware-canucks-reviews/60166-nvid...

March 14, 2013 | 01:03 PM - Posted by chefbenito (not verified)

Wow- This is some amazing work. Lots of sites benchmark, but guys you are creating a superior universal bench and that's huge!

So in case I missed it, when can we get our hands on this utility, I'd love to see real FPS instead of FRAPS avgs.

Also. I see no bias towards NV or AMD on this site at all. I think Josh is sort of an AMD fanboy as per all of his reviews and comments on the podcasts but I don't see any bias at all in any of the articles or editorial on this site.

April 21, 2013 | 04:49 PM - Posted by Anonymous (not verified)

This is an interesting benchmarking program, I must say... I myself use 2x 4gb 680s in SLI. I'm actually quite surprised at this, but I think that dropped (unrendered) frame would be an interesting spot to examine for AMD's source of driver issues and stuttering... However it isn't a large problem, it would definitely need fixing. I myself like AMD, but they do have issues that need fixing.

April 21, 2013 | 04:49 PM - Posted by Anonymous (not verified)

This is an interesting benchmarking program, I must say... I myself use 2x 4gb 680s in SLI. I'm actually quite surprised at this, but I think that dropped (unrendered) frame would be an interesting spot to examine for AMD's source of driver issues and stuttering... However it isn't a large problem, it would definitely need fixing. I myself like AMD, but they do have issues that need fixing.

January 10, 2014 | 08:44 PM - Posted by Anonymous (not verified)

Yeah, interesting theory...how about we not let one data set decide all things. mmmkay.

January 13, 2014 | 06:50 AM - Posted by bl4C (not verified)

what about testing Lucid Virtu MVP ?
the results might be interesting !

or at least we will finally know what the influence of this technology can be, instead of only speculation

oddly enough, i have not found any tests including Lucid technilogy anywhere ...

(if there are, please provide the link :) )

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Lines and paragraphs break automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd> <blockquote><p><br>
  • Web page addresses and e-mail addresses turn into links automatically.

More information about formatting options

By submitting this form, you accept the Mollom privacy policy.