What to Look For, Test Setup

Because of the complexity and sheer amount of data we have gathered using our Frame Rating performance methodology, we are breaking it up into several articles that each feature different GPU comparisons.  Here is the schedule:

We are back again with another edition of our continued reveal of data from the capture-based Frame Rating GPU performance methods.  In this third segment we are moving on down the product stack to the NVIDIA GeForce GTX 660 Ti and the AMD Radeon HD 7950 - both cards that fall into a similar price range.

gtx660ti.JPG

I have gotten many questions about why we are using the cards in each comparison and the answer is pretty straight forward: pricing.  In our first article we looked at the Radeon HD 7970 GHz Edition and the GeForce GTX 680 while in the second we compared the Radeon HD 7990 (HD 7970s in CrossFire), the GeForce GTX 690 and the GeForce GTX Titan.  This time around we have the GeForce GTX 660 Ti ($289 on Newegg.com) and the Radeon HD 7950 ($299 on Newegg.com) but we did not include the GeForce GTX 670 because it sits much higher at $359 or so.  I know some of you are going to be disappointed that it isn't in here, but I promise we'll see it again in a future piece!


If you are just joining this article series today, you have missed a lot!  If nothing else you should read our initial full release article that details everything about the Frame Rating methodology and why we are making this change to begin with.  In short, we are moving away from using FRAPS for average frame rates or even frame times and instead are using a secondary hardware capture system to record all the frames of our game play as they would be displayed to the gamer, then doing post-process analyzation on that recorded file to measure real world performance.

Because FRAPS measures frame times at a different point in the game pipeline (closer to the game engine) its results can vary dramatically from what is presented to the end user on their display.  Frame Rating solves that problem by recording video through a dual-link DVI capture card that emulates a monitor to the testing system and by simply applying a unique overlay color on each produced frame from the game, we can gather a new kind of information that tells a very unique story.

card1.jpg

The capture card that makes all of this work possible.

I don't want to spend too much time on this part of the story here as I already wrote a solid 16,000 words on the topic in our first article and I think you'll really find the results fascinating.  So, please check out my first article on the topic if you have any questions before diving into these results today!

Test System Setup
CPU Intel Core i7-3960X Sandy Bridge-E
Motherboard ASUS P9X79 Deluxe
Memory Corsair Dominator DDR3-1600 16GB
Hard Drive OCZ Agility 4 256GB SSD
Sound Card On-board
Graphics Card NVIDIA GeForce GTX 660 Ti 2GB
AMD Radeon HD 7950 3GB
Graphics Drivers AMD: 13.2 beta 7
NVIDIA: 314.07 beta
Power Supply Corsair AX1200i
Operating System Windows 8 Pro x64

 

On to the results! 

Continue reading our review of the GTX 660 Ti and HD 7950 using Frame Rating!!

Summary Thus Far

Because of the complexity and sheer amount of data we have gathered using our Frame Rating performance methodology, we are breaking it up into several articles that each feature different GPU comparisons.  Here is the schedule:

Welcome to the second in our intial series of articles focusing on Frame Rating, our new graphics and GPU performance technology that drastically changes how the community looks at single and multi-GPU performance.  In the article we are going to be focusing on a different set of graphics cards, the highest performing single card options on the market including the GeForce GTX 690 4GB dual-GK104 card, the GeForce GTX Titan 6GB GK110-based monster as well as the Radeon HD 7990, though in an emulated form.  The HD 7990 was only recently officially announced by AMD at this years Game Developers Conference but the specifications of that hardware are going to closely match what we have here on the testbed today - a pair of retail Radeon HD 7970s in CrossFire. 

titancard.JPG

Will the GTX Titan look as good in Frame Rating as it did upon its release?

If you are just joining this article series today, you have missed a lot!  If nothing else you should read our initial full release article that details everything about the Frame Rating methodology and why we are making this change to begin with.  In short, we are moving away from using FRAPS for average frame rates or even frame times and instead are using a secondary hardware capture system to record all the frames of our game play as they would be displayed to the gamer, then doing post-process analyzation on that recorded file to measure real world performance.

Because FRAPS measures frame times at a different point in the game pipeline (closer to the game engine) its results can vary dramatically from what is presented to the end user on their display.  Frame Rating solves that problem by recording video through a dual-link DVI capture card that emulates a monitor to the testing system and by simply applying a unique overlay color on each produced frame from the game, we can gather a new kind of information that tells a very unique story.

card1.jpg

The capture card that makes all of this work possible.

I don't want to spend too much time on this part of the story here as I already wrote a solid 16,000 words on the topic in our first article and I think you'll really find the results fascinating.  So, please check out my first article on the topic if you have any questions before diving into these results today!

 

Test System Setup
CPU Intel Core i7-3960X Sandy Bridge-E
Motherboard ASUS P9X79 Deluxe
Memory Corsair Dominator DDR3-1600 16GB
Hard Drive OCZ Agility 4 256GB SSD
Sound Card On-board
Graphics Card NVIDIA GeForce GTX TITAN 6GB
NVIDIA GeForce GTX 690 4GB
AMD Radeon HD 7970 CrossFire 3GB
Graphics Drivers AMD: 13.2 beta 7
NVIDIA: 314.07 beta (GTX 690)
NVIDIA: 314.09 beta (GTX TITAN)
Power Supply Corsair AX1200i
Operating System Windows 8 Pro x64

 

On to the results! 

Continue reading our review of the GTX Titan, GTX 690 and HD 7990 using Frame Rating!!

How Games Work

 

Because of the complexity and sheer amount of data we have gathered using our Frame Rating performance methodology, we are breaking it up into several articles that each feature different GPU comparisons.  Here is the schedule:

 

Introduction

The process of testing games and graphics has been evolving even longer than I have been a part of the industry: 14+ years at this point. That transformation in benchmarking has been accelerating for the last 12 months. Typical benchmarks test some hardware against some software and look at the average frame rate which can be achieved. While access to frame time has been around for nearly the full life of FRAPS, it took an article from Scott Wasson at the Tech Report to really get the ball moving and investigate how each frame contributes to the actual user experience. I immediately began research into testing actual performance perceived by the user, including the "microstutter" reported by many in PC gaming, and pondered how we might be able to test for this criteria even more accurately.

The result of that research is being fully unveiled today in what we are calling Frame Rating – a completely new way of measuring and validating gaming performance.

The release of this story for me is like the final stop on a journey that has lasted nearly a complete calendar year.  I began to release bits and pieces of this methodology starting on January 3rd with a video and short article that described our capture hardware and the benefits that directly capturing the output from a graphics card would bring to GPU evaluation.  After returning from CES later in January, I posted another short video and article that showcased some of the captured video and stepping through a recorded file frame by frame to show readers how capture could help us detect and measure stutter and frame time variance. 

card4.jpg

Finally, during the launch of the NVIDIA GeForce GTX Titan graphics card, I released the first results from our Frame Rating system and discussed how certain card combinations, in this case CrossFire against SLI, could drastically differ in perceived frame rates and performance while giving very similar average frame rates.  This article got a lot more attention than the previous entries and that was expected – this method doesn’t attempt to dismiss other testing options but it is going to be pretty disruptive.  I think the remainder of this article will prove that. 

Today we are finally giving you all the details on Frame Rating; how we do it, what we learned and how you should interpret the results that we are providing.  I warn you up front though that this is not an easy discussion and while I am doing my best to explain things completely, there are going to be more questions going forward and I want to see them all!  There is still much to do regarding graphics performance testing, even after Frame Rating becomes more common. We feel that the continued dialogue with readers, game developers and hardware designers is necessary to get it right.

Below is our full video that features the Frame Rating process, some example results and some discussion on what it all means going forward.  I encourage everyone to watch it but you will definitely need the written portion here to fully understand this transition in testing methods.  Subscribe to your YouTube channel if you haven't already!

Continue reading our analysis of the new Frame Rating performance testing methodology!!

PCPer Live! Tomb Raider Game Stream - Win Games and Graphics Cards from AMD!

Subject: Graphics Cards | March 4, 2013 - 04:31 PM |
Tagged: video, tomb raider, tahiti, radeon, never settle reloaded, live, crysis, amd

UPDATE: Missed the live stream?  Relive the incredible experience right here!  

On March 5th on the PC Perspective Live! page we will be streaming some game action of the new Tomb Raider.  In what might be one of the most impressive game series reboots in history, this iteration of the action-adventure gameplay is definitely the most impressive looking to date.  And don't forget all the hair animation we are likely to see...

We will be teaming up with AMD once again to provide a fun and exciting PCPer Game Stream that includes game demonstrations and of course, prizes and game keys for those that watch the event LIVE! 

UPDATE: We are excited to announce that Crystal Dynamics' Brian Horton, Senior Art Director for Tomb Raider, will be joining us on the PC Perpsective Game Stream to answer questions and to give us more detail on the visual effects at work in the PC version of the game!

tr1.jpg

Tomb Raider Game Stream

5pm PT / 8pm ET - March 5th

PC Perspective Live! Page

Warning: this one will DEFINITELY have mature language and content!!

The stream will be sponsored by AMD and its Never Settle Reloaded game bundles which we previously told you about.  Depending on the AMD Radeon HD 7000 series GPU that you buy, you could get some amazing free games including:

  • Radeon HD 7900 Series
    • FREE Crysis 3
    • FREE Bioshock Infinite
  • Radeon HD 7800 Series
    • FREE Bioshock Infinite
    • FREE Tomb Raider
  • Radeon HD 7900 CrossFire Set
    • FREE Crysis 3
    • FREE Bioshock Infinite
    • FREE Tomb Raider
    • FREE Far Cry 3
    • FREE Hitman: Absolution
    • FREE Sleeping Dogs

nsr_matrix.jpg

AMD's Antal Tungler (@ColoredRocks on twitter) will be joining us via Skype to talk about the game's technology, performance considerations as well as helping me with some co-op gaming!

Of course, just to sweeten the deal a bit we have some prizes lined up for those of you that participate in our Tomb Raider Game Stream:

  • 2 x Gigabyte Radeon HD 7870 OC 2GB graphics cards (plus Tomb Raider & Bioshock Infinite)
  • 1 x HIS 7850 iPower IceQ Turbo 4GB graphics card (plus Tomb Raider & Bioshock Infinite)
  • 3 x Combo codes for both Tomb Raider AND Bioshock Infinite

gboc.png

Pretty nice, huh?  All you have to do to win is be present on the PC Perspective Live! Page during the event as we will announce both the content/sweepstakes method AND the winners!

Stop in on March 5h for some PC gaming fun!!

tr2.jpg

Author:
Manufacturer: PC Perspective

In case you missed it...

UPDATE: We have now published full details on our Frame Rating capture and analysis system as well as an entire host of benchmark results.  Please check it out!!

In one of the last pages of our recent NVIDIA GeForce GTX TITAN graphics card review we included an update to our Frame Rating graphics performance metric that details the testing method in more detail and showed results for the first time.  Because it was buried so far into the article, I thought it was worth posting this information here as a separate article to solict feedback from readers and help guide the discussion forward without getting lost in the TITAN shuffle.  If you already read that page of our TITAN review, nothing new is included below. 

I am still planning a full article based on these results sooner rather than later; for now, please leave me your thoughts, comments, ideas and criticisms in the comments below!


Why are you not testing CrossFire??

If you haven't been following our sequence of stories that investigates a completely new testing methodology we are calling "frame rating", then you are really missing out.  (Part 1 is here, part 2 is here.)  The basic premise of Frame Rating is that the performance metrics that the industry is gathering using FRAPS are inaccurate in many cases and do not properly reflect the real-world gaming experience the user has.

Because of that, we are working on another method that uses high-end dual-link DVI capture equipment to directly record the raw output from the graphics card with an overlay technology that allows us to measure frame rates as they are presented on the screen, not as they are presented to the FRAPS software sub-system.  With these tools we can measure average frame rates, frame times and stutter, all in a way that reflects exactly what the viewer sees from the game.

We aren't ready to show our full sets of results yet (soon!) but the problems lie in that AMD's CrossFire technology shows severe performance degradations when viewed under the Frame Rating microscope that do not show up nearly as dramatically under FRAPS.  As such, I decided that it was simply irresponsible of me to present data to readers that I would then immediately refute on the final pages of this review (Editor: referencing the GTX TITAN article linked above.) - it would be a waste of time for the reader and people that skip only to the performance graphs wouldn't know our theory on why the results displayed were invalid.

Many other sites will use FRAPS, will use CrossFire, and there is nothing wrong with that at all.  They are simply presenting data that they believe to be true based on the tools at their disposal.  More data is always better. 

Here are these results and our discussion.  I decided to use the most popular game out today, Battlefield 3 and please keep in mind this is NOT the worst case scenario for AMD CrossFire in any way.  I tested the Radeon HD 7970 GHz Edition in single and CrossFire configurations as well as the GeForce GTX 680 and SLI.  To gather results I used two processes:

  1. Run FRAPS while running through a repeatable section and record frame rates and frame times for 60 seconds
  2. Run our Frame Rating capture system with a special overlay that allows us to measure frame rates and frame times with post processing.

Here is an example of what the overlay looks like in Battlefield 3.

fr_sli_1.jpg

Frame Rating capture on GeForce GTX 680s in SLI - Click to Enlarge

The column on the left is actually the visuals of an overlay that is applied to each and every frame of the game early in the rendering process.  A solid color is added to the PRESENT call (more details to come later) for each individual frame.  As you know, when you are playing a game, multiple frames will make it on any single 60 Hz cycle of your monitor and because of that you get a succession of colors on the left hand side.

By measuring the pixel height of those colored columns, and knowing the order in which they should appear beforehand, we can gather the same data that FRAPS does but our results are seen AFTER any driver optimizations and DX changes the game might make.

fr_cf_1.jpg

Frame Rating capture on Radeon HD 7970 CrossFire - Click to Enlarge

Here you see a very similar screenshot running on CrossFire.  Notice the thin silver band between the maroon and purple?  That is a complete frame according to FRAPS and most reviews.  Not to us - we think that frame rendered is almost useless. 

Continue reading our 3rd part in a series of Frame Rating and to see our first performance results!!

PCPer Live! Crysis 3 Game Stream - Win Games and Graphics Cards from AMD!

Subject: Graphics Cards | February 19, 2013 - 08:00 PM |
Tagged: video, tahiti, radeon, never settle reloaded, live, Crysis 3, crysis, amd

UPDATE: If you missed the live stream you can still catch the YouTube replay right here!!

On February 19th on the PC Perspective Live! page we will be streaming some single player game action of the new Crysis 3.  If there has ever been a game that defined the world of PC gaming graphics and technology, it is the Crysis series. 

"Sure, but can it play Crysis?"

There is probably no more famous line of dialogue that pigeon hole's new hardware releases. 

With the release of the latest version of Crysis 3 on February 19th, we will be teaming up with AMD once again to provide a fun and exciting PCPer Game Stream that includes game demonstrations and of course, prizes and game keys for those that watch the event LIVE! 

crysis1.jpg

Crysis 3 Game Stream

5pm PT / 8pm ET - February 19th

PC Perspective Live! Page

Warning: this one will DEFINITELY have mature language and content!!

The stream will be sponsored by AMD and its Never Settle Reloaded game bundles which we previously told you about.  Depending on the AMD Radeon HD 7000 series GPU that you buy, you could get some amazing free games including:

  • Radeon HD 7900 Series
    • FREE Crysis 3
    • FREE Bioshock Infinite
  • Radeon HD 7800 Series
    • FREE Bioshock Infinite
    • FREE Tomb Raider
  • Radeon HD 7900 CrossFire Set
    • FREE Crysis 3
    • FREE Bioshock Infinite
    • FREE Tomb Raider
    • FREE Far Cry 3
    • FREE Hitman: Absolution
    • FREE Sleeping Dogs

nsr_matrix.jpg

AMD's Robert Hallock (@Thracks on twitter) will be joining us via Skype to talk about the game's technology, performance considerations as well as helping me with some co-op gaming!

Of course, just to sweeten the deal a bit we have some prizes lined up for those of you that participate in our Crysis 3 Game Stream:

  • 2 x Radeon HD 7970 3GB graphics cards
  • 4 x Combo codes for both Crysis 3 AND Bioshock Infinite

Pretty nice, huh?  All you have to do to win is be present on the PC Perspective Live! Page during the event as we will announce both the content/sweepstakes method AND the winners!

Stop in on February 19th for some PC gaming fun!!

crysis2.jpg

AMD wants to set the record straight on its future GPU strategy

Subject: Graphics Cards | February 15, 2013 - 01:50 PM |
Tagged: southern islands, Solar System, Sea Islands, radeon, oland, mars, holycrapiamtotallyconfused, amd

Remember that story we posted last week and then discussed on the podcast about AMD not releasing any new GPUs in 2013?  Today we had a call with AMD that attempted to answer some questions, clear up some confusion and give us some insight to the company's direction.  I say 'attempted' because after a 53 minute discussion, we have some answers, but we also have some interesting questions that remain.

First, some definitions.  If you have heard about code names like "Solar System" and "Sea Islands" you might not know what they refer to.  Sea Islands is a new line that will fall into the 8000-series of products and will be a refresh, slightly different architecture based heavily on the Southern Islands parts you've come to love in the Radeon HD 7000 parts.  Solar System is the name AMD has given to the sub-category of Sea Islands directly related to mobile products, the 8000M. 

amd20131.jpg

The slide that started this confusion - and our questions.

What might make things even more confusing is that there are some 8000-series parts that are already shipping in OEM desktops and notebooks that use verbatim HD 7000 GPU specs.  So what you have is a combination series with Radeon HD 8000 that is made up of some rebrands and at least a couple of "new" chips thus far.  Those two new GPUs, Mars and Oland (Radeon HD 8650 and HD 8670) depending on the mobile or desktop target, are already out and you can find them if you look hard.  They are NOT available in the channel or for DIY PC users. 

Our readers might be disappointed to learn that Sea Islands is heavily focused on the notebook and mobile markets though AMD did indicate that there some good things coming for the channel users in the future in 2013. 

We also learned that the HD 7900-series will remain the company's high end parts through the end of 2013 but AMD said that there are new SKUs set to be released in this series sometime this year as well.  Will that be the elusive HD 7990 dual-GPU product or maybe just something in the mainstream 7800 segments?  They wouldn't tell us but we are definitely hoping for higher performance parts.  You might also expect to see these new 7000-series parts to use Sea Islands silicon...

7970.jpg

The Radeon HD 7970 looks like it will stay a focus for AMD throughout 2013.

Many readers might be wondering why AMD is breaking its standard cadence of near-yearly GPU releases.  The answer came from AMD's Roy Taylor, VP of Channel Sales, who said that "7000 series parts are continuing to ramp UP, sales are increasing" so it is premature for AMD, as a company intending to make money, to introduce a new series or architecture. 

In fact Roy was very emphatic about relieving us of potential ambiguity. 

We have products, we have a road map.  We are not announcing them now because we want to reposition the ones we have now.  We are not sitting still, we do not lack resources, we do not lack imagination.

So what can you expect for the future?  Sea Islands chips will continue to be released and eventually in the desktop, channel market and some of them will be branded as 7000-series parts and some of them will be branded as 8000-series parts.  They wouldn't give us information on whether or not you'll see BIGGER chips (which we would assume would be faster) than the current HD 7900 cards or if they would all be in the mainstream segment. 

crysis1.jpg

AMD thinks its partnerships with key games like Crysis 3 will help keep momentum in 2013.

The residual message from this call was that AMD wants everyone to know that they have the best products on the market today and to maintain that momentum, AMD will enhance drivers, establish big partnerships with gaming companies and developers and release SOME new GPUs. 

AMD was cagey again when asked about the possibility of a new architecture by the end of 2013 but based on the reactions of AMD reps I tend to believe we will see it, though probably very very close to the end of that time.  (Update: AMD did in fact say that an entire new product stack would be releaed by the end of 2013.)

That all clear now?

AMD plans no new graphics cards in 2013, report

Subject: Graphics Cards | February 8, 2013 - 10:03 PM |
Tagged: amd, radeon

In a report first spotted by Rage3D from source website 4gamer.net, news is filtering out that AMD may in fact have no new discrete graphics card releases for the remainder of 2013!  While talking with the APAC media about the fantastic Never Settle Reloaded game bundle, they showed THIS slide. 

amd20131.jpg

That seems to indicate that at the very least through the 3rd quarter of 2013, AMD has no plans to update or add to its discrete graphics card roadmap.  We had heard whispers of this fact while at CES in January but this pretty much puts a cap on it.  And with the wording of "throughout 2013" it could indicate we won't see new product until 2014.

amd20132.jpg

Also shown, this product comparison between AMD and NVIDIA, put together by AMD, is a bit lopsided and less than 100% accurate in my eyes.  With the release of the new 3DMark Fire Strike benchmark AMD has a distinct advantage and it seems the slide here is based completely on that....blech

Regardless, what does it mean if AMD actually has no new discrete, enthusiast class cards for 2013?  We know the rumors are swirling about the NVIDIA GeForce Titan based on the GK110 and sporting 2688 CUDA cores and it will likely take the place as the fastest single GPU card on the market.  AMD has been depending on its partners to build multi-GPU options based on Southern Islands like the ASUS ARES II and Powercolor Devil 13 but they have been pretty low volume.  Our original review of the HD 7970 launched in December 2011....this could be quite a drought. 

Source: 4gamer.net

More Oland leaks, this time with open sauce goodness

Subject: General Tech | February 5, 2013 - 02:16 PM |
Tagged: Sea Islands, radeon, GCN, amd, 8970, oland, hd 8000, RadeonSI, gallium, mesa

Phoronix has good news for Linux users about the "RadeonSI" Gallium3D driver which AMD has slowly been developing for the HD 7000 series, MESA has announced the driver is being developed for the HD 8000 series.  The project commit is a candidate for MESA 9.1 and the Linux 3.9 kernel which could lead to some issues as most Linux flavours are using 3.8 or earlier but should bode well for the future.  This hopefully signals a greater commitment to OpenCL and other projects AMD has started but not managed to fully develop.  We also have quite a few PCI IDs from the commit statement, 0x6600, 0x6601, 0x6602, 0x6603, 0x6606, 0x6607, 0x6610, 0x6611, 0x6613, 0x6620, 0x6621, 0x6623, and 0x6631 are all listed.

g3d_phor.jpg

"While AMD has yet to officially introduce their Radeon HD 8000 series, published today was the initial open-source Linux graphics driver support for handling the Radeon HD 8800 "Oland" graphics cards."

Here is some more Tech News from around the web:

Tech Talk

Source: Phoronix
Author:
Manufacturer: PC Perspective

Another update

In our previous article and video, I introduced you to our upcoming testing methodology for evaluating graphics cards based not only frame rates but on frame smoothness and the efficiency of those frame rates.  I showed off some of the new hardware we are using for this process and detailed how direct capture of graphics card output allows us to find interesting frame and animation anomalies using some Photoshop still frames.

d31.jpg

Today we are taking that a step further and looking at a couple of captured videos that demonstrate a "stutter" and walking you through, frame by frame, how we can detect, visualize and even start to measure them.

dis1.jpg

This video takes a couple of examples of stutter in games, DiRT 3 and Dishonored to be exact, and shows what they look like in real time, at 25% speed and then finally in a much more detailed frame-by-frame analysis.

 

Video Loading...

 

Obviously this is just a couple instances of what a stutter is and there are often times less apparent in-game stutters that are even harder to see in video playback.  Not to worry - this capture method is capable of seeing those issues as well and we plan on diving into the "micro" level as well shortly.

We aren't going to start talking about whose card and what driver is being used yet and I know that there are still a lot of questions to be answered on this topic.  You will be hearing more quite soon from us and I thank you all for your comments, critiques and support.

Let me know below what you thought of this video and any questions that you might have.