What to Look For, Test Setup

Because of the complexity and sheer amount of data we have gathered using our Frame Rating performance methodology, we are breaking it up into several articles that each feature different GPU comparisons.  Here is the schedule:

We are back again with another edition of our continued reveal of data from the capture-based Frame Rating GPU performance methods.  In this third segment we are moving on down the product stack to the NVIDIA GeForce GTX 660 Ti and the AMD Radeon HD 7950 - both cards that fall into a similar price range.

gtx660ti.JPG

I have gotten many questions about why we are using the cards in each comparison and the answer is pretty straight forward: pricing.  In our first article we looked at the Radeon HD 7970 GHz Edition and the GeForce GTX 680 while in the second we compared the Radeon HD 7990 (HD 7970s in CrossFire), the GeForce GTX 690 and the GeForce GTX Titan.  This time around we have the GeForce GTX 660 Ti ($289 on Newegg.com) and the Radeon HD 7950 ($299 on Newegg.com) but we did not include the GeForce GTX 670 because it sits much higher at $359 or so.  I know some of you are going to be disappointed that it isn't in here, but I promise we'll see it again in a future piece!


If you are just joining this article series today, you have missed a lot!  If nothing else you should read our initial full release article that details everything about the Frame Rating methodology and why we are making this change to begin with.  In short, we are moving away from using FRAPS for average frame rates or even frame times and instead are using a secondary hardware capture system to record all the frames of our game play as they would be displayed to the gamer, then doing post-process analyzation on that recorded file to measure real world performance.

Because FRAPS measures frame times at a different point in the game pipeline (closer to the game engine) its results can vary dramatically from what is presented to the end user on their display.  Frame Rating solves that problem by recording video through a dual-link DVI capture card that emulates a monitor to the testing system and by simply applying a unique overlay color on each produced frame from the game, we can gather a new kind of information that tells a very unique story.

card1.jpg

The capture card that makes all of this work possible.

I don't want to spend too much time on this part of the story here as I already wrote a solid 16,000 words on the topic in our first article and I think you'll really find the results fascinating.  So, please check out my first article on the topic if you have any questions before diving into these results today!

Test System Setup
CPU Intel Core i7-3960X Sandy Bridge-E
Motherboard ASUS P9X79 Deluxe
Memory Corsair Dominator DDR3-1600 16GB
Hard Drive OCZ Agility 4 256GB SSD
Sound Card On-board
Graphics Card NVIDIA GeForce GTX 660 Ti 2GB
AMD Radeon HD 7950 3GB
Graphics Drivers AMD: 13.2 beta 7
NVIDIA: 314.07 beta
Power Supply Corsair AX1200i
Operating System Windows 8 Pro x64

 

On to the results! 

Continue reading our review of the GTX 660 Ti and HD 7950 using Frame Rating!!

Summary Thus Far

Because of the complexity and sheer amount of data we have gathered using our Frame Rating performance methodology, we are breaking it up into several articles that each feature different GPU comparisons.  Here is the schedule:

Welcome to the second in our intial series of articles focusing on Frame Rating, our new graphics and GPU performance technology that drastically changes how the community looks at single and multi-GPU performance.  In the article we are going to be focusing on a different set of graphics cards, the highest performing single card options on the market including the GeForce GTX 690 4GB dual-GK104 card, the GeForce GTX Titan 6GB GK110-based monster as well as the Radeon HD 7990, though in an emulated form.  The HD 7990 was only recently officially announced by AMD at this years Game Developers Conference but the specifications of that hardware are going to closely match what we have here on the testbed today - a pair of retail Radeon HD 7970s in CrossFire. 

titancard.JPG

Will the GTX Titan look as good in Frame Rating as it did upon its release?

If you are just joining this article series today, you have missed a lot!  If nothing else you should read our initial full release article that details everything about the Frame Rating methodology and why we are making this change to begin with.  In short, we are moving away from using FRAPS for average frame rates or even frame times and instead are using a secondary hardware capture system to record all the frames of our game play as they would be displayed to the gamer, then doing post-process analyzation on that recorded file to measure real world performance.

Because FRAPS measures frame times at a different point in the game pipeline (closer to the game engine) its results can vary dramatically from what is presented to the end user on their display.  Frame Rating solves that problem by recording video through a dual-link DVI capture card that emulates a monitor to the testing system and by simply applying a unique overlay color on each produced frame from the game, we can gather a new kind of information that tells a very unique story.

card1.jpg

The capture card that makes all of this work possible.

I don't want to spend too much time on this part of the story here as I already wrote a solid 16,000 words on the topic in our first article and I think you'll really find the results fascinating.  So, please check out my first article on the topic if you have any questions before diving into these results today!

 

Test System Setup
CPU Intel Core i7-3960X Sandy Bridge-E
Motherboard ASUS P9X79 Deluxe
Memory Corsair Dominator DDR3-1600 16GB
Hard Drive OCZ Agility 4 256GB SSD
Sound Card On-board
Graphics Card NVIDIA GeForce GTX TITAN 6GB
NVIDIA GeForce GTX 690 4GB
AMD Radeon HD 7970 CrossFire 3GB
Graphics Drivers AMD: 13.2 beta 7
NVIDIA: 314.07 beta (GTX 690)
NVIDIA: 314.09 beta (GTX TITAN)
Power Supply Corsair AX1200i
Operating System Windows 8 Pro x64

 

On to the results! 

Continue reading our review of the GTX Titan, GTX 690 and HD 7990 using Frame Rating!!

How Games Work

 

Because of the complexity and sheer amount of data we have gathered using our Frame Rating performance methodology, we are breaking it up into several articles that each feature different GPU comparisons.  Here is the schedule:

 

Introduction

The process of testing games and graphics has been evolving even longer than I have been a part of the industry: 14+ years at this point. That transformation in benchmarking has been accelerating for the last 12 months. Typical benchmarks test some hardware against some software and look at the average frame rate which can be achieved. While access to frame time has been around for nearly the full life of FRAPS, it took an article from Scott Wasson at the Tech Report to really get the ball moving and investigate how each frame contributes to the actual user experience. I immediately began research into testing actual performance perceived by the user, including the "microstutter" reported by many in PC gaming, and pondered how we might be able to test for this criteria even more accurately.

The result of that research is being fully unveiled today in what we are calling Frame Rating – a completely new way of measuring and validating gaming performance.

The release of this story for me is like the final stop on a journey that has lasted nearly a complete calendar year.  I began to release bits and pieces of this methodology starting on January 3rd with a video and short article that described our capture hardware and the benefits that directly capturing the output from a graphics card would bring to GPU evaluation.  After returning from CES later in January, I posted another short video and article that showcased some of the captured video and stepping through a recorded file frame by frame to show readers how capture could help us detect and measure stutter and frame time variance. 

card4.jpg

Finally, during the launch of the NVIDIA GeForce GTX Titan graphics card, I released the first results from our Frame Rating system and discussed how certain card combinations, in this case CrossFire against SLI, could drastically differ in perceived frame rates and performance while giving very similar average frame rates.  This article got a lot more attention than the previous entries and that was expected – this method doesn’t attempt to dismiss other testing options but it is going to be pretty disruptive.  I think the remainder of this article will prove that. 

Today we are finally giving you all the details on Frame Rating; how we do it, what we learned and how you should interpret the results that we are providing.  I warn you up front though that this is not an easy discussion and while I am doing my best to explain things completely, there are going to be more questions going forward and I want to see them all!  There is still much to do regarding graphics performance testing, even after Frame Rating becomes more common. We feel that the continued dialogue with readers, game developers and hardware designers is necessary to get it right.

Below is our full video that features the Frame Rating process, some example results and some discussion on what it all means going forward.  I encourage everyone to watch it but you will definitely need the written portion here to fully understand this transition in testing methods.  Subscribe to your YouTube channel if you haven't already!

Continue reading our analysis of the new Frame Rating performance testing methodology!!

Podcast #243 - ASUS Crosshair V Formula-Z, MSI Z77A-G45 Thunderbolt, 2TB SSDs and more!

Subject: General Tech | March 21, 2013 - 02:54 PM |
Tagged: z77a-g45 thunderbolt, video, tegra, quadro, podcast, GTX 690, GTC 2013, DDR3-3000, Crosshair V Formula Z, 2tb ssd

PC Perspective Podcast #243 - 03/21/2013

Join us this week as we discuss the ASUS Crosshair V Formula-Z, MSI Z77A-G45 Thunderbolt, 2TB SSDs and more!

You can subscribe to us through iTunes and you can still access it directly through the RSS page HERE.

The URL for the podcast is: http://pcper.com/podcast - Share with your friends!

  • iTunes - Subscribe to the podcast directly through the iTunes Store
  • RSS - Subscribe through your regular RSS reader
  • MP3 - Direct download link to the MP3 file

Hosts: Josh Walrath, Jeremy Hellstrom, Allyn Malventano, Morry Teitelman, and sometimes Ken Addison

This Podcast is brought to you by MSI!

Program length: 1:18:24

Podcast topics of discussion:
  1. Week in Review:
  1. News items of interest:
  2. Closing:
    1. 1:09:50 Hardware / Software Pick of the Week
      1. Morry: Memory, always more memory - G.Skill Sniper 1866 16GB DDR3
  3. 1-888-38-PCPER or podcast@pcper.com

 

PCPer Live! Tomb Raider Game Stream - Win Games and Graphics Cards from AMD!

Subject: Graphics Cards | March 4, 2013 - 04:31 PM |
Tagged: video, tomb raider, tahiti, radeon, never settle reloaded, live, crysis, amd

UPDATE: Missed the live stream?  Relive the incredible experience right here!  

On March 5th on the PC Perspective Live! page we will be streaming some game action of the new Tomb Raider.  In what might be one of the most impressive game series reboots in history, this iteration of the action-adventure gameplay is definitely the most impressive looking to date.  And don't forget all the hair animation we are likely to see...

We will be teaming up with AMD once again to provide a fun and exciting PCPer Game Stream that includes game demonstrations and of course, prizes and game keys for those that watch the event LIVE! 

UPDATE: We are excited to announce that Crystal Dynamics' Brian Horton, Senior Art Director for Tomb Raider, will be joining us on the PC Perpsective Game Stream to answer questions and to give us more detail on the visual effects at work in the PC version of the game!

tr1.jpg

Tomb Raider Game Stream

5pm PT / 8pm ET - March 5th

PC Perspective Live! Page

Warning: this one will DEFINITELY have mature language and content!!

The stream will be sponsored by AMD and its Never Settle Reloaded game bundles which we previously told you about.  Depending on the AMD Radeon HD 7000 series GPU that you buy, you could get some amazing free games including:

  • Radeon HD 7900 Series
    • FREE Crysis 3
    • FREE Bioshock Infinite
  • Radeon HD 7800 Series
    • FREE Bioshock Infinite
    • FREE Tomb Raider
  • Radeon HD 7900 CrossFire Set
    • FREE Crysis 3
    • FREE Bioshock Infinite
    • FREE Tomb Raider
    • FREE Far Cry 3
    • FREE Hitman: Absolution
    • FREE Sleeping Dogs

nsr_matrix.jpg

AMD's Antal Tungler (@ColoredRocks on twitter) will be joining us via Skype to talk about the game's technology, performance considerations as well as helping me with some co-op gaming!

Of course, just to sweeten the deal a bit we have some prizes lined up for those of you that participate in our Tomb Raider Game Stream:

  • 2 x Gigabyte Radeon HD 7870 OC 2GB graphics cards (plus Tomb Raider & Bioshock Infinite)
  • 1 x HIS 7850 iPower IceQ Turbo 4GB graphics card (plus Tomb Raider & Bioshock Infinite)
  • 3 x Combo codes for both Tomb Raider AND Bioshock Infinite

gboc.png

Pretty nice, huh?  All you have to do to win is be present on the PC Perspective Live! Page during the event as we will announce both the content/sweepstakes method AND the winners!

Stop in on March 5h for some PC gaming fun!!

tr2.jpg

Author:
Subject: Systems
Manufacturer: Sony

An AIO and Tablet Design

When new and interesting architectures and technology are developed, it enables system designers to build creative designs and systems for consumers.  With its renewed focus on power efficiency as well as performance, Intel has helped move the industry towards new form factors like the Next Unit of Computing and the evolution of the All-in-One design.

Today we are taking a look at the new Sony VAIO Tap 20 system, an AIO that not only integrates a 10-point touch screen on a 20-in 1600x900 resolution display and an Ivy Bridge architecture ultra low voltage processor, but also a battery to make the design semi-mobile and ripe for inclusion in high-tech homes. 

Check out our quick video overview below and then follow that up with a full pictorial outline and some more details!

Video Loading...

This isn't Sony's first foray into all-in-one PCs of course but it is among the first to combine this particular set of features.  In what is essentially an Ultrabook design with a large screen, the Tap 20 combines an Ivy Bridge Core i5 processors, 4GB of DDR3 memory and a 750GB hard drive.  Here is the breakdown:

 

Sony VAIO Tap 20 System Setup
CPU Intel Core i5-3317U
Memory 4GB DDR3-1600 (1 x SODIMM)
Hard Drive 750GB XXRPM HDD (2.5-in)
Sound Card On-board
Graphics Card Intel HD 4000 Processor Graphics
Display 1600x900 20-in touch screen (10 point)
Power Supply External
Networking Gigabit Ethernet
802.11n WiFi
I/O 2 x USB 3.0
SD / Memory Stick card reader
Headphone / Mic connection
Operating System Windows 8 x64

 

IMG_9546.JPG

The display is pretty nice with a 1600x900 reoslution though I do wish we had seen a full 1080p screen for HD video playback.  As with most touch screens, the display quality is under that of a non-touch IPS monitor but even up close (as you tend to use touch devices) you'll be hard pressed to find any imperfections.  Viewing angles are great as well which allows for better multi-person usage. 

IMG_9547.JPG

On the left we find the power connection and a hard wired Ethernet connection that compliments the integrated 802.11n WiFi.

Continue reading our review of the Sony VAIO Tap 20 AIO!!!

Podcast #240 - GTX TITAN Benchmarks, Frame Rating, Tegra 4 Details and more!

Subject: General Tech | February 28, 2013 - 03:45 PM |
Tagged: video, titan, sli, R5000, podcast, nvidia, H90, H110, gtx titan, frame rating, firepro, crossfire, amd

PC Perspective Podcast #240 - 02/28/2013

Join us this week as we discuss GTX TITAN Benchmarks, Frame Rating, Tegra 4 Details and more!

You can subscribe to us through iTunes and you can still access it directly through the RSS page HERE.

The URL for the podcast is: http://pcper.com/podcast - Share with your friends!

  • iTunes - Subscribe to the podcast directly through the iTunes Store
  • RSS - Subscribe through your regular RSS reader
  • MP3 - Direct download link to the MP3 file

Hosts: Ryan Shrout, Jeremy Hellstrom, Josh Walrath and Allyn Malventano

This Podcast is brought to you by MSI!

Program length: 1:24:28

Podcast topics of discussion:

  1. 0:01:18 PCPer Podcast BINGO!
  2. Week in Reviews:
    1. 0:03:00 GeForce GTX TITAN Performance Review
    2. 0:21:55 Frame Rating Part 3: First Results from the New GPU Performance Tools
    3. 0:38:00 Corsair Hydro Series H90 and H110 140mm Liquid Cooler Review
  3. 0:40:30 This Podcast is brought to you by MSI!
  4. News items of interest:
    1. 0:41:45 New Offices coming for NVIDIA
    2. 0:45:00 Chromebook Pixel brings high-res to high-price
    3. 0:48:00 GPU graphics market updates from JPR
    4. 0:55:45 Tegra 4 graphics details from Mobile World Congress
    5. 1:01:00 Unreal Engine 4 on PS4 has reduced quality
    6. 1:04:10 Micron SAS SSDs
    7. 1:08:25 AMD FirePro R5000 PCoIP Card
  5. Closing:
    1. 1:13:35 Hardware / Software Pick of the Week
      1. Ryan: NOT this 3 port HDMI switch
      2. Jeremy: Taxidermy + PICAXE, why didn't we think of this before?
      3. Josh: Still among my favorite headphones
      4. Allyn: Cyto
  1. 1-888-38-PCPER or podcast@pcper.com
  2. http://pcper.com/podcast
  3. http://twitter.com/ryanshrout and http://twitter.com/pcper
  4. Closing/outro

Be sure to subscribe to the PC Perspective YouTube channel!!

 

Author:
Manufacturer: PC Perspective

In case you missed it...

UPDATE: We have now published full details on our Frame Rating capture and analysis system as well as an entire host of benchmark results.  Please check it out!!

In one of the last pages of our recent NVIDIA GeForce GTX TITAN graphics card review we included an update to our Frame Rating graphics performance metric that details the testing method in more detail and showed results for the first time.  Because it was buried so far into the article, I thought it was worth posting this information here as a separate article to solict feedback from readers and help guide the discussion forward without getting lost in the TITAN shuffle.  If you already read that page of our TITAN review, nothing new is included below. 

I am still planning a full article based on these results sooner rather than later; for now, please leave me your thoughts, comments, ideas and criticisms in the comments below!


Why are you not testing CrossFire??

If you haven't been following our sequence of stories that investigates a completely new testing methodology we are calling "frame rating", then you are really missing out.  (Part 1 is here, part 2 is here.)  The basic premise of Frame Rating is that the performance metrics that the industry is gathering using FRAPS are inaccurate in many cases and do not properly reflect the real-world gaming experience the user has.

Because of that, we are working on another method that uses high-end dual-link DVI capture equipment to directly record the raw output from the graphics card with an overlay technology that allows us to measure frame rates as they are presented on the screen, not as they are presented to the FRAPS software sub-system.  With these tools we can measure average frame rates, frame times and stutter, all in a way that reflects exactly what the viewer sees from the game.

We aren't ready to show our full sets of results yet (soon!) but the problems lie in that AMD's CrossFire technology shows severe performance degradations when viewed under the Frame Rating microscope that do not show up nearly as dramatically under FRAPS.  As such, I decided that it was simply irresponsible of me to present data to readers that I would then immediately refute on the final pages of this review (Editor: referencing the GTX TITAN article linked above.) - it would be a waste of time for the reader and people that skip only to the performance graphs wouldn't know our theory on why the results displayed were invalid.

Many other sites will use FRAPS, will use CrossFire, and there is nothing wrong with that at all.  They are simply presenting data that they believe to be true based on the tools at their disposal.  More data is always better. 

Here are these results and our discussion.  I decided to use the most popular game out today, Battlefield 3 and please keep in mind this is NOT the worst case scenario for AMD CrossFire in any way.  I tested the Radeon HD 7970 GHz Edition in single and CrossFire configurations as well as the GeForce GTX 680 and SLI.  To gather results I used two processes:

  1. Run FRAPS while running through a repeatable section and record frame rates and frame times for 60 seconds
  2. Run our Frame Rating capture system with a special overlay that allows us to measure frame rates and frame times with post processing.

Here is an example of what the overlay looks like in Battlefield 3.

fr_sli_1.jpg

Frame Rating capture on GeForce GTX 680s in SLI - Click to Enlarge

The column on the left is actually the visuals of an overlay that is applied to each and every frame of the game early in the rendering process.  A solid color is added to the PRESENT call (more details to come later) for each individual frame.  As you know, when you are playing a game, multiple frames will make it on any single 60 Hz cycle of your monitor and because of that you get a succession of colors on the left hand side.

By measuring the pixel height of those colored columns, and knowing the order in which they should appear beforehand, we can gather the same data that FRAPS does but our results are seen AFTER any driver optimizations and DX changes the game might make.

fr_cf_1.jpg

Frame Rating capture on Radeon HD 7970 CrossFire - Click to Enlarge

Here you see a very similar screenshot running on CrossFire.  Notice the thin silver band between the maroon and purple?  That is a complete frame according to FRAPS and most reviews.  Not to us - we think that frame rendered is almost useless. 

Continue reading our 3rd part in a series of Frame Rating and to see our first performance results!!

Join PCPer and NVIDIA for a GeForce GTX TITAN Live Review!

Subject: Graphics Cards | February 21, 2013 - 01:12 PM |
Tagged: video, titan, nvidia, live review, live, kepler, geforce titan, geforce

Missed the live event?  Here is the full replay feature me and Tom Petersen!

Hopefully by now you have read our review of the NVIDIA GeForce GTX TITAN 6GB graphics card that was just released.  This is definitely a product release that highlights a generations of GPUs and I would really encourage you to read the article and offer your feedback.

However, we have another event to promote right now: NVIDIA's Tom Petersen will be joining me on PCPer Live! at 11am PT / 2pm ET to talk about the GeForce GTX TITAN and its performance, features, pricing and more! 

pcperlive2.png

GeForce GTX TITAN Live Review Stream

11am PT / 2pm ET - February 21st

PC Perspective Live! Page

If you have questions for Tom or me, you can leave them in the comments below (no registration required)!

nvidia1.jpg

Author:
Manufacturer: NVIDIA

TITAN is back for more!

Our NVIDIA GeForce GTX TITAN Coverage Schedule:

If you are reading this today, chances are you were here on Tuesday when we first launched our NVIDIA GeForce GTX TITAN features and preview story (accessible from the link above) and were hoping to find benchmarks then.  You didn't, but you will now.  I am here to show you that the TITAN is indeed the single fastest GPU on the market and MAY be the best graphics cards (single or dual GPU) on the market depending on what usage models you have.  Some will argue, some will disagree, but we have an interesting argument to make about this $999 gaming beast.

A brief history of time...er, TITAN

In our previous article we talked all about TITAN's GK110-based GPU, the form factor, card design, GPU Boost 2.0 features and much more and I would highly press you all to read it before going forward.  If you just want the cliff notes, I am going to copy and paste some of the most important details below.

IMG_9502.JPG

From a pure specifications standpoint the GeForce GTX TITAN based on GK110 is a powerhouse.  While the full GPU sports a total of 15 SMX units, TITAN will have 14 of them enabled for a total of 2688 shaders and 224 texture units.  Clock speeds on TITAN are a bit lower than on GK104 with a base clock rate of 836 MHz and a Boost Clock of 876 MHz.  As we will show you later in this article though the GPU Boost technology has been updated and changed quite a bit from what we first saw with the GTX 680.

The bump in the memory bus width is also key, being able to feed that many CUDA cores definitely required a boost from 256-bit to 384-bit, a 50% increase.  Even better, the memory bus is still running at 6.0 GHz resulting in total memory bandwdith of 288.4 GB/s.

blockdiagram2.jpg

Speaking of memory - this card will ship with 6GB on-board.  Yes, 6 GeeBees!!  That is twice as much as AMD's Radeon HD 7970 and three times as much as NVIDIA's own GeForce GTX 680 card.  This is without a doubt a nod to the super-computing capabilities of the GPU and the GPGPU functionality that NVIDIA is enabling with the double precision aspects of GK110.

Continue reading our full review of the NVIDIA GeForce GTX TITAN graphics card with benchmarks and an update on our Frame Rating process!!