Frame Rating Dissected: Full Details on Capture-based Graphics Performance Testing
The Overlay Software
Now, let’s dive into the process of going from a game to the graphs we will be showing you on the coming pages. We want to be completely transparent in detailing this methadology, knowing there will be questions and those that question us.
The overlay is really a very simple idea: take a solid color bar and apply it to the left hand side of every frame as it leaves the game engine, but before it gets into the graphics abstraction. By changing the color that is applied to the frame in a pre-determined, consecutive pattern, you can then produce of data from the result.
Remember that since frame times rarely match the rate of refresh of your monitor exactly (we will discuss Vsync and what it means in another page) the result will be captured frames with multiple colors of bar on the left hand side.
This diagram shows what you can expect, and below is a screenshot of an actual capture used in this review.
Click to Enlarge
Even with just the eye test this kind of data gives us all kind of great information. For example, because we know the pre-determined pattern of colors, we can find any frames that are “dropped” or frames that are asked to be rendered by the game but are never actually shown on the screen. If the pattern we expect to see is “lime-white-orange-red” but we actually see “lime-white-red” then we know that something has happened to the frame rendered with the orange overlay.
Even more interesting is the phenomena we are calling “runts” – frames that are rendered and shown, but of so few scanlines that they could be considered irrelevant to the frame rate, or even detrimental to the animation smoothness.
Click to Enlarge
We actually showed our readers a runt in our first article about Frame Rating. The implications of runts and drops should be pretty apparent to you if you are following our logic thus far, but just to be sure, let’s elaborate. In both cases, drop and runt, FRAPS essentially thinks the frame is being shown to the user like just any other frame. With a drop though, this isn’t the case – the user actually never sees the frame on the screen and thus the FRAPS data is just wrong.
For runts the debate is more interesting; even though the frame is being shown to the user the “value” of that frame is much lower (or gone completely) because it takes up so little space on the screen. If you look at the screenshot above you can see that the thin strip of a frame isn’t able to make any kind of significant animation sequence addition though it counts exactly the same as the frames below and above it when we consider your average frame rate, even your average frame rate per second.
I know some of our readers will be interested to know that we have spent a LOT of time with the overlay and have even had time to swap between having the overlay on and turning it off, all while continuing to capture the video. The visual tearing that is shown above as runts are still apparent without the overlay, though the overlay just makes them easier to detect visually and allows software to easily scan and generate mass data. It should also go without saying that I have seen no indication that the overlay is affecting performance or runt generation over many months of use.