The New 3DMark Benchmark - Testing Smartphones to Multi-GPU Gaming PCs
The Ice Storm Test
Love it or hate it, 3DMark has a unique place in the world of PC gaming and enthusiasts. Since 3DMark99 was released...in 1998...with a target on DirectX 6, Futuremark has been developing benchmarks on a regular basis in time with major API changes and also major harware changes. The most recent release of 3DMark11 has been out since late in 2010 and has been a regular part of our many graphics card reviews on PC Perspective.
Today Futuremark is not only releasing a new version of the benchmark but is also taking fundamentally different approach to performance testing and platforms. The new 3DMark, just called "3DMark", will not only target high-end gaming PCs but integrated graphics platforms and even tablets and smartphones.
We interviewed the President of Futuremark, Oliver Baltuch, over the weekend and asked some questions about this new direction for 3DMark, how mobile devices were going to affect benchmarks going forward and asked about the new results patterns, stuttering and more. Check out the video below!
Make no bones about it, this is a synthetic benchmark and if you have had issues with that in the past because it is not a "real world" gaming test, you will continue to have those complaints. Personally I see the information that 3DMark provides to be very informative though it definitely shouldn't be depended on as the ONLY graphics performance metric.
A Unified Approach to a Benchmark
The new 3DMark is now made up of three unique scenarios, each with a pair of graphics tests, a physics test in an attempt to judge both GPU and CPU performance. What you might find interesting is that Ice Storm, the most basic, is able to be run on mobile devices with Android and iOS as well as x86 based PCs. There are more details below on how Futuremark is able to do that with unique ideas like rendering to common frame size (1280x720) regardless of the actual SCREEN size. When those versions of the test are available we should have a great data point to compare the performance of just about any device.
Cloud Gate is another test with mid-range assets and is meant for integrated graphics comparisons like AMD's APUs and Intel's HD graphics. You can still run it on discrete graphics systems as well but as you'll see in our results there are some bottlenecks on the CPU side.
The New User Interface
Luckily for us Futuremark didn't go with a "button" style design to match the release of Windows 8 but there are still some interesting changes compared to previous versions - it is simplified and streamlined. There is the capability to easily "run all tests" though Futuremark says that shouldn't be used often - there is little use to run Fire Strike and Ice Storm on the same system. We did that for informative reasons but you'll see why that won't be necessary going forward.
Running each test is as easy as hitting a single button and you can choose to disable the demo that runs but doesn't report any performance metrics. Only the Fire Strike test, meant for enthusiast PCs, has a secondary quality setting, which we'll detail later.
Futuremark still has flexibility here to run a specific test with different quality settings but obviously you'll only get an official score when you run one of the presets.
For professional users 3DMark does continue to include the ability render a specific frame to be used for image quality comparisons from system to system.
Below are details about the various benchmarks, Ice Storm, Cloud Gate and Fire Strike. The information comes directly from technical documentation provided by Futuremark with some comments from me (that will NOT be in italics) along the way.
Ice Storm Benchmark
Check out a recording of the Ice Storm benchmark!
Ice Storm is a cross-platform benchmark for mobile devices. Use it to test the performance of your smartphone, tablet, ultra-portable notebook or entry-level PC. Ice Storm includes two graphics tests focusing on GPU performance and a physics test targeting CPU performance.
On Windows, Ice Storm uses a DirectX 11 engine limited to Direct3D feature level 9, making it the ideal benchmark for modern portable devices targeting that feature level.
On Android and iOS, Ice Storm uses OpenGL ES 2.0.
Ice Storm's test content, settings and rendering resolution are the same on all platforms and scores can be compared across Windows, Windows RT, Android and iOS.
Graphics test 1
Ice Storm Graphics test 1 stresses the hardware’s ability to process lots of vertices while keeping the pixel load relatively light.
Hardware on this level may have dedicated capacity for separate vertex and pixel processing. Stressing both capacities individually reveals the hardware’s limitations in both aspects.
In an average frame, 530,000 vertices are processed leading to 180,000 triangles rasterized either to the shadow map or to the screen. At the same time, 4.7 million pixels are processed per frame.
Pixel load is kept low by excluding expensive post processing steps, and by not rendering particle effects.
Graphics test 2
Graphics test 2 stresses the hardware’s ability to process lots of pixels. It tests the ability to read textures, do per pixel computations and write to render targets.
On average, 12.6 million pixels are processed per frame. The additional pixel processing compared to Graphics test 1 comes from including particles and post processing effects such as bloom, streaks and motion blur.
In each frame, an average 75,000 vertices are processed. This number is considerably lower than in Graphics test 1 because shadows are not drawn and the processed geometry has a lower number of polygons.
The purpose of the Physics test is to benchmark the hardware’s ability to do gameplay physics simulations on CPU. The GPU load is kept as low as possible to ensure that only the CPU’s capabilities are stressed.
The test has four simulated worlds. Each world has two soft bodies and two rigid bodies colliding with each other. One thread per available CPU core is used to run simulations. All physics are computed on the CPU with soft body vertex data updated to the GPU each frame. The background is drawn as a static image for the least possible GPU load.
The Physics test uses the Bullet Open Source Physics Library.