Review Index:
Feedback

The New 3DMark Benchmark - Testing Smartphones to Multi-GPU Gaming PCs

Author:
Manufacturer: Futuremark

The Ice Storm Test

Love it or hate it, 3DMark has a unique place in the world of PC gaming and enthusiasts.  Since 3DMark99 was released...in 1998...with a target on DirectX 6, Futuremark has been developing benchmarks on a regular basis in time with major API changes and also major harware changes.  The most recent release of 3DMark11 has been out since late in 2010 and has been a regular part of our many graphics card reviews on PC Perspective

Today Futuremark is not only releasing a new version of the benchmark but is also taking fundamentally different approach to performance testing and platforms.  The new 3DMark, just called "3DMark", will not only target high-end gaming PCs but integrated graphics platforms and even tablets and smartphones. 

We interviewed the President of Futuremark, Oliver Baltuch, over the weekend and asked some questions about this new direction for 3DMark, how mobile devices were going to affect benchmarks going forward and asked about the new results patterns, stuttering and more.  Check out the video below!

Video Loading...

Make no bones about it, this is a synthetic benchmark and if you have had issues with that in the past because it is not a "real world" gaming test, you will continue to have those complaints.  Personally I see the information that 3DMark provides to be very informative though it definitely shouldn't be depended on as the ONLY graphics performance metric. 

Continue reading our story on the new 3DMark benchmark and the first performance results!!

A Unified Approach to a Benchmark

View Full Size

The new 3DMark is now made up of three unique scenarios, each with a pair of graphics tests, a physics test in an attempt to judge both GPU and CPU performance.  What you might find interesting is that Ice Storm, the most basic, is able to be run on mobile devices with Android and iOS as well as x86 based PCs.  There are more details below on how Futuremark is able to do that with unique ideas like rendering to common frame size (1280x720) regardless of the actual SCREEN size.  When those versions of the test are available we should have a great data point to compare the performance of just about any device.

Cloud Gate is another test with mid-range assets and is meant for integrated graphics comparisons like AMD's APUs and Intel's HD graphics.  You can still run it on discrete graphics systems as well but as you'll see in our results there are some bottlenecks on the CPU side.

 

The New User Interface

View Full Size

Luckily for us Futuremark didn't go with a "button" style design to match the release of Windows 8 but there are still some interesting changes compared to previous versions - it is simplified and streamlined.  There is the capability to easily "run all tests" though Futuremark says that shouldn't be used often - there is little use to run Fire Strike and Ice Storm on the same system.  We did that for informative reasons but you'll see why that won't be necessary going forward.

View Full Size

Running each test is as easy as hitting a single button and you can choose to disable the demo that runs but doesn't report any performance metrics.  Only the Fire Strike test, meant for enthusiast PCs, has a secondary quality setting, which we'll detail later. 

View Full Size

Futuremark still has flexibility here to run a specific test with different quality settings but obviously you'll only get an official score when you run one of the presets. 

View Full Size

For professional users 3DMark does continue to include the ability render a specific frame to be used for image quality comparisons from system to system.

 

Below are details about the various benchmarks, Ice Storm, Cloud Gate and Fire Strike.  The information comes directly from technical documentation provided by Futuremark with some comments from me (that will NOT be in italics) along the way.

Ice Storm Benchmark

View Full Size

Check out a recording of the Ice Storm benchmark!

Ice Storm is a cross-platform benchmark for mobile devices. Use it to test the performance of your smartphone, tablet, ultra-portable notebook or entry-level PC. Ice Storm includes two graphics tests focusing on GPU performance and a physics test targeting CPU performance.

On Windows, Ice Storm uses a DirectX 11 engine limited to Direct3D feature level 9, making it the ideal benchmark for modern portable devices targeting that feature level.

On Android and iOS, Ice Storm uses OpenGL ES 2.0.

Ice Storm's test content, settings and rendering resolution are the same on all platforms and scores can be compared across Windows, Windows RT, Android and iOS.

View Full Size

Graphics test 1

Ice Storm Graphics test 1 stresses the hardware’s ability to process lots of vertices while keeping the pixel load relatively light.

Hardware on this level may have dedicated capacity for separate vertex and pixel processing. Stressing both capacities individually reveals the hardware’s limitations in both aspects.

In an average frame, 530,000 vertices are processed leading to 180,000 triangles rasterized either to the shadow map or to the screen. At the same time, 4.7 million pixels are processed per frame.
Pixel load is kept low by excluding expensive post processing steps, and by not rendering particle effects.

Graphics test 2

Graphics test 2 stresses the hardware’s ability to process lots of pixels. It tests the ability to read textures, do per pixel computations and write to render targets.

On average, 12.6 million pixels are processed per frame. The additional pixel processing compared to Graphics test 1 comes from including particles and post processing effects such as bloom, streaks and motion blur.

In each frame, an average 75,000 vertices are processed. This number is considerably lower than in Graphics test 1 because shadows are not drawn and the processed geometry has a lower number of polygons.

Physics test

The purpose of the Physics test is to benchmark the hardware’s ability to do gameplay physics simulations on CPU. The GPU load is kept as low as possible to ensure that only the CPU’s capabilities are stressed.

The test has four simulated worlds. Each world has two soft bodies and two rigid bodies colliding with each other. One thread per available CPU core is used to run simulations. All physics are computed on the CPU with soft body vertex data updated to the GPU each frame. The background is drawn as a static image for the least possible GPU load.

The Physics test uses the Bullet Open Source Physics Library.

February 4, 2013 | 10:01 AM - Posted by Humanitarian

"Y'know?"

Yeah, I know.

February 4, 2013 | 12:53 PM - Posted by ezjohny

This sounds very interesting for a hardware bench mark that targets all platforms, I will be using fire strike for my desktop PC gentleman.

February 5, 2013 | 09:33 AM - Posted by xbeaTX (not verified)

Hi Ryan!...there are rumors that you have received a special package from Nvidia .... fits well with this benchmark? :P

February 5, 2013 | 03:54 PM - Posted by Ryan Shrout

No clue what you are talking about!  :)

February 6, 2013 | 08:45 AM - Posted by Raghunathan (not verified)

if a HD 7970(925 mhz) is faster than GTX 680 then the HD 7970 Ghz is pretty much the boss in this test. I have always felt that Nvidia got away this generation with the majority of sales and accolades as AMD did not get their launch and early drivers (first 6 months till June 2012) performance as good as it could have been. From 12.11 beta onwards AMD is showing how capable their hardware is.

March 22, 2014 | 07:56 AM - Posted by Katie (not verified)

I'd like to thank you for the efforts you've put in writing this site.
I'm hoping to check out the same high-grade blog posts by you
in the future as well. In fact, your creative writing abilities has motivated me to get my very own website now
;)

Have a look at my website ... gate 2014 result

February 6, 2013 | 09:45 AM - Posted by YTech2 (not verified)

Wow, PCper Advertisement on a PCper video channel :)

Nice show! I enjoy these types of learning discussions.

February 8, 2013 | 05:40 AM - Posted by BiggieShady

Hey Ryan, correct me if I'm wrong, but It doesn't seem that 3Dmark graph shows frametime fluctuations consistent with inside-the-second analysis as Oliver suggested when you asked about it. It's more like the graph shows average fps in a coresponding time window which is one second or even larger.

February 9, 2013 | 09:20 AM - Posted by Ryan Shrout

You are correct. It doesn't go into the detail we'll want for our future testing but we can use our other tools for that a bit later.

February 13, 2013 | 03:45 AM - Posted by Brash (not verified)

Ryan, do you see your framerating benchmark as having relevance to this new 3dMark software? It would be interesting to merge the two.

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Lines and paragraphs break automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd> <blockquote><p><br>
  • Web page addresses and e-mail addresses turn into links automatically.

More information about formatting options

By submitting this form, you accept the Mollom privacy policy.