Our first DX12 Performance Results
Late last week, Microsoft approached me to see if I would be interested in working with them and with Futuremark on the release of the new 3DMark API Overhead Feature Test. Of course I jumped at the chance, with DirectX 12 being one of the hottest discussion topics among gamers, PC enthusiasts and developers in recent history. Microsoft set us up with the latest iteration of 3DMark and the latest DX12-ready drivers from AMD, NVIDIA and Intel. From there, off we went.
First we need to discuss exactly what the 3DMark API Overhead Feature Test is (and also what it is not). The feature test will be a part of the next revision of 3DMark, which will likely ship in time with the full Windows 10 release. Futuremark claims that it is the "world's first independent" test that allows you to compare the performance of three different APIs: DX12, DX11 and even Mantle.
It was almost one year ago that Microsoft officially unveiled the plans for DirectX 12: a move to a more efficient API that can better utilize the CPU and platform capabilities of future, and most importantly current, systems. Josh wrote up a solid editorial on what we believe DX12 means for the future of gaming, and in particular for PC gaming, that you should check out if you want more background on the direction DX12 has set.
One of DX12 keys for becoming more efficient is the ability for developers to get closer to the metal, which is a phrase to indicate that game and engine coders can access more power of the system (CPU and GPU) without having to have its hand held by the API itself. The most direct benefit of this, as we saw with AMD's Mantle implementation over the past couple of years, is improved quantity of draw calls that a given hardware system can utilize in a game engine.
Subject: Graphics Cards, Processors | March 5, 2015 - 01:46 AM | Ryan Shrout
Tagged: GDC, gdc 15, API, dx12, DirectX 12, dx11, Mantle, 3dmark, Futuremark
It's probably not a surprise to most that Futuremark is working on a new version of 3DMark around the release of DirectX 12. What might be new for you is that this version will include an API overhead test, used to evaluate a hardware configuration's ability to affect performance in Mantle, DX11 and DX12 APIs.
While we don't have any results quite yet (those are pending and should be very soon), Intel was showing the feature test running at an event at GDC tonight. In what looks like a simple cityscape being rendered over and over, the goal is to see how many draw calls, or how fast the CPU can react to a game engine, the API and hardware can be.
The test was being showcased on an Intel-powered notebook using a 5th Generation Core processor, code named Broadwell. Obviously this points to the upcoming support for DX12 (though obviously not Mantle) that Intel's integrated GPUs will provide.
It should be very interesting to see how much of an advantage DX12 offers over DX11, even on Intel's wide ranges of consumer and enthusiast processors.
Subject: General Tech, Graphics Cards, Processors, Mobile | August 23, 2013 - 05:31 AM | Scott Michaud
Tagged: Futuremark, AnTuTu, benchmarking
VR-Zone tossed the bees nest in a paint shaker and received a fairly sedate outcome.
A little background information is required. AnTuTu, a mobile benchmark developed by AnTuTu Labs, has been accused of inaccurate scores and bias towards specific hardware. Leaked BayTrail-T benchmarks, surpassing our expectations of Intel's capabilities, were harshly refuted based on AnTuTu's credibility. More recently, certain Samsung GPUs have been allegedly recorded self-overclocking during that benchmark but not elsewhere.
Scene from Cloud Gate, latest Windows 3DMark.
Oliver Baltuch, president of Futuremark, accepted an interview with VR-Zone to discuss business and ethics in their marketplace. Futuremark is a direct competitor to AnTuTu and a household name in the benchmarking community. Being modest Fins, self-proclaimed, they did not wish to discuss whether AnTuTu was less honest than they are. Futuremark does disagree with AnTuTu's process, however, and has some suggestions for better results.
The design process for 3D Mark Android begun with 25 pages of specification proposal. Each vendor is given a chance to reply to that proposal and these responses are compared. Changes to the specification must be reviewed by a committee sitting between the financial department and the engineering department.
Baltuch made the point that all of their finances for the last five years, according to Finnish law, can be reviewed for about $7 USD. Despite being a private company, the law mandates no deals can be made in secret.
On the engineering side of things, drivers are approved only if they follow specific guidelines. Unapproved results will be removed from their website and leaderboards followed by a polite conversation with the manufacturer. Drivers are not allowed to identify their benchmarks intent on modifying settings due to that information.
Almost every benchmark they release gets negative responses from some upset vendor or vendors.
The relatively short interview is wrapped up with commentary on iOS benchmarks. Futuremark is nearing completion of their first benchmarking app. Apple disallows apps to exceed 60 frames per second, through vsync, which unnecessarily hinders benchmark scores. Working around this, Futuremark developed a method to render frames which are not displayed on screen to keep the processors from idling once at frame rate cap.
Ryan must love that idea...
This concept has, according to the interview, reached internal QA review and is expected to be released in a few weeks.
Futuremark develops benchmarks for x86 Windows, Windows RT, Android, and iOS. Scores are intended to scale linearly to their metrics and are designed to allow cross-platform performance comparisons.
Subject: General Tech | April 3, 2013 - 10:53 AM | Tim Verry
Tagged: ice storm extreme, ice storm, Futuremark, benchmarking, Android, 3dmark
Futuremark recently unveiled its latest 3DMark benchmarking suite for Android devices. Compatible with over 1,000 devices, the new 3DMark is a free benchmark that incorporates both the Ice Storm and Ice Storm Extreme tests. The benchmark was developed by Futuremark in cooperation with a number of industry companies including Broadcom, Imagination Technologies, Intel, NVIDIA, and Qualcomm. The Ice Storm Extreme test is also coming to the Windows version of 3DMark, and the tests can be used to compare benchmark scores across platforms.
Both the benchmarking tests are based on OpenGL ES 2.0. Ice Storm runs through two graphical tests to stress the GPU and one physics test to measure CPU performance. The ice Storm Extreme benchmark takes things further by bumping up the resolution to 1080 and swapping in higher quality textures and post processing effects.
The benchmark is compatible with a number of mobile smartphones and tablets running Android 3.1 or higher. It is a free download from the Google Play store.
The iOS and Windows RT versions of 3DMark are still in development. More information can be found in the press release.
Read more about Futuremark's 3DMark benchmarking suite at PC Perspective.
The Ice Storm Test
Love it or hate it, 3DMark has a unique place in the world of PC gaming and enthusiasts. Since 3DMark99 was released...in 1998...with a target on DirectX 6, Futuremark has been developing benchmarks on a regular basis in time with major API changes and also major harware changes. The most recent release of 3DMark11 has been out since late in 2010 and has been a regular part of our many graphics card reviews on PC Perspective.
Today Futuremark is not only releasing a new version of the benchmark but is also taking fundamentally different approach to performance testing and platforms. The new 3DMark, just called "3DMark", will not only target high-end gaming PCs but integrated graphics platforms and even tablets and smartphones.
We interviewed the President of Futuremark, Oliver Baltuch, over the weekend and asked some questions about this new direction for 3DMark, how mobile devices were going to affect benchmarks going forward and asked about the new results patterns, stuttering and more. Check out the video below!
Make no bones about it, this is a synthetic benchmark and if you have had issues with that in the past because it is not a "real world" gaming test, you will continue to have those complaints. Personally I see the information that 3DMark provides to be very informative though it definitely shouldn't be depended on as the ONLY graphics performance metric.
Subject: Graphics Cards | February 2, 2013 - 08:29 AM | Tim Verry
Tagged: gpus, gaming, Futuremark, benchmarking, 3dmark
Futuremark, developers of the popular 3DMark and PCMark computer hardware benchmarks has announced an official release date for the next version of 3DMark. The company has teased gamers and reviewers with screenshots as well as hinted that the name would no longer have the release year tacked onto the end of the name, but now the benchmark is finally official.
The new 3DMark will come in several different flavors aimed at Windows PCs, iOS, Android, and Windows RT devices. It will continue the trend of offline benchmarking and scoring paired with a web interface where users can see detailed benchmark run analysis.
The new 3DMark benchmark will include feature tests, a DX10 benchmark called Cloud Gate, and a DX11 benchmark called Fire Strike. Once the benchmark has completed, users will be able to dig into the web interface to access charts and graphs that cover the benchmarking runs from beginning to end. The graphs will track CPU clockspeed and utilization as well as temperatures for both the processor and graphics card(s).
On the mobile side of things, 3DMark will use a graphics test called Ice Storm that is more suited to ARM SoCs with integrated graphics processors. No DX11 goodness here, obviously.
The PC version of 3DMark will be available for download on February 4, 2013 at 18:00 UTC. Unfortunately, there is no official release dates for the mobile versions. Futuremark has indicated that they will be released over the next few weeks as they are finalized.
You can find more information on the next 3DMark benchmark on the Futuremark website.
Subject: General Tech, Storage | June 29, 2012 - 07:25 PM | Jeremy Hellstrom
Tagged: hdd, Futuremark, thailand
While it is easy to understand why the destruction of a good portion of the HDD industries manufacturing capabilities caused by the flooding in Thailand would effect both the availability and pricing of HDDs it is not so easy to explain what those manufacturers are doing now. It is not just the reduction in warranty to 1 year which we previously informed you about, it is the bizarre pricing which adds to the confusion. This is an industry which has collapsed into two major players, with two others appearing to compete but in reality are working with or outright owned by the two major players. They are under siege from the SSD industry which offers longer warranty, better performance and prices which are falling quickly; making the high prices and lousy warranty offered by HDD manufactures quite unattractive. The Tech Report assembled an array of graphs which display the state of the hard drive companies as well as some suggestions on the best current deals in HDDs if you are inclined to pick one up.
"Mechanical hard drive prices rose sharply after last year's Thailand flooding. Prices have fallen since, but their decline has slowed in recent months. We take a closer look at the numbers."
Here is some more Tech News from around the web:
- Fiberglass-reinforced cases expected to be adopted for ultrabooks in 2H12 @ DigiTimes
- Adobe Stops Flash Player Support For Android @ Slashdot
- Techies evac'd as raging wildfire menaces $100m Colorado data centre @ The Register
- Raspberry Pi enclosure turns it into a desktop PC @ Hack a Day
- Netgear WNDR4500 Dual Band Gigabit Router @ X-bit Labs
- I, Cyborg @ The Tech Report
- Win the KFA2 GeForce GTX 680 LTD OC 2048MB @ Kitguru
Subject: General Tech | June 21, 2012 - 02:43 PM | Tim Verry
Tagged: windows 8, windows, Futuremark, directx 11, benchmarking, 3dmark
Popular benchmarking software developers Futuremark recently posted a video of latest 3DMark tech demo. Premiering in its Windows 8 benchmarking software, the tech demo uses complex volumetric lighting with real time scattering, tessellation, visible particles and clouds of smoke. It also uses fluid dynamics, audio by Pedro Macedo Camacho (who also created the 3DMark 11 soundtrack), ambient occlusion, and post processing. Whew, that’s a lot of shiny graphics!
We posted a few screenshots of the tech demo that showed up online a few weeks ago, and now it seems like the company is ready to show it off in video form. The embedded video below shows a mysterious figure walking through a small town nestled in a canyon with smoke, lava, and a flying robot to keep her company. The graphics are very detailed and the particle and fluid physics look really good. It should do a great job of stressing out your graphics cards when it comes out in the latest 3DMark.
Unfortunately, not much is known as far as specific release dates, or even if it will be called 3DMark 12 (or 3DMark for Windows 8). If you are into benchmarking software though, keep your eyes on Futuremark’s website as they release more details.
Subject: General Tech, Mobile | March 29, 2012 - 11:24 PM | Scott Michaud
Tagged: Rovio, Futuremark
Rovio Entertainment purchases Futuremark Games Studio, but not all of Futuremark, for an undisclosed amount.
Rovio is known for making graphically lightweight yet intensely addictive games for about as many desktop and mobile platforms as they can get away with. Futuremark Games Studio is known for making beautiful PC games which are entertaining to some extent. Naturally they make a perfect couple.
So *that’s* how it got shattered!
Of course the real topic for discussion is why Rovio would want to purchase Futuremark Games Studio. One possible reason is that Rovio wishes to challenge Infinity Blade by Epic Games and capture the market of mobile eye-candy games. The other possibility would be that Rovio wishes to expand into making large budget games themselves.
In their purchase, Rovio has only acquired the studio but not any of their intellectual properties. Shattered Horizon and their other games remain property of the original parent company, Futuremark.
What do you speculate is just over the Shattered Horizon?
Subject: General Tech | May 12, 2011 - 07:32 PM | Tim Verry
Tagged: Windows 7, SSD test, performance testing, PCMark 7, Futuremark, benchmark
When it comes to hardware testing, PCMark is a widely known and tested benchmarking suite. Developer Futuremark has now deployed PCMark 7 for WIndows 7 alongside PCMark05 for XP and PCMark Vantage for Vista users.
Designed to test a wide range of hardware from low cost notebooks to high performance gaming systems, PCMark utilizes numerous subsystem tests to provide a final composite score for the computer which can then be accurately compared to other users’ scores.
In the same manner as its predecessor PCMark Vantage, PCMark 7 uses traces of actual Windows’ programs to score a system based on actual usage scenarios. For example, the benchmark suite’s storage tests have been designed to allow both home and business users the ability to compare benchmark scores across systems and upgrades (of the same system). Whether using a solid state drive or a mechanical hard drive, PCMark 7 uses recordings of actions in well known Windows applications, including “Microsoft Word, Windows Live Photo Gallery, Windows Live Movie Maker, Windows Media Center, Windows Media Player, Internet Explorer, Windows Defender (Security Essentials) and even World of Warcraft” to replicate how someone would use the computer in a real world situation. The reasoning behind the use of program traces versus pure synthetic testing is the reliability and benefits of real world comparison. Especially when comparing benchmark scores between a base and upgraded system, synthetic benchmarking can show the potential performance increases; however, program traces can more closely showcase the real world performance increases.
With three versions of the benchmarking suite, there is a version to fit various needs and budgets. The Basic Version, Advanced Version ($39.95), and Professional Version ($995.00) offer increased control over the process. Each can be purchased or downloaded from PCMark.com.
"A benchmark is a highly complex and sophisticated piece of software, yet PCMark 7 is easy to use and requires no specialist knowledge or set up," said Jani Joki, Director of PC Products and Services at Futuremark. "Better yet, PCMark 7 Basic Edition is available as a free download so all PC users can benefit from this industrial strength PC test."
Will you be using PCMark 7 in your next benchmarking run?