O Rayly? Ya Rayly. No Ray!
Microsoft has just announced a raytracing extension to DirectX 12, called DirectX Raytracing (DXR), at the 2018 Game Developer's Conference in San Francisco.
The goal is not to completely replace rasterization… at least not yet. This effect will be mostly implemented for effects that require supplementary datasets, such as reflections, ambient occlusion, and refraction. Rasterization, the typical way that 3D geometry gets drawn on a 2D display, converts triangle coordinates into screen coordinates, and then a point-in-triangle test runs across every sample. This will likely occur once per AA sample (minus pixels that the triangle can’t possibly cover -- such as a pixel outside of the triangle's bounding box -- but that's just optimization).
For rasterization, each triangle is laid on a 2D grid corresponding to the draw surface.
If any sample is in the triangle, the pixel shader is run.
This example shows the rotated grid MSAA case.
A program, called a pixel shader, is then run with some set of data that the GPU could gather on every valid pixel in the triangle. This set of data typically includes things like world coordinate, screen coordinate, texture coordinates, nearby vertices, and so forth. This lacks a lot of information, especially things that are not visible to the camera. The application is free to provide other sources of data for the shader to crawl… but what?
- Cubemaps are useful for reflections, but they don’t necessarily match the scene.
- Voxels are useful for lighting, as seen with NVIDIA’s VXGI and VXAO.
This is where DirectX Raytracing comes in. There’s quite a few components to it, but it’s basically a new pipeline that handles how rays are cast into the environment. After being queued, it starts out with a ray-generation stage, and then, depending on what happens to the ray in the scene, there are close-hit, any-hit, and miss shaders. Ray generation allows the developer to set up how the rays are cast, where they call an HLSL instrinsic instruction, TraceRay (which is a clever way of invoking them, by the way). This function takes an origin and a direction, so you can choose to, for example, cast rays only in the direction of lights if your algorithm was to, for instance, approximate partially occluded soft shadows from a non-point light. (There are better algorithms to do that, but it's just the first example that came off the top of my head.) The close-hit, any-hit, and miss shaders occur at the point where the traced ray ends.
To connect this with current technology, imagine that ray-generation is like a vertex shader in rasterization, where it sets up the triangle to be rasterized, leading to pixel shaders being called.
Even more interesting – the close-hit, any-hit, and miss shaders can call TraceRay themselves, which is used for multi-bounce and other recursive algorithms (see: figure above). The obvious use case might be reflections, which is the headline of the GDC talk, but they want it to be as general as possible, aligning with the evolution of GPUs. Looking at NVIDIA’s VXAO implementation, it also seems like a natural fit for a raytracing algorithm.
Speaking of data structures, Microsoft also detailed what they call the acceleration structure. Each object is composed of two levels. The top level contains per-object metadata, like its transformation and whatever else data that the developer wants to add to it. The bottom level contains the geometry. The briefing states, “essentially vertex and index buffers” so we asked for clarification. DXR requires that triangle geometry be specified as vertex positions in either 32-bit float3 or 16-bit float3 values. There is also a stride property, so developers can tweak data alignment and use their rasterization vertex buffer, as long as it's HLSL float3, either 16-bit or 32-bit.
As for the tools to develop this in…
Microsoft announced PIX back in January 2017. This is a debugging and performance analyzer for 64-bit, DirectX 12 applications. Microsoft will upgrade it to support DXR as soon as the API is released (specifically, “Day 1”). This includes the API calls, the raytracing pipeline resources, the acceleration structure, and so forth. As usual, you can expect Microsoft to support their APIs with quite decent – not perfect, but decent – documentation and tools. They do it well, and they want to make sure it’s available when the API is.
Example of DXR via EA's in-development SEED engine.
In short, raytracing is here, but it’s not taking over rasterization. It doesn’t need to. Microsoft is just giving game developers another, standardized mechanism to gather supplementary data for their games. Several game engines have already announced support for this technology, including the usual suspects of anything top-tier game technology:
- Frostbite (EA/DICE)
- SEED (EA)
- 3DMark (Futuremark)
- Unreal Engine 4 (Epic Games)
- Unity Engine (Unity Technologies)
They also said, “and several others we can’t disclose yet”, so this list is not even complete. But, yeah, if you have Frostbite, Unreal Engine, and Unity, then you have a sizeable market as it is. There is always a question about how much each of these engines will support the technology. Currently, raytracing is not portable outside of DirectX 12, because it’s literally being announced today, and each of these engines intend to support more than just Windows 10 and Xbox.
Still, we finally have a standard for raytracing, which should drive vendors to optimize in a specific direction. From there, it's just a matter of someone taking the risk to actually use the technology for a cool work of art.
If you want to read more, check out Ryan's post about the also-announced RTX, NVIDIA's raytracing technology.
An Data Format for Whole 3D Scenes
The Khronos Group has finalized the glTF 2.0 specification, and they recommend that interested parties integrate this 3D scene format into their content pipeline starting now. It’s ready.
glTF is a format to deliver 3D content, especially full scenes, in a compact and quick-loading data structure. These features differentiate glTF from other 3D formats, like Autodesk’s FBX and even the Khronos Group’s Collada, which are more like intermediate formats between tools, such as 3D editing software (ex: Maya and Blender) and game engines. They don’t see a competing format for final scenes that are designed to be ingested directly, quick and small.
glTF 2.0 makes several important changes.
The previous version of glTF was based on a defined GLSL material, which limited how it could be used, although it did align with WebGL at the time (and that spurred some early adoption). The new version switches to Physically Based Rendering (PBR) workflows to define their materials, which has a few advantages.
First, PBR can represent a wide range of materials with just a handful of parameters. Rather than dictating a specific shader, the data structure can just... structure the data. The industry has settled on two main workflows, metallic-roughness and specular-gloss, and glTF 2.0 supports them both. (Metallic-roughness is the core workflow, but specular-gloss is provided as an extension, and they can be used together in the same scene. Also, during the briefing, I noticed that transparency was not explicitly mentioned in the slide deck, but the Khronos Group confirmed that it is stored as the alpha channel of the base color, and thus supported.) Because the format is now based on existing workflows, the implementation can be programmed in OpenGL, Vulkan, DirectX, Metal, or even something like a software renderer. In fact, Microsoft was a specification editor on glTF 2.0, and they have publicly announced using the format in their upcoming products.
The original GLSL material, from glTF 1.0, is available as an extension (for backward compatibility).
A second advantage of PBR is that it is lighting-independent. When you define a PBR material for an object, it can be placed in any environment and it will behave as expected. Noticeable, albeit extreme examples of where this would have been useful are the outdoor scenes of Doom 3, and the indoor scenes of Battlefield 2. It also simplifies asset creation. Some applications, like Substance Painter and Quixel, have artists stencil materials onto their geometry, like gold, rusted iron, and scuffed plastic, and automatically generate the appropriate textures. It also aligns well with deferred rendering, see below, which performs lighting as a post-process step and thus skip pixels (fragments) that are overwritten.
PBR Deferred Buffers in Unreal Engine 4 Sun Temple.
Lighting is applied to these completed buffers, not every fragment.
glTF 2.0 also improves support for complex animations by adding morph targets. Most 3D animations, beyond just moving, rotating, and scaling whole objects, are based on skeletal animations. This method works by binding vertexes to bones, and moving, rotating, and scaling a hierarchy of joints. This works well for humans, animals, hinges, and other collections of joints and sockets, and it was already supported in glTF 1.0. Morph targets, on the other hand, allow the artist to directly control individual vertices between defined states. This is often demonstrated with a facial animation, interpolating between smiles and frowns, but, in an actual game, this is often approximated with skeletal animations (for performance reasons). Regardless, glTF 2.0 now supports morph targets, too, letting the artists make the choice that best suits their content.
Speaking of performance, the Khronos Group is also promoting “enhanced performance” as a benefit of glTF 2.0. I asked whether they have anything to elaborate on, and they responded with a little story. While glTF 1.0 validators were being created, one of the engineers compiled a list of design choices that would lead to minor performance issues. The fixes for these were originally supposed to be embodied in a glTF 1.1 specification, but PBR workflows and Microsoft’s request to abstract the format away from GLSL lead to glTF 2.0, which is where the performance optimization finally ended up. Basically, there wasn’t just one or two changes that made a big impact; it was the result of many tiny changes that add up.
Also, the binary version of glTF is now a core feature in glTF 2.0.
The slide looks at the potential future of glTF, after 2.0.
Looking forward, the Khronos Group has a few items on their glTF roadmap. These did not make glTF 2.0, but they are current topics for future versions. One potential addition is mesh compression, via the Google Draco team, to further decrease file size of 3D geometry. Another roadmap entry is progressive geometry streaming, via Fraunhofer SRC, which should speed up runtime performance.
Yet another roadmap entry is “Unified Compression Texture Format for Transmission”, specifically Basis by Binomial, for texture compression that remains as small as possible on the GPU. Graphics processors can only natively operate on a handful of formats, like DXT and ASTC, so textures need to be converted when they are loaded by an engine. Often, when a texture is loaded at runtime (rather than imported by the editor) it will be decompressed and left in that state on the GPU. Some engines, like Unity, have a runtime compress method that converts textures to DXT, but the developer needs to explicitly call it and the documentation says it’s lower quality than the algorithm used by the editor (although I haven’t tested this). Suffices to say, having a format that can circumvent all of that would be nice.
Again, if you’re interested in adding glTF 2.0 to your content pipeline, then get started. It’s ready. Microsoft is doing it, too.
Podcast #435 - Qualcomm aptX, FSP Twin 500w PSU, Micro 5100 Enteprise SSDs, AMD Fiscal Results, ASUS Tinker Board, ZeniMax
Subject: Editorial | February 2, 2017 - 10:34 AM | Alex Lustenberg
Tagged: podcast, zenimax, UHD Blu-Ray, toshiba, tinker board, Reundant PSU, qualcomm, micron, Laser Networking, fsp, enterprise ssd, DirectX, delidding, asus, aptX, amd
PC Perspective Podcast #435 - 02/02/17
Join us this week as we discuss Qualcomm aptX, FSP Reundant PSUs, Micron Enterprise SSDs, 5G LTE, AMD Fiscal Year, ZeniMax lawsuit, and more!
The URL for the podcast is: http://pcper.com/podcast - Share with your friends!
- iTunes - Subscribe to the podcast directly through the iTunes Store (audio only)
- Google Play - Subscribe to our audio podcast directly through Google Play!
- RSS - Subscribe through your regular RSS reader (audio only)
- MP3 - Direct download link to the MP3 file
Hosts: Allyn Malventano, Ken Addison, Josh Walrath, Jermey Hellstrom, Sebastian Peak
Program length: 1:46:22
Subject: Graphics Cards | January 27, 2017 - 09:19 PM | Scott Michaud
Tagged: microsoft, DirectX, llvm, dxil, spir-v, vulkan
Over the holidays, Microsoft has published the DirectX Shader Compiler onto GitHub. The interesting part about this is that it outputs HLSL into DirectX Intermediate Language (DXIL) bytecode, which can be ingested by GPU drivers and executed on graphics devices. The reason why this is interesting is that DXIL is based on LLVM, which might start to sound familiar if you have been following along with The Khronos Group and their announcements regarding Vulkan, OpenCL, and SPIR-V.
As it turns out, they were on to something, and Microsoft is working on a DirectX analogue of it.
The main advantage of LLVM-based bytecode is that you can eventually support multiple languages (and the libraries of code developed in them). When SPIR-V was announced with Vulkan, the first thing that came to my mind was compiling to it from HLSL, which would be useful for existing engines, as they are typically written in HLSL and transpiled to the target platform when used outside of DirectX (like GLSL for OpenGL). So, in Microsoft’s case, it would make sense that they start there (since they own the thing) but I doubt that is the end goal. The most seductive outcome for game engine developers would be single-source C++, but there is a lot of steps between there and here.
Another advantage, albeit to a lesser extent, is that you might be able to benefit from performance optimizations, both on the LLVM / language side as well as on the driver’s side.
According to their readme, the minimum support will be HLSL Shader Model 6. This is the most recent shading model, and it introduces some interesting instructions, typically for GPGPU applications, that allow multiple GPU threads to interact, like balloting. Ironically, while DirectCompute and C++AMP don’t seem to be too popular, this would nudge DirectX 12 into a somewhat competent GPU compute API.
DXIL support is limited to Windows 10 Build 15007 and later, so you will need to either switch one (or more) workstation(s) to Insider, or wait until it launches with the Creators Update (unless something surprising holds it back).
Why Two 4GB GPUs Isn't Necessarily 8GB
We're trying something new here at PC Perspective. Some topics are fairly difficult to explain cleanly without accompanying images. We also like to go fairly deep into specific topics, so we're hoping that we can provide educational cartoons that explain these issues.
This pilot episode is about load-balancing and memory management in multi-GPU configurations. There seems to be a lot of confusion around what was (and was not) possible with DirectX 11 and OpenGL, and even more confusion about what DirectX 12, Mantle, and Vulkan allow developers to do. It highlights three different load-balancing algorithms, and even briefly mentions what LucidLogix was attempting to accomplish almost ten years ago.
If you like it, and want to see more, please share and support us on Patreon. We're putting this out not knowing if it's popular enough to be sustainable. The best way to see more of this is to share!
DirectX 12 Has No More Secrets
The DirectX 12 API is finalized and the last of its features are known. Before the BUILD conference, the list consisted of Conservative Rasterization, Rasterizer Ordered Viewed, Typed UAV Load, Volume Tiled Resources, and a new Tiled Resources revision for non-volumetric content. When the GeForce GTX 980 launched, NVIDIA claimed it would be compatible with DirectX 12 features. Enthusiasts were skeptical, because Microsoft did not officially finalize the spec at the time.
Last week, Microsoft announced the last feature of the graphics API: Multiadapter.
We already knew that Multiadapter existed, at least to some extent. It is the part of the specification that allows developers to address multiple graphics adapters to split tasks between them. In DirectX 11 and earlier, secondary GPUs would remain idle unless the graphics driver sprinkled some magic fair dust on it with SLI, CrossFire, or Hybrid CrossFire. The only other way to access this dormant hardware was by spinning up an OpenCL (or similar compute API) context on the side.
Subject: General Tech, Graphics Cards | January 23, 2015 - 07:11 PM | Scott Michaud
Tagged: windows 10, microsoft, dx12, DirectX 12, DirectX
Microsoft has added DirectX 12 with the latest Windows 10 Technical Preview that was released today. Until today, DXDIAG reported DirectX 11 in the Windows 10 Technical Preview. At the moment, there has not been any drivers or software released for it, and the SDK is also no-where to be found. Really, all this means is that one barrier has been lifted, leaving the burden on hardware and software partners (except to release the SDK, that's still Microsoft's responsibility).
No-one needs to know how old my motherboard is...
Note: I have already experienced some issues with Build 9926. Within a half hour of using it, I suffered an instant power-down. There was not even enough time for a bluescreen. When it came back, my Intel GPU (which worked for a few minutes after the update) refused to be activated, along with the monitor it is attached to. My point? Not for production machines.
Update: Looks like a stick of RAM (or some other hardware) blew, coincidentally, about 30 minutes after the update finished, while the computer was running, which also confused my UEFI settings. I haven't got around to troubleshooting much, but it seems like a weirdly-timed, abrupt hardware failure (BIOS is only reporting half of the RAM installed, iGPU is "enabled" but without RAM associated to it, etc.).
The interesting part, to me, is how Microsoft pushed DX12 into this release without, you know, telling anyone. It is not on any changelog that I can see, and it was not mentioned anywhere in the briefing as potentially being in an upcoming preview build. Before the keynote, I had a theory that it would be included but, after the announcement, figured that it might be pushed until GDC or BUILD (but I kept an open mind). The only evidence that it might come this month was an editorial on Forbes that referenced a conversation with Futuremark, who allegedly wanted to release an update to 3DMark (they hoped) when Microsoft released the new build. I could not find anything else, so I didn't report on it -- you would think that there would be a second source for that somewhere. It turns out that he might be right.
The new Windows 10 Technical Preview, containing DirectX 12, is available now from the preview build panel. It looks like Futuremark (and maybe others) will soon release software for it, but no hardware vendor has released a driver... yet.
Introducing Windows 10 (Again)
I did not exactly make too many unsafe predictions, but let's recap the Windows 10 Consumer announcement anyway. The briefing was a bit on the slow side, at least if you are used to E3 keynotes, but it contained a fair amount of useful information. Some of the things discussed are future-oriented, but some will arrive soon. So let's get right into it.
Price and Upgrade Options
Microsoft has not announced an official price for Windows 10, if the intent is to install it on a new PC. If you are attempting to upgrade a machine that currently runs Windows 7 or Windows 8.1, then that will be a free upgrade if done within the first year. Windows Phone 8.1 users are also eligible for a no-cost upgrade to Windows 10 if done in the first year.
Quote Terry Myerson of Microsoft, “Once a device is upgraded to Windows 10, we will be keeping it current for the supported lifetime of the device.” This is not elaborated on, but it seems like a weird statement given what we have traditionally expected from Windows. One possible explanation is that Microsoft intends for Windows to be a subscription service going forward, which would be the most obvious extension of “Windows as a Service”. On the other hand, they could be going for the per-device revenue option with Bing, Windows Store, and other initiatives being long tail. If so, I am a bit confused about what constitutes a new device for systems that are regularly upgraded, like what our readers are typically interested in. All of that will eventually be made clear, but not yet.
A New Build for Windows 10
Late in the keynote, Microsoft announced the availability of new preview builds for Windows 10. This time, users of Windows Phone 8.1 will also be able to see the work in progress. PC “Insiders” will get access to their build “in the next week” and phones will get access “in Feburary”. Ars Technica seems to believe that this is scheduled for Sunday, February 1st, which is a really weird time to release a build but their source might be right.
We don't know exactly what will be in it, though. In my predictions, I guessed that a DirectX 12 SDK might be available (or at least some demos) in the next build. That has not been mentioned, which probably would have been if it were true. I expect the next possibility (if we're not surprised in the next one-to-ten days when the build drops) is Game Developers Conference (GDC 2015), which starts on March 2nd.
The New Web Browser: Project Spartan
My guess was that Spartan would be based on DirectX 12. Joe Belfiore said that it is using a new, standards-compliant rendering engine and basically nothing more. The event focused on specific features. The first is note taking, which basically turns the web browser into a telestrator that can also accept keyboard comment blocks. The second is a reading mode that alters content into a Microsoft Word-like column. The third is “reading lists”, which is basically a “read it later” feature that does offline caching. The fourth is Adobe PDF support, which works with the other features of Spartan such as note taking and reading lists.
Which Transitions Into Cortana
The fifth feature of Spartan is Cortana integration, which will provide auto-suggestions based on the information that the assistant software has. The example they provided was auto-suggesting the website for his wife's flight. Surprisingly, when you attempt to control a Spartan, Cortana does not say “There's two of us in here now, remember?” You know, in an attempt to let you know she's service that's integrated into the browser.
Otherwise, it's an interesting demo. I might even end up using it when it comes out, but these sorts of things do not really interest me too much. We have been at the point where, for my usage, the operating system is really not in the way anymore. It feels like there is very little friction between me and getting what I want done, done. Of course, people felt that way about rotary phones until touch-tone came out, and I keep an open mind to better methods. It's just hard to get me excited about voice-activated digital assistants.
As I stated before, DirectX 12 was mentioned but a release date was not confirmed. What they did mention was a bit of relative performance. DirectX 12 supposedly uses about half of the power consumption of DirectX 11, which is particularly great for mobile applications. It can also handle scenes with many more objects. A FutureMark demo was displayed, with the DirectX 11 version alongside a DirectX 12 version. The models seem fairly simple, but the DirectX 12 version appears to running at over 100 FPS when the DirectX 11 version outright fails.
Other gaming features were mentioned. First, Windows 10 will allow shadow recording the last 30 seconds of footage from any game. You might think that NVIDIA would be upset about that, and they might be, but that is significantly less time than ShadowPlay or other recording methods. Second, Xbox One will be able to stream gameplay to any PC in your house. I expect this is the opposite direction than what people hope for, rather wishing for high-quality PC footage to be easily streamed to TVs with a simple interface. It will probably serve a purpose for some use case, though.
Well that was a pretty long event, clocking in at almost two-and-a-half hours. The end had a surprise announcement of an augmented reality (not virtual reality) headset, called the “HoloLens”, which is developed by the Kinect team. I am deliberately not elaborating on it because I was not at the event and I have not tried it. I will say that the most interesting part about it, for me, is the Skype integration, because that probably hints at Microsoft's intentions with the product.
For the rest of us, it touched on a number of interesting features but, like the Enterprise event, did not really dive in. It would have been nice to get some technical details about DirectX 12, but that obviously does not cater to the intended audience. Unless an upcoming build soft-launches a DirectX 12 preview (or Spartan) so that we can do our own discovery, we will probably need to wait until GDC and/or BUILD to find out more.
Until then, you could watch the on-demand version at Microsoft's website.
Subject: General Tech | January 20, 2015 - 09:45 PM | Scott Michaud
Tagged: windows 10, windows, spartan, microsoft, dx12, DirectX 12, DirectX, cortana
Microsoft will hold a briefing tomorrow (Wednesday, January 21st at 12pm EST/5pm UTC) about “The Next Chapter” of Windows 10. This has been described as the Consumer keynote, mirroring the original one that was supposedly intended for the enterprise. Otherwise, there are few official comments regarding the event, but there are also things that we can speculate on.
Here is what I expect to see:
A New Build for Windows 10
Maybe it will not be released on the same day as the speech, but it cannot really be too far behind. We are about two-thirds through January and December was skipped, so it must be happening soon. When 9879 was released, Microsoft said that it would be the last build of 2014 and that “We'll have something new to share with you early in 2015”. Whatever that is (or those things are) will probably be discussed at the event, which means that the build is probably not too far behind it.
When the graphics API was announced, they specifically said the following (see our recap for the second slide that was posted at 10:48am PST):
- Targeting Holiday 2015 games
- Preview release coming later this year
- Don't want to wait that long? Early access!
The preview release later in 2014 did not happen, but the early access did. As such, I am guessing that the date slipped to either the next Windows 10 build, or maybe a build or two after. Whenever it happens specifically, I am guessing that it will be mentioned at this event and available for developers soon (and not just a hand-picked group of Early Access members). Sure, it could wait until Build 2015 in April, but the original slide sounds like they were targeting the end of 2014.
Also, the DirectX 12 Twitter Account just retweeted the live stream and Phil Spencer will be there.
'Spartan' Browser (Maybe with DirectX 12 Support?)
Speaking of DirectX 12, its goal is to utilize GPU shader cores as efficiently as possible, reducing the time it holds up the CPU and balancing its load across multiple cores. This leads to power efficiency and the ability to load many more tasks on the GPU.
These are all things that a web browser vendor would love! Web standards are inherently difficult to multi-thread, because they are designed as sets of stages which build upon other stages. DirectX 12 could probably help immensely, at least with the drawing stage. Web content tends to be fairly simple, but there can be a lot of it, especially for complex Canvas animations (and especially for mobile devices).
It was also recently rumored that Trident, the rendering engine behind Internet Explorer and the not-quite-confirmed Spartan browser, was forked into two maintained versions. The expectation is that this was for compatibility reasons, where the new version can be developed to W3C (and other) standards without worrying about legacy, Internet Explorer-based compatibility cruft. If porting a DirectX 11 applications to DirectX 12 will be annoying, I can see why Microsoft chose to draw the compatibility line just behind that initiative. And honestly, how many people care about rendering, power, and multi-core performance increases for IE8-designed, and therefore desktop-based, web applications?
Continuum, Cortana, and Other Changes
Again, this is what Microsoft considers a Consumer event. As such, it would make sense for them to describe an ideal consumer device, which probably includes two-in-ones. Cortana should also be discussed as well, which is intended to bring value to the users and probably lead them to Bing services. Leaks have also suggested that they are preparing a dark theme.
Am I right? We'll see tomorrow.
Subject: Graphics Cards | October 3, 2014 - 03:18 AM | Scott Michaud
Tagged: microsoft, DirectX, DirectX 12, windows 10, threshold, windows
A Microsoft blog posting confirms: "The final version of Windows 10 will ship with DirectX 12". To me, this seems like a fairly obvious statement. The loose dates provided for both the OS and the availability of retail games suggest that the two would be launching at roughly the same time. The article also claims that DirectX 12 "Early Access" members will be able to develop with the Windows 10 Technical Preview. Apart from Unreal Engine 4 (for Epic Games subscribers), Intel will also provide source access to their Asteroids demo, shown at Siggraph 2014, to all accepted early access developers.
Our readers might find this information slightly disappointing as it could be interpreted that DirectX 12 would not be coming to Windows 7 (or even 8.x). While it does not look as hopeful as before, they never, at any point, explicitly say that it will not come to older operating systems. It still might.