An Data Format for Whole 3D Scenes
The Khronos Group has finalized the glTF 2.0 specification, and they recommend that interested parties integrate this 3D scene format into their content pipeline starting now. It’s ready.
glTF is a format to deliver 3D content, especially full scenes, in a compact and quick-loading data structure. These features differentiate glTF from other 3D formats, like Autodesk’s FBX and even the Khronos Group’s Collada, which are more like intermediate formats between tools, such as 3D editing software (ex: Maya and Blender) and game engines. They don’t see a competing format for final scenes that are designed to be ingested directly, quick and small.
glTF 2.0 makes several important changes.
The previous version of glTF was based on a defined GLSL material, which limited how it could be used, although it did align with WebGL at the time (and that spurred some early adoption). The new version switches to Physically Based Rendering (PBR) workflows to define their materials, which has a few advantages.
First, PBR can represent a wide range of materials with just a handful of parameters. Rather than dictating a specific shader, the data structure can just... structure the data. The industry has settled on two main workflows, metallic-roughness and specular-gloss, and glTF 2.0 supports them both. (Metallic-roughness is the core workflow, but specular-gloss is provided as an extension, and they can be used together in the same scene. Also, during the briefing, I noticed that transparency was not explicitly mentioned in the slide deck, but the Khronos Group confirmed that it is stored as the alpha channel of the base color, and thus supported.) Because the format is now based on existing workflows, the implementation can be programmed in OpenGL, Vulkan, DirectX, Metal, or even something like a software renderer. In fact, Microsoft was a specification editor on glTF 2.0, and they have publicly announced using the format in their upcoming products.
The original GLSL material, from glTF 1.0, is available as an extension (for backward compatibility).
A second advantage of PBR is that it is lighting-independent. When you define a PBR material for an object, it can be placed in any environment and it will behave as expected. Noticeable, albeit extreme examples of where this would have been useful are the outdoor scenes of Doom 3, and the indoor scenes of Battlefield 2. It also simplifies asset creation. Some applications, like Substance Painter and Quixel, have artists stencil materials onto their geometry, like gold, rusted iron, and scuffed plastic, and automatically generate the appropriate textures. It also aligns well with deferred rendering, see below, which performs lighting as a post-process step and thus skip pixels (fragments) that are overwritten.
PBR Deferred Buffers in Unreal Engine 4 Sun Temple.
Lighting is applied to these completed buffers, not every fragment.
glTF 2.0 also improves support for complex animations by adding morph targets. Most 3D animations, beyond just moving, rotating, and scaling whole objects, are based on skeletal animations. This method works by binding vertexes to bones, and moving, rotating, and scaling a hierarchy of joints. This works well for humans, animals, hinges, and other collections of joints and sockets, and it was already supported in glTF 1.0. Morph targets, on the other hand, allow the artist to directly control individual vertices between defined states. This is often demonstrated with a facial animation, interpolating between smiles and frowns, but, in an actual game, this is often approximated with skeletal animations (for performance reasons). Regardless, glTF 2.0 now supports morph targets, too, letting the artists make the choice that best suits their content.
Speaking of performance, the Khronos Group is also promoting “enhanced performance” as a benefit of glTF 2.0. I asked whether they have anything to elaborate on, and they responded with a little story. While glTF 1.0 validators were being created, one of the engineers compiled a list of design choices that would lead to minor performance issues. The fixes for these were originally supposed to be embodied in a glTF 1.1 specification, but PBR workflows and Microsoft’s request to abstract the format away from GLSL lead to glTF 2.0, which is where the performance optimization finally ended up. Basically, there wasn’t just one or two changes that made a big impact; it was the result of many tiny changes that add up.
Also, the binary version of glTF is now a core feature in glTF 2.0.
The slide looks at the potential future of glTF, after 2.0.
Looking forward, the Khronos Group has a few items on their glTF roadmap. These did not make glTF 2.0, but they are current topics for future versions. One potential addition is mesh compression, via the Google Draco team, to further decrease file size of 3D geometry. Another roadmap entry is progressive geometry streaming, via Fraunhofer SRC, which should speed up runtime performance.
Yet another roadmap entry is “Unified Compression Texture Format for Transmission”, specifically Basis by Binomial, for texture compression that remains as small as possible on the GPU. Graphics processors can only natively operate on a handful of formats, like DXT and ASTC, so textures need to be converted when they are loaded by an engine. Often, when a texture is loaded at runtime (rather than imported by the editor) it will be decompressed and left in that state on the GPU. Some engines, like Unity, have a runtime compress method that converts textures to DXT, but the developer needs to explicitly call it and the documentation says it’s lower quality than the algorithm used by the editor (although I haven’t tested this). Suffices to say, having a format that can circumvent all of that would be nice.
Again, if you’re interested in adding glTF 2.0 to your content pipeline, then get started. It’s ready. Microsoft is doing it, too.
Subject: Graphics Cards | May 10, 2017 - 03:53 AM | Scott Michaud
Tagged: vulkan, ShadowPlay, opengl, nvidia, geforce experience
The latest version of GeForce Experience, 3.6, adds video capture (including screenshots and live streaming) support for OpenGL and Vulkan games. The catalog of titles support by ShadowPlay, which I’m pretty sure NVIDIA wants to call Share now, despite referring to it by its old name in the blog post, now includes No Man’s Sky, DOOM, and Microsoft’s beloved OpenGL title: Minecraft.
The rest of the update focuses on tweaking a few interface elements, including its streaming panel, its video and screenshot upload panel, and its gallery. Access to the alternative graphics APIs was the clear headline-maker, however, opening the door to several large gaming groups, and potentially even more going forward.
Subject: Graphics Cards | April 3, 2017 - 03:15 PM | Jeremy Hellstrom
Tagged: mad max, linux, kepler, maxwell, pascal, NVIDA, vulkan, opengl
With Vulkan support being added to Mad Max, at least in beta form, Phoronix decided to take advantage of the release to test the performance of a wide variety of NVIDIA cards on the API. They grabbed over a dozen cards encompassing three different architectures, from the GTX 680 through to the GTX 1080 Ti, so you get a very good look at the change in performance of NVIDIA on Vulkan. The results are clear, in every case Vulkan was superior to OpenGL and in many cases framerate more than doubled. Drop by for a look at what some predicted was a DOA API.
"Yesterday game porter firm Feral Interactive released a public beta of Mad Max that features a Vulkan renderer in place of its OpenGL API for graphics rendering on Linux. In addition to Radeon Vulkan numbers, I posted some NVIDIA Mad Max Linux benchmarks with both renderers. Those results were exciting on the few Pascal cards tested so I have now extended that comparison to feature a line-up of 14 NVIDIA GeForce graphics cards from Kepler, Maxwell, and Pascal families while looking at this game's OpenGL vs. Vulkan performance."
Here are some more Graphics Card articles from around the web:
Subject: Graphics Cards | November 5, 2016 - 08:19 PM | Scott Michaud
Tagged: linux, DOTA 2, valve, nvidia, vulkan, opengl
Phoronix published interesting benchmark results for OpenGL vs Vulkan on Linux, across a wide spread of thirteen NVIDIA GPUs. Before we begin, the CPU they chose was an 80W Intel Xeon E3-1280 v5, which fits somewhere between the Skylake-based Core i7-6700k and Core i7-6700 (no suffix). You may think that Xeon v5 would be based on Broadwell, but, for some reason, Intel chose the E3-1200 series to be based on Skylake. Regardless, the choice of CPU will come in to play.
They will apparently follow up this article with AMD results.
A trend arose throughout the whole article. At 1080p, everything, from the GTX 760 to the GTX 1080, was rendering at ~101 FPS on OpenGL and ~115 FPS on Vulkan. The obvious explanation is that the game is 100% CPU-bound on both APIs, but Vulkan is able to relax the main CPU thread enough to squeeze out about 14% more frames.
The thing is, the Xeon E3-1280 v5 is about as high-end of a mainstream CPU as you can get. It runs the most modern architecture and it can achieve clocks up to 4 GHz on all cores. DOTA 2 can get harsh on the CPU when a lot of units are on screen, but this is a little surprisingly low. Then again, I don't have any experience running DOTA 2 benchmarks, so maybe it's a known thing, or maybe even a Linux-version thing?
Moving on, running the game at 4K, the results get more interesting. In GPU-bound scenarios, NVIDIA's driver shows a fairly high performance gain on OpenGL. Basically all GPUs up to the GTX 1060 run at a higher frame rate in OpenGL, only switching to Vulkan with the GTX 1070 and GTX 1080, where OpenGL hits that 101 FPS ceiling and Vulkan goes a little above.
Again, it will be interesting to see how AMD fairs against this line of products, both in Vulkan and OpenGL. Those will apparently come “soon”.
Why Two 4GB GPUs Isn't Necessarily 8GB
We're trying something new here at PC Perspective. Some topics are fairly difficult to explain cleanly without accompanying images. We also like to go fairly deep into specific topics, so we're hoping that we can provide educational cartoons that explain these issues.
This pilot episode is about load-balancing and memory management in multi-GPU configurations. There seems to be a lot of confusion around what was (and was not) possible with DirectX 11 and OpenGL, and even more confusion about what DirectX 12, Mantle, and Vulkan allow developers to do. It highlights three different load-balancing algorithms, and even briefly mentions what LucidLogix was attempting to accomplish almost ten years ago.
If you like it, and want to see more, please share and support us on Patreon. We're putting this out not knowing if it's popular enough to be sustainable. The best way to see more of this is to share!
Subject: General Tech | February 18, 2016 - 02:16 PM | Ken Addison
Tagged: x16 LTE, vulkan, video, ssd, Samsung, qualcomm, podcast, pb328q, opengl, nvidia, micron, Khronos, gtx 950, asus, apple, 840 evo, 750ti, 750 evo, 3d nand
PC Perspective Podcast #387 - 02/18/2016
Join us this week as we discuss the ASUS PB328Q, Samsung 750 EVO SSD, the release of Vulkan and more!
The URL for the podcast is: http://pcper.com/podcast - Share with your friends!
- iTunes - Subscribe to the podcast directly through the Store (audio only)
- RSS - Subscribe through your regular RSS reader (audio only)
- MP3 - Direct download link to the MP3 file
Hosts: Ryan Shrout, Jeremy Hellstrom, Josh Walrath, and Allyn Malventano
Program length: 1:34:18
Week in Review:
0:35:00 This episode of the PC Perspective Podcast is brought to you by Audible, the world's leading provider of audiobooks with more than 180,000 downloadable titles across all types of literature including fiction, nonfiction, and periodicals. For your free audiobook, go to audible.com/pcper
News items of interest:
1:07:00 This episode of PC Perspective Podcast is brought to you by Braintree. Even the best mobile app won’t work without the right payments API. That’s where the Braintree v.0 SDK comes in. One amazingly simple integration gives you every way to pay. Try out the sandbox and see for yourself at braintreepayments.com/pcper
Hardware/Software Picks of the Week
Subject: Graphics Cards | December 29, 2015 - 07:05 AM | Scott Michaud
Tagged: opengl, mesa, linux, Intel
The open-source driver for Intel is known to be a little behind on Linux. Because Intel does not provide as much support as they should, the driver still does not support OpenGL 4.0, although that is changing. One large chunk of that API is support for tessellation, which comes from DirectX 11, and recent patches are adding it for supported hardware. Proprietary drivers exist, at least for some platforms, but they have their own issues.
According to the Phoronix article, once the driver succeeds in supporting OpenGL 4.0, it will not be too long to open the path to 4.2. Tessellation is a huge hurdle, partially because it involves adding two whole shading stages to the rendering pipeline. Broadwell GPUs were recently added, but a patch that was committed yesterday will expand that to Ivy Bridge and Haswell. On Windows, Intel is far ahead -- pushing OpenGL 4.4 for Skylake-based graphics, although that platform only has proprietary drivers. AMD and NVIDIA are up to OpenGL 4.5, which is the latest version.
While all of this is happening, Valve is working on an open-source Vulkan driver for Intel on Linux. This API will be released adjacent to OpenGL, and is built for high-performance graphics and compute. (Note that OpenCL is more sophisticated than Vulkan "1.0" will be on the compute side of things.) As nice as it would be to get high-end OpenGL support, especially for developers who want a more simplified structure to communicate to GPUs with, Vulkan will probably be the API that matters most for high-end video games. But again, that only applies to games that are developed for it.
Subject: Graphics Cards | December 21, 2015 - 01:04 PM | Jeremy Hellstrom
Tagged: GameWorks VR 1.1, nvidia, Oculus, opengl, vive
If you are blessed with the good fortune of already having a VR headset and happen to be running an NVIDIA GPU then there is a new driver you want to grab as soon as you can. The driver includes a new OpenGL extension that enables NVIDIA SLI support for OpenGL apps that display on an Oculus or Vive. NVIDIA's PR suggests you can expect your performance to improve 1.7 times, not quite doubling but certainly offering a noticeable performance improvement. The update is for both GeForce and Quadro cards.
They describe how the update will generate images in their blog post here. They imply that the changes to your code in order to benefit from this update will be minimal and it will also reduce the CPU overhead required to display the images for the right and left eye. Read on if you are interested in the developer side of this update, otherwise download your new driver and keep an eye out for application updates that enable support for SLI in VR.
Subject: Graphics Cards | October 7, 2015 - 07:01 AM | Scott Michaud
Tagged: opengl, metal, apple
Ars Technica took it upon themselves to benchmark Metal in the latest OSX El Capitan release. Even though OpenGL on Mac OSX is not considered to be on par with its Linux counterparts, which is probably due to the driver situation until recently, it pulls ahead of Metal in many situations.
Image Credit: Ars Technica
Unlike the other graphics APIs, Metal uses the traditional binding model. Basically, you have a GPU object that you attach your data to, then call one of a handful of “draw” functions to signal the driver. DirectX 12, Vulkan, and Mantle, on the other hand, treat work like commands on queues. The latter model works better in multi-core environments, and it aligns with GPU compute APIs, but the former is easier to port OpenGL and DirectX 11 applications to.
Ars Technica notes that faster GPUs, such as the NVIDIA GeForce GTX 680MX, show higher gains than slower ones. Their “best explanation” is that “faster GPUs can offload more work from the CPU”. That is pretty much true, yes. The new APIs are designed to keep GPUs loaded and working as much as possible, because they really do sit around doing nothing a lot. If you are able to keep a GPU loaded, because it can't accept much load in the first place, then there is little benefit to decreasing CPU load or spreading out across multiple cores.
Granted, there are many ways that benchmarks like these could be incorrectly used. I'll assume that Ars Technica and GFXBench are not making any simple mistakes, though, but it's good to be critical just in case.
Subject: Graphics Cards, Processors, Mobile, Shows and Expos | August 10, 2015 - 09:01 AM | Scott Michaud
Tagged: vulkan, spir, siggraph 2015, Siggraph, opengl sc, OpenGL ES, opengl, opencl, Khronos
When the Khronos Group announced Vulkan at GDC, they mentioned that the API is coming this year, and that this date is intended to under promise and over deliver. Recently, fans were hoping that it would be published at SIGGRAPH, which officially begun yesterday. Unfortunately, Vulkan has not released. It does hold a significant chunk of the news, however. Also, it's not like DirectX 12 is holding a commanding lead at the moment. The headers were public only for a few months, and the code samples are less than two weeks old.
The organization made announcements for six products today: OpenGL, OpenGL ES, OpenGL SC, OpenCL, SPIR, and, as mentioned, Vulkan. They wanted to make their commitment clear, to all of their standards. Vulkan is urgent, but some developers will still want the framework of OpenGL. Bind what you need to the context, then issue a draw and, if you do it wrong, the driver will often clean up the mess for you anyway. The briefing was structure to be evident that it is still in their mind, which is likely why they made sure three OpenGL logos greeted me in their slide deck as early as possible. They are also taking and closely examining feedback about who wants to use Vulkan or OpenGL, and why.
As for Vulkan, confirmed platforms have been announced. Vendors have committed to drivers on Windows 7, 8, 10, Linux, including Steam OS, and Tizen (OSX and iOS are absent, though). Beyond all of that, Google will accept Vulkan on Android. This is a big deal, as Google, despite its open nature, has been avoiding several Khronos Group standards. For instance, Nexus phones and tablets do not have OpenCL drivers, although Google isn't stopping third parties from rolling it into their devices, like Samsung and NVIDIA. Direct support of Vulkan should help cross-platform development as well as, and more importantly, target the multi-core, relatively slow threaded processors of those devices. This could even be of significant use for web browsers, especially in sites with a lot of simple 2D effects. Google is also contributing support from their drawElements Quality Program (dEQP), which is a conformance test suite that they bought back in 2014. They are going to expand it to Vulkan, so that developers will have more consistency between devices -- a big win for Android.
While we're not done with Vulkan, one of the biggest announcements is OpenGL ES 3.2 and it fits here nicely. At around the time that OpenGL ES 3.1 brought Compute Shaders to the embedded platform, Google launched the Android Extension Pack (AEP). This absorbed OpenGL ES 3.1 and added Tessellation, Geometry Shaders, and ASTC texture compression to it. It was also more tension between Google and cross-platform developers, feeling like Google was trying to pull its developers away from Khronos Group. Today, OpenGL ES 3.2 was announced and includes each of the AEP features, plus a few more (like “enhanced” blending). Better yet, Google will support it directly.
Next up are the desktop standards, before we finish with a resurrected embedded standard.
OpenGL has a few new extensions added. One interesting one is the ability to assign locations to multi-samples within a pixel. There is a whole list of sub-pixel layouts, such as rotated grid and Poisson disc. Apparently this extension allows developers to choose it, as certain algorithms work better or worse for certain geometries and structures. There were probably vendor-specific extensions for a while, but now it's a ratified one. Another extension allows “streamlined sparse textures”, which helps manage data where the number of unpopulated entries outweighs the number of populated ones.
OpenCL 2.0 was given a refresh, too. It contains a few bug fixes and clarifications that will help it be adopted. C++ headers were also released, although I cannot comment much on it. I do not know the state that OpenCL 2.0 was in before now.
And this is when we make our way back to Vulkan.
SPIR-V, the code that runs on the GPU (or other offloading device, including the other cores of a CPU) in OpenCL and Vulkan is seeing a lot of community support. Projects are under way to allow developers to write GPU code in several interesting languages: Python, .NET (C#), Rust, Haskell, and many more. The slide lists nine that Khronos Group knows about, but those four are pretty interesting. Again, this is saying that you can write code in the aforementioned languages and have it run directly on a GPU. Curiously missing is HLSL, and the President of Khronos Group agreed that it would be a useful language. The ability to cross-compile HLSL into SPIR-V means that shader code written for DirectX 9, 10, 11, and 12 could be compiled for Vulkan. He expects that it won't take long for a project to start, and might already be happening somewhere outside his Google abilities. Regardless, those who are afraid to program in the C-like GLSL and HLSL shading languages might find C# and Python to be a bit more their speed, and they seem to be happening through SPIR-V.
As mentioned, we'll end on something completely different.
For several years, the OpenGL SC has been on hiatus. This group defines standards for graphics (and soon GPU compute) in “safety critical” applications. For the longest time, this meant aircraft. The dozens of planes (which I assume meant dozens of models of planes) that adopted this technology were fine with a fixed-function pipeline. It has been about ten years since OpenGL SC 1.0 launched, which was based on OpenGL ES 1.0. SC 2.0 is planned to launch in 2016, which will be based on the much more modern OpenGL ES 2 and ES 3 APIs that allow pixel and vertex shaders. The Khronos Group is asking for participation to direct SC 2.0, as well as a future graphics and compute API that is potentially based on Vulkan.
The devices that this platform intends to target are: aircraft (again), automobiles, drones, and robots. There are a lot of ways that GPUs can help these devices, but they need a good API to certify against. It needs to withstand more than an Ouya, because crashes could be much more literal.