GDC 2018: Qualcomm Talks Future of VR and AR with Upcoming Dev Kit

Subject: General Tech | March 21, 2018 - 09:20 AM |
Tagged: xr, VR, Tobii, qualcomm, HMD, GDC 2018, GDC, eye-tracking, developers, dev kit, AR

We have recently covered news of Qualcomm's ongoing VR/AR efforts (the two terms now combine as "XR", for eXtended reality), with news of the Snapdragon 845-powered reference HMD and more recently the collaboration with Tobii to bring eye-tracking to the Qualcomm development platform. Today at GDC Qualcomm is mapping out their vision for the future of XR, and providing additional details about the Snapdragon 845 dev kit - and announcing support for the HTC Vive Wave SDK.


From Qualcomm:

For the first time, many new technologies that are crucial for an optimal and immersive VR user experience will be supported in the Snapdragon 845 Virtual Reality Development Kit. These include:


  • Room-scale 6DoF SLAM: The Snapdragon 845 Virtual Reality Development Kit is engineered to help VR developers create applications that allow users to explore virtual worlds, moving freely around in a room, rather than being constrained to a single viewing position. Un-tethered mobile VR experiences like these can benefit from the Snapdragon 845 Virtual Reality Development Kit’s pre-optimized hardware and software for room-scale six degrees of freedom (6DoF) with “inside-out” simultaneous localization and mapping (SLAM). All of this is designed to be accomplished without any external setup in the room by the users, and without any cables or wires.
  • Qualcomm® Adreno™ Foveation: Our eyes are only able to observe significant details in a very small center of our field of vision - this region is called the “fovea”. Foveated rendering utilizes this understanding to boost performance & save power, while also improving visual quality. This is accomplished through multiple technology advancements for multi-view, tile-based foveation with eye-tracking and fine grain preemption to help VR application developers deliver truly immersive visuals with optimal power efficiency.


  • Eye Tracking: Users naturally convey intentions about how and where they want to interact within virtual worlds through their eyes. Qualcomm Technologies worked with Tobii AB to develop an integrated and optimized eye tracking solution for the Snapdragon 845 VR Development Kit. The cutting-edge eye tracking solution on Snapdragon 845 VR Development Kit is designed to help developers utilize Tobii’s EyeCore™ eye tracking algorithms to create content that utilizes gaze direction for fast interactions, and superior intuitive interfaces.
  • Boundary System: The new SDK for the Snapdragon 845 VR Development Kit supports a boundary system that is engineered to help VR application developers accurately visualize real-world spatial constraints within virtual worlds, so that their applications can effectively manage notifications and play sequences for VR games or videos, as the user approaches the boundaries of the real-world play space.


In addition to enhancing commercial reach for the VR developer community, Qualcomm Technologies is excited to announce support for the HTC Vive Wave™ VR SDK on the Snapdragon 845 Virtual Reality Development Kit, anticipated to be available later this year. The Vive Wave™ VR SDK is a comprehensive tool set of APIs that is designed to help developers create high-performance, Snapdragon-optimized content across diverse hardware vendors at scale, and offer a path to monetizing applications on future HTC Vive ready products via the multi-OEM Viveport™ application store.

The Snapdragon 845 HMD/dev kit and SDK are expected to be available in Q2 2018.

Source: Qualcomm

Tobii and Qualcomm Announce Collaboration on Mobile VR Headsets with Eye-Tracking

Subject: General Tech | March 16, 2018 - 09:45 AM |
Tagged: xr, VR, Tobii, snapdragon 845, qualcomm, mobile, HMD, head mounted display, eye tracking, AR, Adreno 630

Tobii and Qualcomm's collaboration in the VR HMD (head-mounted display) space is a convergence of two recent stories, with Tobii's impressing showing of a prototype HMD device at CES featuring their eye-tracking technology, and Qualcomm's unvieling last month of their updated mobile VR platform, featuring the new Snapdragon 845.


The Qualcomm Snapdragon 845 Mobile VR Reference Platform

What does this new collaboration mean for the VR industry? For now it means a new reference design and dev kit with the latest tech from Tobii and Qualcomm:

"As a result of their collaboration, Tobii and Qualcomm are creating a full reference design and development kit for the Qualcomm Snapdragon 845 Mobile VR Platform, which includes Tobii's EyeCore eye tracking algorithms and hardware design. Tobii will license its eye tracking technologies and system and collaborate with HMD manufacturers on the optical solution for the reference design."

The press release announcing this collaboration recaps the benefits of Tobii eye tracking in a mobile VR/AR device, which include:

  • Foveated Rendering: VR/AR devices become aware of where you are looking and can direct high-definition graphics processing power to that exact spot in real time. This enables higher definition displays, more efficient devices, longer battery life and increased mobility.
  • Interpupillary Distance: Devices automatically orient images to align with your pupils. This enables devices to adapt to the individual user, helping to increase the visual quality of virtual and augmented reality experiences.
  • Hand-Eye Coordination: By using your eyes in harmony with your hands and associated controllers, truly natural interaction and immersion, not possible without the use of gaze, is realized.
  • Interactive Eye Contact: Devices can accurately track your gaze in real time, enabling content creators to express one of the most fundamental dimensions of human interaction – eye contact. VR technologies hold the promise of enabling a new and immersive medium for social interaction. The addition of true eye contact to virtual reality helps deliver that promise.


Tobii's prototype eye-tracking HMD

For its part, Qualcomm's Snapdragon 845-powered VR mobile platform promises greater portability of a better VR experience, with expanded freedom on top of the improved graphics horsepower from the new Adreno 630 GPU in the Snapdragon 845. This portability includes 6DoF (6 degrees of freedom) using external cameras to identify location within a room, eliminating the need for external room sensors.

"Together, 6DoF and SLAM deliver Roomscale - the ability to track the body and location within a room so you can freely walk around your XR environment without cables or separate room sensors – the first on a mobile standalone device. Much of this is processed on the new dedicated Qualcomm Hexagon Digital Signal Processor (DSP) and Adreno Graphics Processing Unit within the Snapdragon 845. Qualcomm Technologies’ reference designs have supported some of the first wave of standalone VR devices from VR ecosystem leaders like Google Daydream, Oculus and Vive."

It is up to developers, and consumer interest in VR moving forward, to see what this collaboration will produce. To editorialize briefly, from first-hand experience I can vouch for the positive impact of eye-tracking with an HMD, and if future products live up to the promise of a portable, high-performance VR experience (with a more natural feel from less rapid head movement) a new generation of VR enthusiasts could be born.

Source: PR Newswire

Qualcomm Announces Snapdragon 845 Mobile VR Reference Platform

Subject: General Tech | February 21, 2018 - 07:50 PM |
Tagged: VR, snapdragon 845, reference platform, qualcomm, mobile, headset, development

Qualcomm has another mobile-related announcement ahead of MWC, introducing a new VR reference platform based on the Snapdragon 845 in collaboration with HTC. As the new Snapdragon 845 boasts much more powerful graphics from its Adreno 630 GPU compared to the Snapdragon 835 - which was behind the previous mobile VR platform - this represents an important step forward in the mobile VR space.


The increased graphics horsepower isn't the only aspect of the Adreno 630 that should translate into a better mobile VR/AR (now rolled together into the term "XR" for extended reality) experience, as the gains in graphics performance we saw from the SDM845 reference platform are said to come with 30% power savings as well as Adreno Foveation, which allows eye-tracking to direct resources only to the area where the user is looking. Thus Foveation allows for, literally, focused GPU resource allocation, which should translate into better performance with less hardware overhead.

Right view.jpg

The Snapdragon 845 Mobile VR Platform also boasts 6DoF, or "6 degrees of freedom", which incorporates external cameras to improve free movement compared to the previous 3DoF limitations:

"Together, 6DoF and SLAM deliver Roomscale - the ability to track the body and location within a room so you can freely walk around your XR environment without cables or separate room sensors – the first on a mobile standalone device. Much of this is processed on the new dedicated Qualcomm Hexagon Digital Signal Processor (DSP) and Adreno Graphics Processing Unit within the Snapdragon 845. Qualcomm Technologies’ reference designs have supported some of the first wave of standalone VR devices from VR ecosystem leaders like Google Daydream, Oculus and Vive."

Left view.jpg

Qualcomm's goal with the new Snapdragon 845 Mobile VR Platform is to support "the next wave of smartphone and standalone VR headsets", and it seems that mobile hardware is starting to catch up to the ambitions of what is now being called XR.

Source: Qualcomm

Valve Supporting AMD's GPU-Powered TrueAudio Next In Latest Steam Audio Beta

Subject: General Tech, Graphics Cards | February 7, 2018 - 09:02 PM |
Tagged: VR, trueaudio next, TrueAudio, steam audio, amd

Valve has announced support for AMD's TrueAudio Next technology in its Steam Audio SDK for developers. The partnership will allow game and VR application developers to reserve a portion of a GCN-based GPU's compute units for audio processing and increase the quality and quantity of audio sources as a result. AMD's OpenCL-based TrueAudio Next technology can run CPUs as well but it's strength is in the ability to run on a dedicated portion of the GPU to improve both frame times and audio quality since threads are not competing for the same GPU resources during complex scenes and the GPU can process complex audio scenes and convolutions much more efficiently than a CPU (especially as the number of sources and impulse responses increase) respectively.

AMD True Audio Next In Steam Audio.jpg

Steam Audio's TrueAudio Next integration is being positioned as an option for developers and the answer to increasing the level of immersion in virtual reality games and applications. While TrueAudio Next is not using ray tracing for audio, it is physics-based and can be used to great effect to create realistic scenes with large numbers of direct and indirect audio sources, ambisonics, increased impulse response lengths, echoes, reflections, reverb, frequency equalization, and HRTF (Head Related Transfer Function) 3D audio. According to Valve indirect audio from multiple sources with convolution reverb is one of the most computationally intensive parts of Steam Audio, and TAN is able to handle it much more efficiently and accurately without affecting GPU frame times and freeing the CPU up for additional physics and AI tasks which it is much better at anyway. Convolution is a way of modeling and filtering audio to create effects such as echoes and reverb. In the case of indirect audio, Steam Audio uses ray tracing to generate an impulse response (it measures the distance and path audio would travel from source to listener) and then convolution is used to generate a reverb effect which, while very accurate, can be quite computationally intensive with it requiring hundreds of thousands of sound samples. Ambisonics further represent the directional nature of indirect sound which helps to improve positional audio and the immersion factor as sounds are more real-world modeled.

GPU Convolution Performance versus CPU.png

GPU versus CPU convolution (audio filtering) performance. Lower is better.

In addition to the ability of developers to dedicate a portion (up to 20 to 25%) of a GPU's compute units to audio processing, developers can enable/disable TrueAudio processing including the level of acoustic complexity and detail on a scene-by-scene basis. Currently it appears that Unity, FMOD Studio, and C API engines can hook into Steam Audio and the TrueAudio Next features, but it remains up to developers to use the features and integrate them into their games.

Note that GPU-based TrueAudio Next requires a GCN-based graphics card of the RX 470, RX 480, RX 570, RX 580, R9 Fury, R9 Fury X, Radeon Pro Duo, RX Vega 56, and RX Vega 64 variety in order to work, so that is a limiting factor in adoption much like the various hair and facial tech is for AMD and NVIDIA on the visual side of things where the question of is the target market large enough to encourage developers to put in the time and effort to enable X optional feature arises.

I do not pretend to be an audio engineer, nor do I play a GPU programmer on TV but more options are always good and I hope that developers take advantage of the resource reservation and GPU compute convolution algorithms of TrueAudio Next to further the immersion factor of audio as much as they have the visual side of things. As VR continues to become more relevant I think that developers will have to start putting more emphasis on accurate and detailed audio and that's a good thing for an aspect of gaming that has seemingly taken a backseat since Windows Vista. 

What are your thoughts on the state of audio in gaming and Steam Audio's new TrueAudio Next integration?

Also read:

Source: Valve

Tobii Eye Tracking Showed Impressive VR Headset Integration at CES

Subject: General Tech | February 1, 2018 - 05:53 PM |
Tagged: VR, virtual reality, Tobii, htc vive, eye tracking, CES 2018, CES

Last month in Tobii's suite at CES I was given a demonstration of a prototype VR headset that looked like any other HTC Vive - except for the ring of Tobii eye-tracking sensors inside and around each lens. While this might seem like a bit of an odd concept at first I was patient as the benefits were explained to me, and then blown away when I actually tried it myself.


As you know, if you have used a VR headset like the Oculus Rift or HTC Vive, the basic mechanics of VR interaction involve pointing your head in the direction you want to look, reaching with your hand (and controller) to point to an object, and then pressing a button on the controller to act. I will be completely honest here: I don't like it. After a little while the fatigue and general unnatural feeling of rapid, bird-like head movements kills whatever enthusiasm I might have for the experience, and I was the last person to give high praise to a new VR product. HOWEVER, I will attempt to explain why simply adding eye tracking actually made the entire experience 1000 times better (for me, anyway).


When I put on the prototype headset, the only setup I had to do was quickly follow a dot in my field of vision as it moved up/down/left/right, like a vision test for a driver's license. That's the entire calibration process, and with that out of the way I was suddenly able to look around without moving my head, which made the head movements when they followed feel completely natural. I would instinctively look up, or to the side, with my head following when I decided to focus attention on that area. The amount of physical head movements was reduced to normal, human levels, which alone prevented me from feeling sick after a few minutes. Of course, this was not the only demonstrated feature of the integrated eye-tracking, and if you are familiar with Tobii you will know what's next.


This looks primitive, but it was an effective demo of the eye-tracking integration

The ability of the headset to know exactly where you are looking allows you to aim based on your line of sight if the game implements it, and I tried some target practice (throwing rocks at glass bottles in the demo world) and it felt completely natural. After launching a few rocks at distant bottles I instantly decided that this should be the mechanic of fantastic VR football video game - that I could throw at different receivers just by looking them down.

I also received a demo of simulated AR integration (still within the VR world), and a demo of what eye-tracking adds to a home theater experience - and it was pretty convincing. I could scroll around and select movie titles from an interface by simply looking around, and within the VR world it was as if I was looking up at a big projection screen. Throughout the different demos I kept thinking about how much more natural everything felt when I wasn't constantly moving my head around and pointing at things with my controller.


Finally, there was another side to everything I experienced - and it might have been the most interesting thing from a PC enthusiast perspective: if the VR headset can track your focus, the GPU doesn't have to render anything else at full resolution. That alone could make this something of a breakthrough addition to the current VR headset space, as performance is very expensive (even before the mining craze) and absolutely necessary for a smooth, high frame-rate experience. After 45 minutes with the headset on, I felt totally fine - and that was a change.

So what is the takeaway from all this? I'm just an editor who had a meeting with Tobii at CES, and I walked out of the meeting with a couple of business cards and nothing else. I admit that I am a VR skeptic who went into the meeting with no expectations. And I still left thinking it was the best product I saw at the show.

More information and media about the CES demos are available from Tobii on their CES blog post.

Source: Tobii

CES 2018: Lenovo Announces Mirage VR Camera and Untethered Motion-Tracking Daydream Headset

Subject: General Tech | January 9, 2018 - 02:15 PM |
Tagged: VR, lenovo mirage, Lenovo, daydream, CES 2018, CES, camera

UPDATE (2018-01-09 7:38 PM EST): Lenovo has announced new pricing information for its Mirage Solo headset and Mirage VR Camera. In lieu of the specific pricing originally reported, the Mirage Camera will now start at "under $300" while the Mirage Solo will start "under $400." Original article follows below.

Lenovo today announced the Mirage Solo, the first standalone headset compatible with Google's Daydream virtual reality platform. In addition to keeping the processing power, display, and power all confined to the headset for a completely untethered experience, the Mirage Solo also supports "WorldSense," a motion tracking technology that lets users move within a 1.5-meter range of compatible Daydream experiences without the need for external cameras or sensors.


The Mirage Solo is based on the Qualcomm Snapdragon 835 VR platform with 4GB of RAM. On-board storage is provided by microSD and can be expanded up to 256GB. The 256x1440 display offers a 110-degree filed of view, and the whole headset weighs in at 1.42 pounds.


The headset's 4000mAh battery is good for up to 7 hours per charge, although Lenovo notes that this number may vary based on the complexity of the Daydream content and the amount of motion involved.

The Mirage Solo includes a wireless Daydream controller and will start at $449.99 "under $400" when it ships in the second quarter.

Mirage Camera

In addition to the Mirage Solo headset, Lenovo is also launching a product for those who wish to create and share virtual reality experiences. The Mirage Camera is a compact VR capture device that records 180 degrees of video via two 13MP fisheye lenses.


Once captured, the camera features integrated uploading to Google Photos and YouTube for easy sharing. The videos are recorded in the VR180 format, which allows for immersive 3D playback with compatible devices, or controllable 2D video for those without a headset.

The Mirage Camera includes 16GB of built-in storage and can be expanded with up to 128GB of additional storage via a microSD card. The removable battery offers up to two hours of continuous recording per charge.


The Mirage Camera starts at $299.99 "under $300" and will launch in the second quarter. A model with integrated LTE is also planned, but Lenovo has not yet revealed pricing or release window for that version.

Source: Lenovo

CES 2018: HTC announces Vive Pro and Vive Wireless Adapter

Subject: General Tech, Graphics Cards | January 8, 2018 - 06:22 PM |
Tagged: wigig, VR, vive wireless adapter, vive pro, vive, steamvr, Oculus, htc, CES 2018, CES

As it turns out, HTC's teasing of a new Vive product late last week was well warranted. Today, at their press conference before CES 2018, HTC announced two new hardware products—the Vive Pro and the Vive Wireless adapter.

Copy of VIVE-Pro_KV-B_FA.jpg

Just like their teaser indicated, one of the major features of this new Vive hardware is an increased display resolution. The Vive Pro's resolution is 2880x1600 (combined), a 78% increase from the standard 2160×1200 resolution shared by the original Vive and the Oculus Rift.

Currently, there are no details about upgraded optics in the form of new lenses, but considering Valve's announcement of new lenses for future VR headsets, I would expect these to be included in the Vive Pro.

Copy of Vive Pro Profile.png

In addition to the display improvements, there are also some design changes in the Vive Pro that aim to allow users to quickly put on the headset and adjust it for maximum comfortability. The Vive Pro now features a dial on the back of the head strap to adjust the headset rather than having to adjust velcro straps. This setup is very reminiscent of the PSVR headset which is widely regarded as one of the most comfortable VR headsets currently on the market.

Copy of Vive Pro Sizing Dial.png

While we've already seen some of these design changes like integrated headphones in the currently shipping Deluxe Audio Strap for Vive, the Vive Pro is built from the ground up with this new strap instead of it being a replacement option.

Vive Pro Head on.png

HTC was very quiet about the change from a single front-facing camera on the standard Vive to dual front cameras on the Vive Pro. Having stereo cameras on the device have the potential to provide a lot of utility ranging from a stereo view of your surroundings when you are nearing the chaperone boundaries to potential AR applications.


The Vive Pro will work with the current 1.0 base stations for positional tracking, as well as Valve's previously announced but unreleased 2.0 base stations. When using SteamVR 2.0 tracking, the Vive Pro supports up to 4 base stations, allowing for a  significantly larger play area of up to 10m x 10m.

Initially, the Vive Pro is slated to ship this quarter as a headset-only upgrade for customers who already have the original Vive with its 1.0 base stations. The full Vive Pro kit with 2.0 tracking is said to ship in the Summer time frame. Pricing for both configurations is yet to be announced.

In addition to new headset hardware, HTC also announced their first official solution for wireless VR connectivity. 

Copy of VIVE_Pro_Wireless_KV_Square.jpg

Built in partnership with Intel, the Vive Wireless Adapter will use 60 GHz WiGig technology to provide a low latency experience for wirelessly streaming video to the HMD. Both the original Vive and the Vive Pro will support this adapter set to be available this summer. We also have no indications of pricing on the Vive Wireless Adapter.

HTC's announcements today are impressive and should help push PC VR forward. We have yet to get hands-on experience with either the Vive Pro or the Vive Wireless adapter, but we have a demo appointment tomorrow, so keep checking PC Perspective for our updated impressions of the next generation of VR!

Source: HTC

CES 2018: HTC Teases new VIVE device announcement

Subject: General Tech | January 5, 2018 - 03:53 PM |
Tagged: VR, vive focus, vive, valve, steam vr, Oculus, knuckles, htc, CES 2018

In the run-up to CES 2018, the HTC Vive account on twitter tweeted an image teasing their press conference on Monday, January 8. 

While we've already heard about the standalone Snapdragon 835-powered HTC VIVE Focus set to start shipping in China later this month, it seems unlikely for this teaser to be referring to that device which HTC has already shown off to the public and press.


Rather, it seems more likely that HTC is ready to start to talk about their next generation VIVE device for PC.

Both the graphic and tagline of "New Year's Resolution" obviously seem to be alluding to improved visual quality, and hopefully less of a screen door effect when wearing the headset. Whether or not this is achieved by a higher resolution display, or improved optics, or both has yet to be determined. 


Another thing to look out for at this event will be the long-awaited "Knuckles" controllers showed off by Valve earlier this year. With a new generation of VR headset, I would expect the Knuckles controllers to be the bundled option going forward.

We'll be at HTC's VIVE press conference on the 8th, and have some time scheduled late in the week for demos of whatever they are showing off, so stay tuned for more details about their announcements as they happen!

Source: HTC

Top Gun (1986) in 3D VR Promotion via Bigscreen

Subject: General Tech | December 30, 2017 - 03:59 PM |
Tagged: VR, steamvr, oculus vr, bigscreen, htc vive, oculus rift

Until their last showing at 2am tonight, Bigscreen, Inc. is playing Top Gun in 3D through their VR application for Windows 10. It can be used with the Oculus Rift, SteamVR, and Windows Mixed Reality headsets. Both the movie and the app are free, at least during this promotion, but it is only available in the United States. As such, I have not been able to try it out and give my opinion. They plan to add more content throughout 2018, including TV shows, spread out between free and paid showings.


As for the showings? It makes sense from the standpoint of creating a social experience. You know that everyone is watching the same thing at the same time, which could lead to some interesting features like voice chat (for those who want to opt-in to talking throughout a movie).

The company is planning on bringing the app to Samsung GearVR, Google Daydream, and Oculus Go. This is obviously good, because the barrier to entry will be much lower if you can get away with a cheap-ish accessory to a phone that you already have for other reasons. There’s still a question of whether VR will take off in the consumer space at all, but this is a cute application none-the-less. That said, I regularly use YouTube VR with Google Daydream, mostly for relaxation videos before bed; you can decide for yourself whether I’m an outlier or simply ahead of my time.

Source: Bigscreen

A walrus on virtual rollerskates

Subject: General Tech | November 9, 2017 - 02:02 PM |
Tagged: hyneman, roller skates, VR, Vortrex Shoes

Jamie Hyneman is pitching a project to build prototype VR roller skates; not as a game but as a way to save your shins while using a VR headset.  The design places motorized wheels under your heel and a track under the ball of your foot which will move your foot back to its starting position if you walk forward.  If all goes as planned this should allow you to walk around in virtual worlds without running into walls, chairs or spectators and perhaps allow games to abandon the point and teleport currently in vogue.  There are a lot of challenges as previous projects have discovered but perhaps a Mythbuster can help out.  You can watch his pitch video over at The Register.


"Hyneman's pitch video points out that when one straps on goggles and gloves to enter virtual reality, your eyes are occupied and you therefore run the risk of bumping into stuff if you try to walk in meatspa ce while simulating walking in a virtual world. And bumping into stuff is dangerous."

Here is some more Tech News from around the web:

Tech Talk

Source: The Register