Subject: General Tech | March 21, 2018 - 09:20 AM | Sebastian Peak
Tagged: xr, VR, Tobii, qualcomm, HMD, GDC 2018, GDC, eye-tracking, developers, dev kit, AR
We have recently covered news of Qualcomm's ongoing VR/AR efforts (the two terms now combine as "XR", for eXtended reality), with news of the Snapdragon 845-powered reference HMD and more recently the collaboration with Tobii to bring eye-tracking to the Qualcomm development platform. Today at GDC Qualcomm is mapping out their vision for the future of XR, and providing additional details about the Snapdragon 845 dev kit - and announcing support for the HTC Vive Wave SDK.
For the first time, many new technologies that are crucial for an optimal and immersive VR user experience will be supported in the Snapdragon 845 Virtual Reality Development Kit. These include:
- Room-scale 6DoF SLAM: The Snapdragon 845 Virtual Reality Development Kit is engineered to help VR developers create applications that allow users to explore virtual worlds, moving freely around in a room, rather than being constrained to a single viewing position. Un-tethered mobile VR experiences like these can benefit from the Snapdragon 845 Virtual Reality Development Kit’s pre-optimized hardware and software for room-scale six degrees of freedom (6DoF) with “inside-out” simultaneous localization and mapping (SLAM). All of this is designed to be accomplished without any external setup in the room by the users, and without any cables or wires.
- Qualcomm® Adreno™ Foveation: Our eyes are only able to observe significant details in a very small center of our field of vision - this region is called the “fovea”. Foveated rendering utilizes this understanding to boost performance & save power, while also improving visual quality. This is accomplished through multiple technology advancements for multi-view, tile-based foveation with eye-tracking and fine grain preemption to help VR application developers deliver truly immersive visuals with optimal power efficiency.
- Eye Tracking: Users naturally convey intentions about how and where they want to interact within virtual worlds through their eyes. Qualcomm Technologies worked with Tobii AB to develop an integrated and optimized eye tracking solution for the Snapdragon 845 VR Development Kit. The cutting-edge eye tracking solution on Snapdragon 845 VR Development Kit is designed to help developers utilize Tobii’s EyeCore™ eye tracking algorithms to create content that utilizes gaze direction for fast interactions, and superior intuitive interfaces.
- Boundary System: The new SDK for the Snapdragon 845 VR Development Kit supports a boundary system that is engineered to help VR application developers accurately visualize real-world spatial constraints within virtual worlds, so that their applications can effectively manage notifications and play sequences for VR games or videos, as the user approaches the boundaries of the real-world play space.
In addition to enhancing commercial reach for the VR developer community, Qualcomm Technologies is excited to announce support for the HTC Vive Wave™ VR SDK on the Snapdragon 845 Virtual Reality Development Kit, anticipated to be available later this year. The Vive Wave™ VR SDK is a comprehensive tool set of APIs that is designed to help developers create high-performance, Snapdragon-optimized content across diverse hardware vendors at scale, and offer a path to monetizing applications on future HTC Vive ready products via the multi-OEM Viveport™ application store.
The Snapdragon 845 HMD/dev kit and SDK are expected to be available in Q2 2018.
Subject: General Tech | March 16, 2018 - 09:45 AM | Sebastian Peak
Tagged: xr, VR, Tobii, snapdragon 845, qualcomm, mobile, HMD, head mounted display, eye tracking, AR, Adreno 630
Tobii and Qualcomm's collaboration in the VR HMD (head-mounted display) space is a convergence of two recent stories, with Tobii's impressing showing of a prototype HMD device at CES featuring their eye-tracking technology, and Qualcomm's unvieling last month of their updated mobile VR platform, featuring the new Snapdragon 845.
The Qualcomm Snapdragon 845 Mobile VR Reference Platform
What does this new collaboration mean for the VR industry? For now it means a new reference design and dev kit with the latest tech from Tobii and Qualcomm:
"As a result of their collaboration, Tobii and Qualcomm are creating a full reference design and development kit for the Qualcomm Snapdragon 845 Mobile VR Platform, which includes Tobii's EyeCore eye tracking algorithms and hardware design. Tobii will license its eye tracking technologies and system and collaborate with HMD manufacturers on the optical solution for the reference design."
The press release announcing this collaboration recaps the benefits of Tobii eye tracking in a mobile VR/AR device, which include:
- Foveated Rendering: VR/AR devices become aware of where you are looking and can direct high-definition graphics processing power to that exact spot in real time. This enables higher definition displays, more efficient devices, longer battery life and increased mobility.
- Interpupillary Distance: Devices automatically orient images to align with your pupils. This enables devices to adapt to the individual user, helping to increase the visual quality of virtual and augmented reality experiences.
- Hand-Eye Coordination: By using your eyes in harmony with your hands and associated controllers, truly natural interaction and immersion, not possible without the use of gaze, is realized.
- Interactive Eye Contact: Devices can accurately track your gaze in real time, enabling content creators to express one of the most fundamental dimensions of human interaction – eye contact. VR technologies hold the promise of enabling a new and immersive medium for social interaction. The addition of true eye contact to virtual reality helps deliver that promise.
Tobii's prototype eye-tracking HMD
For its part, Qualcomm's Snapdragon 845-powered VR mobile platform promises greater portability of a better VR experience, with expanded freedom on top of the improved graphics horsepower from the new Adreno 630 GPU in the Snapdragon 845. This portability includes 6DoF (6 degrees of freedom) using external cameras to identify location within a room, eliminating the need for external room sensors.
"Together, 6DoF and SLAM deliver Roomscale - the ability to track the body and location within a room so you can freely walk around your XR environment without cables or separate room sensors – the first on a mobile standalone device. Much of this is processed on the new dedicated Qualcomm Hexagon Digital Signal Processor (DSP) and Adreno Graphics Processing Unit within the Snapdragon 845. Qualcomm Technologies’ reference designs have supported some of the first wave of standalone VR devices from VR ecosystem leaders like Google Daydream, Oculus and Vive."
It is up to developers, and consumer interest in VR moving forward, to see what this collaboration will produce. To editorialize briefly, from first-hand experience I can vouch for the positive impact of eye-tracking with an HMD, and if future products live up to the promise of a portable, high-performance VR experience (with a more natural feel from less rapid head movement) a new generation of VR enthusiasts could be born.
Subject: General Tech | November 5, 2017 - 08:14 PM | Scott Michaud
Tagged: Unity, zspace, xr, AR, VR
The Unity Educator Toolkit was created by Unity3D to integrate learning game development into the K-12 public curriculum. Now zSpace, which we’ve mentioned a few times, is joining in to the initiative with their mixed-reality platform. The company is known for creating displays that, when viewed with their glasses, track where you are and make the object appear to be in front of you. They also have a stylus that lets you interact with the virtual object.
They are focused on the educational side of VR and AR.
It’s not entirely clear what this means, because a lot of the details are behind a sign-up process. That said, if you’re an educator, then check out the package to see if it’s relevant for you. Creating games is an interesting, albeit challenging and somewhat daunting, method of expressing oneself. Giving kids the tools to make little game jam-style expressions, or even using the technology in your actual lessons, will reach a new group of students.
Subject: Mobile | June 27, 2017 - 08:00 PM | Ryan Shrout
Tagged: xr, VR, qualcomm, google, daydream, AR
Qualcomm has put forward steady work on creating the vibrant hardware ecosystem for mobile VR to facilitate broad adoption of wireless, dedicated head mounted displays. Though the value of Samsung’s Gear VR and Google’s Daydream View cannot but overstated in moving the perception of consumer VR forward, the need to utilize your smart phone in a slot-in style design has its limitations. It consumes battery that you may require for other purposes, it limits the kinds of sensors that the VR system can utilize, and creates a sub-optimal form factor in order to allow for simple user installation.
The Qualcomm Snapdragon 835 VR Reference Device
Qualcomm created the first standalone VR HMD reference design back in early 2016, powered by the Snapdragon 820 processor. Google partnered with Qualcomm at I/O to create the Daydream standalone VR headset reference design with the updated Snapdragon 835 Mobile Platform at its core, improving performance and graphical capability along the way. OEMs like Lenovo and HTC have already committed to Daydream standalone units, with Qualcomm at the heart of the hardware.
Qualcomm Technologies recently announced a HMD Accelerator Program (HAP) to help VR device manufacturers quickly develop premium standalone VR HMDs. At the core of this program is the standalone VR HMD reference design. It goes beyond a simple prototype device, offering a detailed reference design that allows manufacturers to apply their own customizations while utilizing our engineering, design, and experience in VR. The reference design is engineered to minimize software changes, hardware issues, and key component validation.
- Hugo Swart, Qualcomm Atheros, Inc.
As part of this venture, and to continue pushing the VR industry forward to more advanced capabilities like XR (extended reality, a merger of VR and AR), Qualcomm is announcing agreements with key component vendors aiming to tighten and strengthen the VR headset ecosystem.
Hugo Swart, Senior Director, Product Management, Qualcomm Atheros, Inc.
Ximmerse has built a high-precision and drift-free controller for VR applications that offers low latency input and 3DoF (3 degrees of freedom) capability. This can “provide just about any interaction, such as pointing, selecting, grabbing, shooting, and much more. For precise 6 DoF positional tracking of your head, tight integration is required between the sensor fusion processing (Snapdragon) and the data from both the camera and inertial sensors.”
Bosch Sensortec has the BMX055 absolute orientation sensor that performs the function that its name would imply: precisely locating the user in the real world and tracking movement via accelerometer, gyroscope, and magnetometer.
Finally, OmniVision integrates the OV9282 which is a 1MP high speed shutter image sensor for feature tracking.
These technologies, paired with the work Qualcomm has already done for the Snapdragon 835 VR Development Kit, including on the software side, is an important step to the growth of this segment of the market. I don’t know of anyone that doesn’t believe standalone, wireless headsets are the eventual future of VR and AR and the momentum created by Qualcomm, Google, and others continues its steady pace of development.
Subject: General Tech | May 27, 2017 - 10:11 PM | Scott Michaud
Tagged: xr, VR, mr, istaging, AR
iStaging is virtual-, augmented-, and mixed-reality company that focuses on the real estate, interior design, furniture, and related industries. The news that lead to this post is that Yungching Realty Group, based out of Taiwan, has partnered with iStaging to enhance their real estate business with VR and AR. The demo that they are showing at their press conference was a virtual street, which presented information about restaurants, schools, and other points of interest for someone researching the neighborhood.
I’d expect our audience is more interested in the technology side of this, although let us know in the comments (or via email – my address is in my author page linked on the byline) if you’re interested in the enterprise / real-estate side. From the technology standpoint, it’s interesting to see applications like these push high-end graphics into more and more businesses, large and small. Likewise, these applications give a stable income that XR technology companies (ex: HTC Vive) can rely upon while they find a foothold in fickle, but potentially lucrative consumer market.
Lastly, I’m curious what applications will be possible when another round of innovation learns from this generation. What does this enable, even if only by expanding what people think is possible?
Definitely something to think about.