Subject: General Tech | March 16, 2018 - 09:45 AM | Sebastian Peak
Tagged: xr, VR, Tobii, snapdragon 845, qualcomm, mobile, HMD, head mounted display, eye tracking, AR, Adreno 630
Tobii and Qualcomm's collaboration in the VR HMD (head-mounted display) space is a convergence of two recent stories, with Tobii's impressing showing of a prototype HMD device at CES featuring their eye-tracking technology, and Qualcomm's unvieling last month of their updated mobile VR platform, featuring the new Snapdragon 845.
The Qualcomm Snapdragon 845 Mobile VR Reference Platform
What does this new collaboration mean for the VR industry? For now it means a new reference design and dev kit with the latest tech from Tobii and Qualcomm:
"As a result of their collaboration, Tobii and Qualcomm are creating a full reference design and development kit for the Qualcomm Snapdragon 845 Mobile VR Platform, which includes Tobii's EyeCore eye tracking algorithms and hardware design. Tobii will license its eye tracking technologies and system and collaborate with HMD manufacturers on the optical solution for the reference design."
The press release announcing this collaboration recaps the benefits of Tobii eye tracking in a mobile VR/AR device, which include:
- Foveated Rendering: VR/AR devices become aware of where you are looking and can direct high-definition graphics processing power to that exact spot in real time. This enables higher definition displays, more efficient devices, longer battery life and increased mobility.
- Interpupillary Distance: Devices automatically orient images to align with your pupils. This enables devices to adapt to the individual user, helping to increase the visual quality of virtual and augmented reality experiences.
- Hand-Eye Coordination: By using your eyes in harmony with your hands and associated controllers, truly natural interaction and immersion, not possible without the use of gaze, is realized.
- Interactive Eye Contact: Devices can accurately track your gaze in real time, enabling content creators to express one of the most fundamental dimensions of human interaction – eye contact. VR technologies hold the promise of enabling a new and immersive medium for social interaction. The addition of true eye contact to virtual reality helps deliver that promise.
Tobii's prototype eye-tracking HMD
For its part, Qualcomm's Snapdragon 845-powered VR mobile platform promises greater portability of a better VR experience, with expanded freedom on top of the improved graphics horsepower from the new Adreno 630 GPU in the Snapdragon 845. This portability includes 6DoF (6 degrees of freedom) using external cameras to identify location within a room, eliminating the need for external room sensors.
"Together, 6DoF and SLAM deliver Roomscale - the ability to track the body and location within a room so you can freely walk around your XR environment without cables or separate room sensors – the first on a mobile standalone device. Much of this is processed on the new dedicated Qualcomm Hexagon Digital Signal Processor (DSP) and Adreno Graphics Processing Unit within the Snapdragon 845. Qualcomm Technologies’ reference designs have supported some of the first wave of standalone VR devices from VR ecosystem leaders like Google Daydream, Oculus and Vive."
It is up to developers, and consumer interest in VR moving forward, to see what this collaboration will produce. To editorialize briefly, from first-hand experience I can vouch for the positive impact of eye-tracking with an HMD, and if future products live up to the promise of a portable, high-performance VR experience (with a more natural feel from less rapid head movement) a new generation of VR enthusiasts could be born.
Subject: General Tech | November 5, 2017 - 08:14 PM | Scott Michaud
Tagged: Unity, zspace, xr, AR, VR
The Unity Educator Toolkit was created by Unity3D to integrate learning game development into the K-12 public curriculum. Now zSpace, which we’ve mentioned a few times, is joining in to the initiative with their mixed-reality platform. The company is known for creating displays that, when viewed with their glasses, track where you are and make the object appear to be in front of you. They also have a stylus that lets you interact with the virtual object.
They are focused on the educational side of VR and AR.
It’s not entirely clear what this means, because a lot of the details are behind a sign-up process. That said, if you’re an educator, then check out the package to see if it’s relevant for you. Creating games is an interesting, albeit challenging and somewhat daunting, method of expressing oneself. Giving kids the tools to make little game jam-style expressions, or even using the technology in your actual lessons, will reach a new group of students.
Subject: Mobile | June 27, 2017 - 08:00 PM | Ryan Shrout
Tagged: xr, VR, qualcomm, google, daydream, AR
Qualcomm has put forward steady work on creating the vibrant hardware ecosystem for mobile VR to facilitate broad adoption of wireless, dedicated head mounted displays. Though the value of Samsung’s Gear VR and Google’s Daydream View cannot but overstated in moving the perception of consumer VR forward, the need to utilize your smart phone in a slot-in style design has its limitations. It consumes battery that you may require for other purposes, it limits the kinds of sensors that the VR system can utilize, and creates a sub-optimal form factor in order to allow for simple user installation.
The Qualcomm Snapdragon 835 VR Reference Device
Qualcomm created the first standalone VR HMD reference design back in early 2016, powered by the Snapdragon 820 processor. Google partnered with Qualcomm at I/O to create the Daydream standalone VR headset reference design with the updated Snapdragon 835 Mobile Platform at its core, improving performance and graphical capability along the way. OEMs like Lenovo and HTC have already committed to Daydream standalone units, with Qualcomm at the heart of the hardware.
Qualcomm Technologies recently announced a HMD Accelerator Program (HAP) to help VR device manufacturers quickly develop premium standalone VR HMDs. At the core of this program is the standalone VR HMD reference design. It goes beyond a simple prototype device, offering a detailed reference design that allows manufacturers to apply their own customizations while utilizing our engineering, design, and experience in VR. The reference design is engineered to minimize software changes, hardware issues, and key component validation.
- Hugo Swart, Qualcomm Atheros, Inc.
As part of this venture, and to continue pushing the VR industry forward to more advanced capabilities like XR (extended reality, a merger of VR and AR), Qualcomm is announcing agreements with key component vendors aiming to tighten and strengthen the VR headset ecosystem.
Hugo Swart, Senior Director, Product Management, Qualcomm Atheros, Inc.
Ximmerse has built a high-precision and drift-free controller for VR applications that offers low latency input and 3DoF (3 degrees of freedom) capability. This can “provide just about any interaction, such as pointing, selecting, grabbing, shooting, and much more. For precise 6 DoF positional tracking of your head, tight integration is required between the sensor fusion processing (Snapdragon) and the data from both the camera and inertial sensors.”
Bosch Sensortec has the BMX055 absolute orientation sensor that performs the function that its name would imply: precisely locating the user in the real world and tracking movement via accelerometer, gyroscope, and magnetometer.
Finally, OmniVision integrates the OV9282 which is a 1MP high speed shutter image sensor for feature tracking.
These technologies, paired with the work Qualcomm has already done for the Snapdragon 835 VR Development Kit, including on the software side, is an important step to the growth of this segment of the market. I don’t know of anyone that doesn’t believe standalone, wireless headsets are the eventual future of VR and AR and the momentum created by Qualcomm, Google, and others continues its steady pace of development.
Subject: General Tech | May 27, 2017 - 10:11 PM | Scott Michaud
Tagged: xr, VR, mr, istaging, AR
iStaging is virtual-, augmented-, and mixed-reality company that focuses on the real estate, interior design, furniture, and related industries. The news that lead to this post is that Yungching Realty Group, based out of Taiwan, has partnered with iStaging to enhance their real estate business with VR and AR. The demo that they are showing at their press conference was a virtual street, which presented information about restaurants, schools, and other points of interest for someone researching the neighborhood.
I’d expect our audience is more interested in the technology side of this, although let us know in the comments (or via email – my address is in my author page linked on the byline) if you’re interested in the enterprise / real-estate side. From the technology standpoint, it’s interesting to see applications like these push high-end graphics into more and more businesses, large and small. Likewise, these applications give a stable income that XR technology companies (ex: HTC Vive) can rely upon while they find a foothold in fickle, but potentially lucrative consumer market.
Lastly, I’m curious what applications will be possible when another round of innovation learns from this generation. What does this enable, even if only by expanding what people think is possible?
Definitely something to think about.