SteamVR HTC Vive In-depth - Lighthouse Tracking System Dissected and Explored
The SteamVR Lighthouse Tracking System
Principles of Operation
Enter the Lighthouse tracking system. Pioneered by Alan Yates of Valve, this system uses Beacons (a.k.a. Base Station) to emit precisely timed IR pulses (blinks) and X/Y axis IR laser sweeps. Instead of cameras or IR LEDs mounted to the HMD and controllers, Lighthouse embeds an array of IR-filtered photodiodes within all items that require tracking. As the X and Y ‘plane’ of IR laser light sweeps past the various sensors embedded within the controllers and HMD, those diodes’ outputs are amplified and passed onto an internal ASIC, which is programmed with the relative location where each input signal was sourced. Provided there are enough inputs (sensors that have a direct line of sight to one or both Base Stations), the ASIC can then work out its own location and orientation within the room.
The HTC Vive HMD uses over a dozen IR sensors to locate itself and its orientation.
That was a lot to digest, so let’s try and get a better handle on things by taking a look inside, shall we?
Going to the Base Station first, we can see a relatively simple looking box with a lot going on inside.
Starting at the top, we have the IR blinker array. The reason for the number of LEDs here is to increase the brightness of the synchronization pulses. It is also possible to encode simple intelligence into the blinks (for identification purposes where there are multiple Base Stations).
At either side here we see a pair of brushless permanent magnet three-phase synchronous motors. I’ve seen a few fans of the Rift complain about these ‘moving parts’ as a point of potential failure. To quench that issue, consider that these are the same types of motors used in hard drives, but these spin at half the speed (3600 RPM), and are designed to spin continuously for years without failure, so no worries there.
Each motor spins an optical flywheel with a ‘line lens’ of the multiple rod type. This lens takes the internal laser beam and fans it out in the form of a line perpendicular to the direction of the rods. The end result is an IR laser line that sweeps around the radius of each spinning rotor. We will get more into the details of how these work later on.
Straight down from the blinker array and between the two flywheels is a single photodiode, which is used to synchronize multiple beacons (mitigating the need for a sync cable, though one can be used in obscured environments). Just below the photodiode we see a small PCB containing a discrete amplifier. A ribbon then carries the amplified signal to the bottom PCB.
At the rear, we find some beefy capacitors, drive electronics for the flywheel motors, and power / sync / USB connectors. The USB connector is used for firmware updates.
Flipping back around to the rear we find two sets of wires leading to a pair of IR laser diodes, which fire across each other and down the center of each flywheel assembly, where a 45 degree mirror reflects it towards the multi-rod laser lens, forming the line that sweeps either horizontally or vertically (depending on which flywheel you are looking at).
We found some examples and even some high-speed video of a Base Station in operation, but most of the information out there is incorrect or based on the development hardware. This meant we had to shoot our own video.
The above video was shot at 960 FPS and slowed to 1/160th of real time speed. The red flashes you see are dimmer than we would like because IR was mostly filtered by our camera’s internal optics, but there is enough to get the point. If you watch the video closely, you’ll note that there is a ‘blink’ from the IR LED array in-between alternating passes of each flywheel lens. The blinks occur every 8.33 ms (120 Hz), and the two rotating lenses are interleaved, each passing by every 16.6 ms (60 Hz). Those with a keen eye will note that the flywheel lasers are only intermittent – the pattern is blink / X-sweep / blink / Y-sweep / blink / (none) / blink / (none). This is different from what you might have seen in other videos, partially because our video was taken with two Base Stations in-sync with each other. Lighthouse can’t work properly with multiple sweeps simultaneously overlapping each other, so each beacon must take a turn. The ‘blinks’ themselves are automatically synchronized across all beacons and intended to act like an IR camera flash across the whole room while the sweeps are interleaved among them. Below is a video representation of what is happening here (but assumes only a single beacon is in operation – no interleaving):
As with any optical positional tracking implementation, there are limits. To get an initial position and orientation ‘lock’, a device must have at least 5 sensors ‘lit’ by a Base Station (or 3 if two Base Stations are in view). Once locked, the device can use data from an internal gyro + accelerometer (IMU) to track its own location in real-time, meaning it knows where it is during the periods of time between Base Station sweeps or even if all sensors have been temporarily blocked from view (a form of dead reckoning). Provided a fix was recently established, the system can still determine its location in the room with just a single sensor and two beacons.
Now that you hopefully have a grasp on how the Base Stations blink and sweep the room, let’s crack open one of the controllers:
This is the top facing portion of the Vive’s controller ring. Here we can make out 11 sensor units.
A closer look at some of the sensors. These are tucked into nearly every crevice of the ring portion of the controller. There are 24 total sensors on the ring.
That about wraps things up here. We hope that answers some of the questions and curiosities that some of you might have had about the Lighthouse Tracking System. Feel free to discuss further in the comments below!