Connection is a new headset that mixes realistic objects with the factual earth.
A few days ago, I played fetch with a robot. I threw a ball, and the robot — named Bridget — zoomed around one of our meeting rooms trying to retrieve it, along the way dodging a table, a box, and a chair. When the game was over, I took off the headset I was wearing and endured a brief moment of panic — because Bridget was gone.
Bridget is a virtual robot that only existed on the iPhone powering Capital's new “mixed reality” headset. That headset, called Bridge, had mapped out the room around me and created an invisible digital overlay that robots like Bridget could react to, making them feel like a real part of the world. During the 5-minute demo, Bridget interacted so well with that digital re-creation of the room I was standing in that my brain was briefly fooled into thinking that this pixelated robot was, on some level, real.
Like a Gear VR, except you can move around in the virtual space.
In some ways, Bridge is a lot like any other mobile VR headset: you snap in an iPhone 6, 6S, or 7, strap it on your head and enjoy smartphone-powered virtual reality. It even comes with a Bluetooth remote that can handle some motion, similar to what you get with Google’s Daydream VR headset. The bridge is like a mix between Microsoft’s Holocene augmented reality glasses and the Samsung Gear VR, though it’s priced closer to the latter. It will be available in limited quantities (several hundred or so) starting next week for $499, and Occidental is planning a wider release in March where it will sell for $399.
But what makes Bridge unique is how it integrates Occidentals' first product: the Structure Sensor, a marker-sized strip of cameras and sensors that can map physical objects and environments with incredible levels of detail. The Structure Sensor adds inside-out positional tracking to the headset, allowing people to move around in real space without external cameras or sensors. This means that instead of just sitting in one spot and spinning around, you could actually lean your head and walk around inside a VR game or experience — something that was previously hard to come by in a mobile setting.
I, unfortunately, didn’t get to try a purely virtual experience, like a video game, with reverse-engineered positional tracking. But I did get to try that inside-out tracking as part of the thing Occidental is most proud of with Bridge: that idea of mixed reality.
“Mixed reality” refers to experiences that mix augmented reality overlays with the immersion of virtual reality — instead of just adding a heads-up display to the world, they actually alter our reality. Microsoft has used the term when talking about its Holocene headset. Still, where Holocene projects images onto a clear pane of glass, Occidental uses the iPhone’s camera (with the help of a lens attachment to widen the view) and mixes that video feed with the data from the Structure Sensor to create a 3D stereo view of the room you’re in.
All this means that the Bridge hardware isn’t necessarily sleek or slim. Thanks to semi-rigid straps that help distribute the weight and a bike helmet-style ratchet system in the back, it's comfortable enough. But it’s not the most approachable hardware. If Google’s Daydream seems like the perfect headset to wear on the couch at the end of the day, Bridge looks more like the kind of thing you’d wear in the Battle of Vendor.
The bridge looks like something you’d wear at the Battle of Vendor.
There’s also a lot more work to be done before Bridge could be considered a mainstream product. While the positional tracking feels accurate, there’s a small delay that can make you a bit queasy. When split into two eyes, the resolution of the iPhone screen is only 640 x 480. And while Bridge outputs at 60 frames per second, the iPhone only allows the app to read the camera feed at 30 frames per second — Occidental is actually using sensors to predict how you’re moving from one frame to the next. Synthetically creating what they think the view should look until the next frame comes in. The result is smoother motion but with some noticeable pudginess.
Still, there’s power in being able to move around a room and have it track to what you see on the headset. To do this previously, you had to invest in incredibly powerful desktop PCs, place sensors or cameras around the room, and your headset needed to be wired. Like the many other companies that have recently shown an interest in inside-out tracking, Occidental is betting some people will accept the trade-offs to gain access to that freedom.
Occidental CEO Jeff Powers says he’s not sure what experiences will work best on a headset like Bridge. Mixed reality presents a whole new set of problems and possibilities, and Occipital wants help sussing those out. “In games and VR, the whole world is yours to decide. You can statically create the world and then decide what happens in that,” he says. "In mixed reality, you’re forced to work within the constraints of the real world. So your creatures, your characters, whatever they are, have to deal with this world. We don’t really know as a developer community all the things we need to deal with.”
This is where Bridget the robot comes in. The software that summons Bridget is part of what Occidental calls the “Bridge engine” — essentially the code that runs everything — and the company plans to open-source this part of the experience so that it can collaborate with developers on creating mixed-reality experiences.
The Bridge demo Occidental gave me wasn’t just about playing fetch with a robot, though that was certainly the most fun part. I got to do other things, like summon a portal that turned one of our meeting rooms into the deck of a spaceship.
Being able to digitally manipulate the environment around you is a big deal.
I was also able to drop accurately scaled furniture into the room with the press of a button on the Bluetooth remote. And while there is plenty of software that helps customers decide on things like new wallpaper or furniture, imagine being able to pop on a headset and see for yourself what that new table would look like in your kitchen, with the ability to move around it in space. The bridge is ready for experiences like this, and it’s an area Occidental is already familiar with — just one month ago, Occidental announced a new app that uses the Structure Sensor to create CAD-quality scans of entire rooms.
“We’ve heard a lot of complaints from people in industries like interior design or architecture, and they want to show VR scenes, but their customers don’t have giant PCs,” Powers says. The bridge could be a powerful way for developers to bring these experiences to mobile settings.
Occidentals are not alone in identifying this sweet spot between current mobile VR headsets and their souped-up desktop counterparts. Microsoft has been working on Holocene for a while now. Google’s Tango uses a flat tablet instead of a headset, but it can already handle things like simulated interior decorating. Intel recently unveiled Project Alloy's own “merged reality” VR headset. And Locus is working on a new headset that allows for positional tracking, called Santa Cruz. But Santa Cruz is still very much in the prototype phase, Intel doesn’t intend to release Alloy commercially, and Holocene, for now, is prohibitively expensive.
Occidental has an advantage in that Bridge is relatively cheap, more open, and more available than these other platforms. It also benefits from being one of the few mobile VR headsets for iPhone that isn’t just folded cardboard or cheap plastic.
Of course, Apple is also rumored to be going down a similar road concerning VR and AR, adding another competitor. When I pointed this out to Powers, he laughed and said: “Hopefully, our software engine will be one of the leading ways to develop for it.”