What a great experience at Oculus Connect 6 this year in San Jose, California! This is CADmech’s first presence at the event and won’t be our last. Below is a recap of the key takeaways from the event from our Lead VR/AR Developer, Victor Truong.
Demos, Demos, and More Demos!
Throughout the event the overwhelmingly hot topic was definitely Hand Tracking that is coming to the Oculus Quest in early 2020. At the event there were two demos that showed just how impressive this new software addition can be. Talespin Farmer’s Insurance Training demo and Magnopus Elixir. Unfortunately, I was only able to try the training demo but here is a link to a review on the Magnopus Elixir demo.
Talespin Farmers Insurance Training Demo
After the opening keynote and reveal of Hand Tracking coming to the Quest, I quickly rushed over to the Talespin’s hand tracking training demo for Farmers Insurance. In this experience, a user is placed into the shoes of a water damage investigator. Users would have 1 min to walk around in a kitchen trying to find all the water damage and assessing how to fix the problem with all the interactions done with hand tracking.
To my surprise, the moment I placed on the headset and placed my hands in front of me there they were in VR! I did some quick tests to really try to trick the hand tracking and for the most part it works very similar to the Leap Motion. It tracks the hands fairly well with some more obvious limitations that are common with vision-based hand tracking.
Instances such as placing the hands together, reaching out with your arm occluding the hands, or fingers overlapping are just some examples of where the hand tracking is lost. Right now, the hand tracking is mainly accomplished through a combination of vision-based tracking and using a deep neural network to make educated guesses of your hand positions based from numerous pictures and videos of other hands. Overall, the experience was very well designed such as making sure the water damage was never eye level to eliminate the concern of the arm occluding the hands and making sure they were in separate corners of the room to force people to walk around the space to fully enjoy the power of VR training. Ultimately, even with the limitations there are tons of possibilities for hand tracking in more immersive training and eliminating the controllers would be a huge plus in adoption of the technology.
Mixed Reality Capture Demo
Next demo that I got to try was the Mixed Reality Capture Demo which allows video capture of a user in a VR experience using just a tablet and without a green screen. First off, this demo seemed to be in the very early stages of development. Users were not allowed to be inside the VR experience but are only allowed to hold an Apple iPad Pro tablet to experience the capture portion. Here’s maybe why. Because there is no green screen to aid with the cropping of the body, the Oculus representative in this case had to ensure their arms were never too close to the body and cannot perform sudden movements so that the processing can be done fast enough without wrong cropping of the body. That being said, if you have the correct hardware and enough processing power this works quite well (as you can see from the video) but I’m curious to see how this will work on regular mobile phones. I guess I’ll just have to keep an eye out for more updates on this feature in the future but what this does hopefully enable is to allow more inclusion in VR experiences which allows spectators a new way to experience VR with those using the headsets. I can also see this being useful for multi-user training applications.
Facebook Shared Spaces
This was more of a social VR demo which allowed users to be in a recreated real-world room in VR while enjoying a ring toss mini game and watching TV with another individual in VR. They placed me in the real-world room first where you got to examine the room before you see the digital representation of it in VR. Once, I got into the experience it was quite amazing. The recreation of the room is a small glance of the efforts at Facebook to create what they like to call the Real World Index which are the separate layers required for full Mixed Reality in our daily lives.
Overall, the recreation of the room was done with a 3D scanner which was able to pick up model meshes that were occluded such as underneath tables, inside lamp shades, and under the sofa which was quite impressive. Can’t wait to see the progression of the digital twin efforts Facebook Reality Labs has in store for the future if this is how far they have already come.
Last Key Takeaways
Oculus link was also another hot topic at OC6. What this enables is for Quest users to accomplish remote rendering with the use of a USB-C cable connected to a gaming PC. This ultimately, allows for all the high-quality Oculus Rift games to be playable right on a Quest device. Offloading the processing power of rendering on the Quest to a remote computer has always been a mandate for Oculus and this is a great first step towards that direction.
The CTRL-Labs acquisition made by Facebook is also huge news. In the future, Mark Zuckerberg believes our interactions in VR and AR will be controlled solely through our brain thoughts which is also known as neural interface. Using neural interface in combination with VR and AR brings us just one step closer to fully immersive experiences. CRTL-Labs also recently acquired Myo Armband patents from North (formerly known as Thalmic Labs) which is an armband that enables computer input through gesture and motion control of the arm and hands. Can’t wait to see what they have in store for the future of neural interfacing.
Last but not least, another cool piece of technology that was shown was the facial reconstruction done by Facebook Reality Labs. In the video, it shows a prototype headset that when placed on a user’s head, can track all of their facial movements. Now they also did mention that it doesn’t always work this well but it still looks like a major achievement so far. Maybe this will enable even more realistic avatars in VR soon.
In case I missed anything, below is a quick 12 min recap of the Oculus Connect 6 VR event done by CNET.
Overall, OC6 was an amazing experience from both a developer and business perspective. I can see many potentials with the VR and AR space and can’t wait to see what’s in store for the future. Please check our Instagram highlights of OC6 for more news coverage.
