What if you could create a 6DOF 3D environment just by recording a 360 video with a consumer 360 camera? Here are sample 3D models generated from a 360 video that you can explore in VR with Oculus Quest, Quest 2, Oculus Go, or Sketchfab, like being able to walk around inside a photo!
Dr. Maxime Lhuillier from CNRS/Institut Pascal/UCA has been researching videogrammetry software that can convert 360 videos into a 3D environment. To capture his videos, he uses a 360 camera mounted on a helmet. Dr. Lhuillier uses Garmin Virb 360 or a GoPro MAX both of which are consumer 360 cameras (although they both have built-in GPS, Dr. Lhuillier does not use the GPS data). After recording the videos, he uses software that analyzes the videos and converts them into a 3D model.
Unlike very costly LIDAR cameras or conventional photogrammetry that requires thousands of photos, this approach with 360 videos is much faster to capture and uses affordable consumer cameras.
Here’s a demo video of one of the 3D models:
Now you can explore several of these environments yourself in VR with Oculus Quest, Quest 2, or Oculus Go, and yes, they do support 6DOF VR with Oculus Quest. Not only can you move forward, backward or side-to-side, but the perspective will also change if you crouch for example. If you don’t have a VR headset, you can also look at the models on Sketchfab.
How to view the 3D models
To view the models in VR, you will need to sideload them into your Quest, Quest 2, or Oculus Go (see instructions below). Once installed, you can view them on your Quest or Quest 2 by going to your Library and selecting Unknown Sources. The 3D models you installed will be listed there. You can then move around physically or by using the right controller’s joystick. By default, the user moves at the same height as the camera. But if you press the A button on the right controller it will bring up a menu where you can disable the camera constraint. In this mode, the height will change depending on the terrain. But please note that this is more likely to cause motion sickness. Another tip: move slowly to avoid nausea.
Option 1: Sidequest
First, install Sidequest. If you haven’t done so, see Step 3 here. I recommend using the most current version of Sidequest.
On Sidequest, you can click on the browser (globe icon in the title bar) and enter this address: https://sidequestvr.com/user/330664 . Then click on the install button for the model you’d like to explore.
Option 2: Sidequest – APK
If you have an older version of Sidequest, it might not install correctly (mine just kept loading). So here’s another option: download the APK from Maxime Lhuiller’s website, which you can install using ADB (see below) or Sidequest. Just drag the APK into Sidequest, which installs them on your Quest.
Option 3: ADB – APK
If you don’t have Sidequest, you can also install the APK (see option 2) via ADB. Here are instructions for using ADB in my ALVR tutorial.
Option 4: Sketchfab
If you don’t have a VR headset, you can also look at the models in Sketchfab. Here is one of them, with 2.3 million triangles and 1.4 million vertices. Note: Sketchfab is also available in VR and technically you can use the Sketchfab VR app to view the models but the scale will be wrong. Also, a user reported that Sketchfab would crash. If you have a VR headset, I highly recommend downloading the APK instead (options 1, 2, or 3).
This is pretty amazing! Would love if it was turned into an app that we could use to capture our own environments.
Yes I’m hoping that will happen!
Thank you Mic Ty! That’s great news!
FWIW, someone posted on Facebook that they were able to do the same thing and it was “easy”. I invited them to post the 3d model and to sell the software. Hopefully it’s real.
Yes! This! Awesome, well done Maxime Lhuilier, good news for others like me who are exploring 360/3D/XR?
I completed the 3D models and their VR apps (viewers) described this page.
there are three new 3D models: Basalt Canyon, Collonges-la-Rouge, Ruin in a Forest
PC VR headsets are supported (not only standalone Go/Quest/Quest 2): using Windows Mixed Reality or using OpenVR (in SteamVR) or in the SteamVR Workshop
the installation is simpler than before for standalone Quest and Quest 2 thanks to App Lab (developer mode is no longer needed).
teleportation can be used to reduce the risk of VR-sickness
the vertical offset of the VR user is adjustable
a map can be activated to show where is the user in a scene
use joystick and touchpad interchangeably
More details on