Insta360 One RS special deal
360 Camera News and Info VR News and Info

Facebook will enable you to step inside your photos and videos – see the demo!

Facebook 3D reconstruction from photos and videos
Facebook 3D reconstruction from photos and videos

Facebook will soon enable you to step inside your photos and videos.  I was lucky enough to see a demo of this technology at IVRPA 2018. Guess what? It really works.

With this technology, Facebook can analyze the photos and videos you’ve taken and use computer vision to reconstruct a 3D model of the place where the photos and videos were taken. You can see it discussed starting at around 1 hr 47 mins of this video of the F8 keynote day 1:

I got to try it out for myself in May, at IVRPA 2018 in Tokyo where I met some representatives from Facebook’s Immersive Media group.  Here’s a video of me trying it out (the video is from a segment in my Samsung Gear 360 review video — please watch the whole video):

Although the video wasn’t super detailed, the critical thing was that the proportions of the objects in the 3D model were very realistic and 1:1 scale.  I could really get a sense of the place.  As I moved around, I could see Photos suspended in the air.  When I click on them, the photos would be enlarged so I could see them more clearly.

Their 3D reconstruction software was smart enough to recognize which objects did not belong in the room and remove them.  For example, if you took photos with people, the software could recognize that the people are not a part of the room, so the 3D model won’t include the people.

Crucially for us, the Facebook reps said that 360 videos are the ideal medium from which to generate a 3D model.  Facebook will be able to create a 3D model from 360 videos much more easily than regular photos or videos.  With only a few seconds of a 360 video, you’ll be able to get a 3D model (compare to this Institut Pascal videogrammetry demo).  This means 360 cameras will become even more indispensable for preserving memories.


About the author

Mic Ty


Click here to post a comment

Leave a Reply to Mic Ty Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

    • Hi Hans! It is in the video I linked, around 12:25 of the video. If you mean an actual VR demo that you can try yourself with an Oculus Rift, sorry I don’t have that. It’s not public yet.
      Best regards,