Apple launched iPhone 15 Pro today. Its most important new feature is the ability to capture spatial videos. What are they?
Although the iPhone 15 Pro looks like any other iPhone, it has a groundbreaking new feature: the ability to capture spatial videos. Spatial videos are not just “3D” videos. They are 6DOF videos (i.e., with 6 degrees of freedom).
That means that you can view the videos and be able to move and the perspective will change accordingly, just like the Princess Leia hologram from Star Wars.
What is the difference between spatial video vs 3D?
Is “spatial video” just a fancy name for plain old stereoscopic 3D videos or maybe VR180 videos? No, spatial video is very different from 3D. With 3D videos and VR180 videos, the perspective never changes regardless of how you view them. If you have two sticks, one behind the other, and you took a 3D or VR180 image of them, you’ll never see the stick at the back even if you move your head to one side or the other.
In the GIF animation above, you can see that the guitar’s headstock occludes the lady in some frames but not in the frontal view. Similarly, the lady is to the left or the right of the cliff in the background, depending on the perspective. 3D and VR180 videos cannot capture this change in perspective.
To be clear, this doesn’t mean that you’ll be able to see the back of the head of the subjects in the video above. That’s because the spatial video was captured from only one point. If you move far enough to the side, the spatial video will appear like hollow mannequins chopped vertically. That is why Apple puts a frame around the spatial video, to limit the range of your movement.
How does iPhone 15 Pro capture 6DOF spatial video? What about older iPhones?
In the Apple launch video, they only showed iPhone 15 Pro’s two lenses. Moreover, the two lenses are less than an inch apart, which means they have very little parallax (the stereoscopic 3D effect won’t be very noticeable unless you are shooting macro subjects). How could the two lenses possibly capture 6DOF spatial video?
The answer is a depth sensor. Every iPhone Pro since iPhone 12 Pro has had LiDAR (Light Detection and Ranging), which uses lasers to detect depth. iPhones also have Face ID which use lasers to analyze depth with even more precision. That depth information, combined with the stereoscopic image from the two lenses are enough to create 6DOF video. In fact, there are already apps in the App Store than enable LiDAR-equipped iPhones to capture 6DOF video.
So does that mean other iPhones with LiDAR will also have spatial video? In the demos, we see that the iPhone 15 Pro captures 6DOF videos from several feet away from the subject. My iPhone 13 Pro doesn’t seem capable of capturing 6DOF video from such distances. I believe this means that iPhone 15 Pro’s depth sensors are far more powerful to cover longer distances, besides being possibly more detailed (similar to the Face ID sensors).
For older iPhones, it is technically possible for Apple to enable spatial videos but only for shorter distances such as a couple of feet away, but I think they won’t do it because it will dilute their marketing message, and will make iPhone 15 Pro’s spatial video appear to be less of an innovation.
What’s the catch?
The catch is that you’ll need to use the $3499 Apple Vision Pro headset to see the spatial videos. The Vision Pro headset itself can also capture spatial videos, but people were saying it’s unrealistic to capture spontaneous moments by first donning a headset. Well, the iPhone 15 Pro addresses that issue and you bet they knew that from the beginning. Apple’s plan for world domination is coming together nicely…
Spatial videos will be available for iPhone 15 Pro at the end of this year. But Vision Pro is not coming out until next year, so how would users see the spatial videos? The answer is through AR. You’ll be able to see the spatial videos as an overlay in your room.
What do you think of spatial videos? Are they the next big thing for imaging, or will they be only for tech geeks? Let me know in the comments!