Apple claimed that spatial videos would be incredibly immersive and would enable you to relive cherished moments “like never before.” But I tried out Vision Pro and found that Apple lied about spatial video. Here’s my analysis from my perspective as a 3D enthusiast for almost 30 years.
One of the key features Apple touted for the Vision Pro was the ability to capture “spatial videos.” Apple was somewhat vague about what spatial videos were, and instead described it as a new way to capture memories that would enable you to relive them as if you were in that same moment.
To emphasize this, Apple showed several supposed demos of spatial video in its presentations. In these demos, you could see the camera panning behind the Vision Pro user watching a spatial video. Crucially, as the camera panned, you could see the background of the spatial video moving independently from the subject, as though they were on different planes.
This effect is called parallax and is an effect that traditional stereoscopic 3D photos and 3D videos cannot show. If you view a traditional 3D photo of an object, moving your head side to side will not change the perspective. The subject will not occlude a different part of the background nor will any other part of the background become visible no matter how you move your head. But Apple’s spatial video “demos” showed that you would indeed be able to see a different perspective as you move your head while looking at a spatial video. That would be revolutionary and something that would make 3D images appear far more realistic, giving credence to Apple’s lofty claims about spatial videos.
I was fortunate enough to get a demo of the Vision Pro on its launch day, and I was curious about spatial video more than any other feature. When the demo got to the part about spatial video, I immediately tested whether it really did show a parallax effect. I stood up, bent down and moved my head side to side. Unfortunately, nothing I did changed the perspective of the spatial video. In fact, I did not observe any difference at all between “spatial videos” and 3D videos that everyone is familiar with.
This was shocking to me because of the way Apple had described spatial videos and how they tried to make it sound like it was truly innovative. I was also extremely disappointed that literally none of the tech reviewers who got an early look at the Vision Pro realized that so-called spatial videos are exactly the same as regular 3D videos. Did they all buy into Apple’s lie, or are they just incredibly ignorant?
Just to clarify, I’m not an Apple hater. In our family, we all use iPhones and all our tablets are iPads. In fact, I’ve had Apple shares for a long time and it comprises more than 10% of my stock portfolio. But this still made me angry.
Parallax from shadow box
What about the shadow box frame effect? When you view spatial videos on a Vision Pro, they appear behind a blurry frame that is somewhat like a blurred shadow box. The shadow box floats in front of the spatial video and if you move your head, yes you can see a parallax effect between the shadow box and the spatial video. However, this does not mean the spatial video itself has parallax. In fact, this shadow box effect is also visible when you view 2D panoramas shot on the iPhone and viewed on the Vision Pro in immersive mode. No one is claiming that these 2D panoramas have a parallax effect, therefore for a similar reason, adding a shadow box effect to spatial videos doesn’t mean they have parallax.
Other issues
There are other ways that Apple misled customers about spatial video. For example, they claimed that only an iPhone 15 Pro was capable of capturing spatial videos because its lenses were reconfigured. However, even two years ago, there was already an app called i3DMovieCam that enabled some iPhones to capture 3D videos using two lenses.
This is not to say that Vision Pro is total junk. In some ways, it is better than any other VR headset I’ve tried (I have several), which I’ll discuss next time. But spatial video is not at all as revolutionary as Apple would want you to believe. If you want to capture 3D photos and videos, you can use the aforementioned i3DMovieCam or i3DPhotoCam. You can also use a Quest 3 to capture 3D photos and videos. Better yet, you can get a dedicated 3D camera such as the Qoocam EGO (reviewed here). If you want to continue using iPhone 15 Pro to capture spatial video, there’s also good news: Meta announced that the next update for the Quest (v62) will add the ability to view Apple spatial videos to the Quest 2, Quest 3 and Quest Pro.
Hi Mic. I’m curious. Your article doesn’t mention Apple’s response. What did they say when you asked them about this apparent discrepancy between what their demo video showed and what you observed?
Hi Dean. Unfortunately, I don’t have any contacts at Apple, and the representative for the demo was also not knowledgeable about 3D videos in general.
shame on you Apple
I truely apreciate your honest words. It is not about to crap the Vision Pro. But you review is a big statement to open people eyes; do not fall into false marketing words.
Apple promised, but Apple did not delivered. They truely lied, while not telling the full truth.
I am missing more guys like you, to tell what is good, what is bad and what is not real as marketing tells.
Your experience allows us to get to the truth.
The bad, there are only influencer reviews with non experience and for them, even an apple is paradise garden. But you show, that there are more fruits beyond Apple’s spatial limits.
Thanks grestguy. I wanted to put this out there so that people aren’t misled into thinking spatial video is anything it isn’t.
Hi Mic,
This proves, once again, that Apple is the best know example of Stockholm syndrome… 😀
lol. I’m getting comments from people who bought it and say I’m wrong about it. Oh well I’m glad they are enjoying their purchase.
The feature is basically the old View Master technique in video form. They are calling their headset “virtual computing” but its actually a virtual room\office with changeable scenery. Quest 3 has those features. Apple is attempting to introduce VR to people who never used VR and rename it something else.
I don’t mind that they called it ‘spatial computing’ because they didn’t show anything that VR couldn’t actually do. Everything they showed is possible on Vision Pro… EXCEPT the fake parallax with spatial video.
It’s unfair such a high price tag for the first version of a device that has its limitations like weight, battery, storage, …etc.
So how much price would jump in future???🤔. 10 K?????
Be smart and never waste your money 😇.
Wait for two years and many competitors will unveil their products.
Mic, thanks for your super honest talk. Turns out that so called “Apple legendary design” (when it comes to ergonomics, ease of use and aesthetics), is practically exactly what that phrase states … it’s just a legend. I can’t believe how Apple could screw up so many things in its VisionPro headset. Ergonomics and user experience with AVP is already painfully unpleasant (too front-heavy, too tight, fingers pinching too annoying/tiring when done for a longer time). Now it also turns out that “spatial” video on it is neither spatial nor special … it’s just plain old narrow field of view 3D (and not even an immersive VR180 version of 3D). And to add insult to injury, the “spatial” 3D content recorded with iPhoney15 series phones not only has a very week depth (due to very narrow distance between two camera lenses), but also is unpleasant to watch, because on most recordings there is a noticeable focus mismatch between Left and Right images (due to very different Focal Length optics in those two cameras lenses used for “spatial” capture). Such a focus mismatch makes it uncomfortable to watch (feels like one of your eyes suddenly needs corrective glasses to see clearly). This focus mismatch is easier to ignore when watching videos (moving content doesn’t give us enough time to explore image details). But on “spatial” photos recorded by iPhoney15 phones there are always some sections of those photos that have either close or far subjects with significantly different fuzziness levels on Left and Right screens.
I assume that “spatial video” since it was on the new iPhone was simply 3D with the lenses CLOSE rather than emulating the distance between your eyes. One of the new teased Canon 3D lenses has the same tactic – the lenses are very close to each other, This reduces the amount of crossing your eyes so the backgrounds are not duplicated. It is the best way do do close up 3D otherwise the distance between the close subject and the background causes problems – like headaches.
Basically all the 3D cameras we are currently offered are WRONG. I have the JVC 3D camcorder and its lenses are closer together. It produces better results. It also allows you to “toe in” the lenses to point at the closer subject which none of the current cameras offer.
use “spatial video” for closeup 3D and if you use green screen you can composite the close 3D with a regular 3D background.
Just remember increasing the size of the frame pushes those things in that frame back in 3D space,
We need a good high resolution 3D camera for close up work to pair with the current cameras, but the iPhone isn’t that.
The question here for me is how to get 360, 180 stereoscopic and 360 stereoscopic still and video that I have shot onto the Vision pro. No one seems to be talking about that surprisingly and I wonder why. Most people seem to be focussing on the shortcomings of the device. Did anyone see the immersive video clips they supplied? Best and sharpest immersive 180 stereo I’ve ever seen.
Lots of discussion about it. The VR apps vendors are working on it. I tested immersive playback of 3D VR360 videos successfully using a custom app I created. It looks amazing on the Vision Pro.
Hi Dean, Discussion where? I’d be interested to know. Also about your app. Bryan
I would also be interested to know. It’s curious that Apple didn’t make it easy to implement 180/360 stereoscopic in Xcode. I would think they would want people to be able to show off the abilities of this thing in interesting ways. Right now, the only people who can do it are, well.. Apple.
In some of the Vision Pro groups on Facebook. I created my app using Xcode and Swift. My main purposes for this were to see the 2D/3D image quality, first-hand, so I could make a purchase decision, see what video format was required, and see what was available in the Apple APIs to display 3D. It’s really unfortunate that Apple hasn’t provided immersive VR360 playback in their apps. Hopefully, that will change. I had fears it might require some low-level access to the core APIs. Fortunately, that was not the case.
HI Dean, Could you be specific about what Facebook group you are referencing and that you are part of? I really would like to find out how to view the videos and stills made with the Canon 180 vr set up on the Vision Pro. Thanks, Bryan
I’m in several groups. I think the main discussion was in two of them. One was this one. I can’t remember the name of the other(s). If you search Facebook you should be able to find them.
https://www.facebook.com/groups/1692265674578109
“spatial video “demos” showed that you would indeed be able to see a different perspective as you move your head while looking at a spatial video. That would be revolutionary and something that would make 3D images appear far more realistic, giving credence to Apple’s lofty claims about spatial videos.”
I must be missing something, but what can be more immersive than a (high quality) VR180 (watched with a headset with a large FOV)? It comes so close to reality: it’s 3D and you can look around as if you’re actually sitting or standing there.
So if Apple would have delivered, it would have been a VR180+parallax, and this would be even more immersive, is that correct?
Hi Kris. VR180 looks very realistic as long as your head only rotates but doesn’t move. The moment your head changes perspective on the X, Y, or Z axis, then the you’ll see the VR180 image moves along with your head, breaking the illusion of object permanence. By contrast, there are volumetric videos that will change perspective as the viewer moves their head. At the very low end, the range of allowed movement is very small but at the very high end of this type of video, you can walk around the video, as if it were a VR video game.
Thanks for the review.
I tend to agree that anyone who has been fascinated and immersed themselves (filmed, edited, viewed etc) for awhile, in recent years, in 360 imagery/video and 3D stereoscopic imagery/video will recognize it as exactly what you see when using the AVP. Done very well. And, of course, then on top of that floating menus appear (which do have some small parallax effect).
I imagine once we have cameras/recording systems that are able to affordably capture the parallax effect which Mic understandably longs for, then it will be available for viewing on AVP and other systems.
I predict Apple expects that 360 & 3D video, plus Augmented Reality, plus iOS apps, and the ability to pin screens at varying spatial locations, etc, delivered in incredibly high clear resolution, in the short term will wow almost everyone who has not already been obsessed with 360 video etc, and in the long term could deliver a new level of content which will bring an entirely new and much larger audience to this world many who frequent this site have been a part of for quite awhile.
I think this first gen AVP and first year are primarily for app and content creators who want to jump on board early hoping to monetize their work in the very early days, like those who built the very first apps for the iPhone.
Moon player Apple Vision Pro app supports playing half/full sbs top/bottom 3D files and also VR180/360 files up to 12k.
Spatial videos are in MV-HEVC format. SBS videos can be converted to it and then played natively though I do not think there is any advantage to doing this. Apples immersive videos (VR180) are also in MV-HEVC format and have Spatial Audio added. The immersive videos in the Apple TV+ app were shot in VR180 8k 90fps supposedly using custom rigs. Would be cool if Mic could find out more information on their rig
The early testing I did with a native custom player showed that MV-HEVC videos could be played back at resolutions and bitrates higher than SBS and OU formats.
Yes- but is your player easily downloadable? Ease of use is important.