Teche announced that they are soon releasing an update that will (may?) add realtime optical flow stitching with realtime stabilization to their 360Anywhere 8K 360 camera. They call the feature Mixsteady and if true, it would be the first 360 camera in the world with such a feature. I’m still confirming it. Here’s why this would be revolutionary IF true.
Realtime optical flow stitching with stabilization is the holy grail of 360 video. It promises that you’ll be able to use 360 video immediately, smoothly stitched and fully stabilized, with no postprocessing necessary. Teche says the 360Anywhere will soon be capable of this feature.
Many 360 cameras have stabilization that uses gyroscopic data and rotates the 360 photo or video to mitigate or counteract the movement recorded in the gyrosocpic data. In almost all cases, the stabilization is applied in postprocessing, using a phone or desktop. A handful of 360 cameras have realtime stabilization. They can render a 360 video in-camera with stabilization. The few cameras that are able to do this include Insta360 Pro, Insta360 Pro 2, Pilot Era, and Pilot One. Now Teche claims their 8K 360 camera, the 360Anywhere ($3499) will be able to do realtime stabilization as well (in addition to realtime optical flow stitching). I’m still trying to confirm this with their staff.
Optical flow stitching
Stitching is the process of combining images from two or more lenses to produce a single panoramic image. There are different methods for doing this but the more common way is to stitch with a template with predetermined field of view, pitch, yaw, roll, and other attributes for each lens. The advantage of template-based stitching is that it is faster. The disadvantage of the template-based approach is that it can achieve a perfect stitch at only one distance. At closer or farther distances, there will be stitching errors, although in some cases, these errors can be made invisible with the correct shooting and editing technique. Some cameras are able to change the stitching distance and adjust it continuously. However, even with such cameras, it is not possible to stitch objects at two different distances simultaneously.
By contrast, another form of stitching is optical flow stitching, where the stitching software alters the pixels (e.g. bending lines, etc.) in order to stitch the images. In many cases, optical flow stitching can produce a smoother stitch particularly where there is significant parallax error. Accordingly, optical flow stitching is one of the more commonly used methods for stitching stereoscopic 360 cameras, which have significant parallax by design.
The primary disadvantage of optical flow stitching is that it is usually more processor-intensive. Thus, there are several cameras that can use in-camera optical flow stitching but only for photos such as all the Ricoh Theta cameras and the GoPro MAX. For videos, there are software that can use optical flow stitching, such as Mistika and the discontinued Autopano Video Pro. However, I’m not aware of a 360 camera that uses in-camera optical flow stitching for video in realtime, until now.
Teche claims that the 360Anywhere will soon be able to use realtime optical flow stitching. I don’t know if this is a mistranslation so I’m still confirming it, but if this is literally true, this would be the world’s first 360 camera with such a feature.
Unprecedented resolution but limited frame rate
With cameras that have realtime stitching with stabilization, the resolution tends to be limited. For example, when using realtime stitching mode, Insta360 Pro and Insta360 Pro 2 resolution is limited to 4K (8K in post-stitching mode). Pilot One and Pilot Era can do 8K in realtime stitching mode but only at a framerate of 7fps. For 30fps, Pilot One and Pilot Era’s resolution is limited to 4K (8K with in-camera post-stitching).
A Teche representative said that with Mixsteady, the 360 Anywhere will have a resolution of 8K, which is 4 times the 4k resolution of the aforementioned cameras with realtime stitching and stabilization. However, the Mixsteady frame rate is 20fps, just shy of the 24fps minimum creators and viewers have come to expect. It remains to be seen if 20fps is a usable frame rate for 360 video. We’ll need to see actual samples.
Teche posts a demo
Teche posted a demo of its realtime optical flow stitching with realtime stabilization, which it calls Mixsteady. The demo was shot with a 360Anywhere camera and they used a screen recording from phone app with a live preview that shows that the video is stabilized and stitched smoothly. The video is accessible only to members of the Facebook 360Anywhere group so I can’t share it here.
The demo was of a Teche representative walking through their office with the 360Anywhere apparently on a monopod. As the representative walked around, he pointed the camera up and down. The screen recording was very low resolution (probably because it was just a live preview) and was in equirectangular format, but it did appear to be stitched more smoothly than would be expected for template-based stitching, which seemed to imply that it was either optical flow stitching or at least template-based stitching with continuous adjustment of stitching distance. As for stabilization, there were some vibrations but was still quite good, especially for a professional 360 camera with four lenses (cameras with more lenses are more difficult to stabilize).
Teche said that Mixsteady is expected to be released on October 28. However, that date is subject to change. Meanwhile a couple of weeks ago, they released firmware 0.17.17 with several improvements including in-camera stitching for photos and videos and a DNG10 mode (which seems to be a Raw burst mode).
Should you buy 360 Anywhere?
Whether the 360 Anywhere’s Mixsteady is truly revolutionary depends on the video quality and whether it truly delivers what they are promising. When I tested Teche’s Phiimax, I had difficulty adjusting the parameters on their stitching software to achieve a good stitch, and I don’t have (can’t afford cough cough) a Mistika subscription. If Mixsteady does indeed stitch smoothly, it would overcome what is in my opinion one of the most significant hurdles to using professional 360 cameras — the workflow.
Let me emphasize that this is such an extraordinary claim, that I would strongly recommend waiting for more proof that it really is what they say. Meanwhile, I have requested Teche to send sample videos and to let me test the 360 Anywhere with Mixsteady. I have a meeting with them on or around October 10. I’ll update this post when I find out more.