Home / Studio Systems / Episode 13
Studio Systems · Episode 13 · 2:57
In immersive video, there is no frame.
How shooting for Vision Pro and other headsets changes production: spatial vs 360 vs Apple Immersive, 180-degree stereo cameras and IPD, 8K per eye at 90 fps, foveated rendering, vergence-accommodation conflict, 3DoF, world-locked graphics, and live immersive sports.
Key takeaways
- Spatial video is 3D in a window and can be shot on iPhone or Vision Pro. 360 puts you in the middle. Apple Immersive is 180-degree 3D with spatial audio and needs a pro immersive camera.
- 180 beats 360 for most shows: every pixel goes where people look, and the crew has somewhere to hide. If you can see the lens, the audience can see you.
- The camera is two lenses spaced like your eyes (IPD). Blackmagic URSA Cine Immersive shoots 8160 x 7200 per eye at 90 fps across 180 degrees.
- Headsets use eye tracking for foveated rendering, but recorded video gets no shortcut: the viewer can look anywhere, so every pixel has to be there.
- The camera is the audience's head: eye height, people out of arm's reach to avoid vergence-accommodation conflict, and no whip pans or zooms.
- Immersive video is 3DoF: turning works, leaning in does not change parallax. Direct attention with light, sound and movement, cut less, and match the eye trace.
- Hide lights in the set, keep graphics world-locked rather than head-locked, and yes, it works live: Lakers games streamed in Apple Immersive.
Chapters
- 0:00The hardest frame there is
- 0:15Spatial video vs 360 vs Apple Immersive
- 0:34Why 180 degrees instead of 360
- 0:46If you can see the lens, the audience can see you
- 0:52The camera: stereo lenses, IPD, 8K per eye at 90 fps
- 1:09Foveated rendering, and why video gets no shortcut
- 1:28The camera is the audience's head (vergence-accommodation conflict)
- 1:50Don't pan left
- 2:013DoF, parallax, and directing without a frame
- 2:19Lighting, world-locked graphics, and going live
- 2:41The camera is a person
Read the full transcript
This is the hardest frame I've ever shot for, because there isn't one. I've worked on Apple launch productions including Vision Pro, and immersive video breaks almost every rule TV taught me. Let's talk about shooting for headsets. First, three formats. Spatial video is 3D in a window, like a memory you can lean into, and your iPhone can shoot it. 360 puts you in the middle of everything. And Apple Immersive is 180 degrees of 3D wrapped around you with spatial audio.
Why 180 instead of 360? Because people mostly look forward, putting every pixel in front of you makes it sharper, and it gives the crew somewhere to hide: behind the camera. Mostly, in 180, if you can see the lens, the audience can see you. The camera is two lenses spaced like your eyes, matched to the average IPD, the interpupillary distance. Blackmagic's URSA Cine Immersive shoots about 8K per eye at 90 frames per second.
That sounds like overkill until you stretch it across half a sphere. And here's why it matters. Headsets use eye tracking for foveated rendering, full detail only where you're looking, less in your periphery. Video doesn't get that shortcut. The viewer can look anywhere, so every pixel has to be there. And 90 frames keeps motion smooth, so nobody gets sick. Which brings us to the big rule. The camera is the audience's head.
Put it at eye height, keep people out of arm's reach, or you trigger vergence accommodation conflict. Your eyes cross for something close while focus stays put. And that's eye strain. And never whip pan or zoom, because when the camera moves, the viewer's stomach moves with it. Ready camera one... and PAN LEFT! Don't pan left. Immersive video is three degrees of freedom. You can turn your head, but leaning in won't change the parallax.
There's no frame to point people. So you direct with light, sound and movement, and you cut less. Hold shots longer and match the eye trace, so every cut lands where they're already looking. Lighting and graphics change too. Any light in front of the lens is in the shot, so you hide lights in the set. Graphics should be world locked, not head locked, or they follow your face around. And yes, it works live. Lakers games streamed to Vision Pro in Apple Immersive last season on live versions of these cameras.
So when you shoot immersive, remember, the camera is a person. Put it somewhere worth standing, don't move it without a reason, and get your crew out of the shot, especially you, craft services.
About the host
Tom "Lucky" McNulty is a broadcast technical director and studio engineer with 20+ years building and running studios, control rooms and live shows: Big Brother and Hell's Kitchen, studio services at Bunim/Murray Productions, Apple launch productions, and festival VJ and IMAG work including Daft Punk's 2006 Coachella set. Studio consulting · Portfolio
More Studio Systems episodes
Before You Buy a Camera: 5 Questions That Shape a StudioEpisode 1 · 3:35
Livestream Checklist: 3 Checks Before You Go LiveEpisode 2 · 1:09
Building a Studio on a Budget: 3 Things That Ruin the ShowEpisode 3 · 1:29
ATEM vs TriCaster: Which Switcher, and Where NDI FitsEpisode 4 · 1:26
Game Day Inside an OB Truck: Who Does What When It's Truly LiveEpisode 5 · 2:18
Festival IMAG Explained: Cameras, Lumens, LED Pixel Pitch & CommsEpisode 6 · 3:50
Studio Lighting vs Concert Lighting: Light for the Lens or the Room?Episode 7 · 1:59
How to Go Viral (According to the Internet)Episode 8 · 0:56
Bells & Lights: How Studios Keep Monsters Off SetEpisode 9 · 1:05
How to Use AI in Video Production: Generated vs Hybrid vs VFXEpisode 10 · 2:23
How Concert Tours Really Work (and Why Bands Ban Brown M&Ms)Episode 11 · 2:45
Control Room Design: Who Sits Where, Monitors, Comms & the CAREpisode 12 · 2:58