Immerse Yourself In Immerge

Written By: Jason Ptaszek

Cinematic Virtual Reality (VR) is in the burgeoning years of development.   It will very likely be one of the premiere technologies that changes the movie industry like sound, color and 3d before it.  

Imagine, you’re sitting on your sofa, oculus rift over your eyes, scrolling through NetflixVR to find the latest Batman movie.  You press play.  

You’re sitting in the batmobile.  Batman is driving.  Fast. The buildings are whipping by, you’re trying to control your motion sickness.  The car gets bumped from the side. This doesn’t help.  You turn around on the and see that the Joker, gaping mouthed, in a giant ice cream truck trying to push you off the road.  

You look over at Batman, calm and calculated, you see him push his foot into the gas, he cuts a corner in Gotham, you swing to the side, as the car lunges around the corner.  Not in time, though, the projectile hits the batmobile.  You see an explosion and watch as the ground is pushed away from you as the batmobile is upended.

Batman pushes a button, the seats in the batmobile slide over, you watch as the other layers of the car fall off to the ground as the car rolls and transitions into the batcycle.  You’re on the back now, looking over Batman’s shoulder as he is cutting in and out of traffic.  You’re frantically looking around trying to see where the Joker will strike next.  He is nowhere in sight.  A buzzing noise can be heard. Batman clicks the intercom on the batcycle.  Alfred appears informing Batman that The Joker is launching a giant evil ice cream cone balloon over Gotham. Batman changes direction.

In order to make this VR Batman scene with today’s technology there would need to be an integrated mixture of computer graphics (CG), live action green screen and a lot of rendering.  It would be both time consuming and costly, and unlikely to be considered for a production.  A more realistic way to create this scene would be through a method of recording both the actors and environments in real time. This way data can be collected to create realistic stories with limited rendering to save studios time and money.

Enter: Lytro, a Light Field imaging company who recently announced a new piece of technology called Immerge to bring my Batman scene to reality, virtual that is.   Immerge captures real time Light Field information for VR to more easily integrate real sets and real people to provide a viewer the experience of riding along with Batman as he’s being chased by a crazed lunatic in an ice cream truck, in the comfort of your pajamas.  

So, you may be asking yourself at this point, how will light field imaging allow you to ride in the batmobile?  The Immerge captures and calculates the volume, position and intensity of light in a room.   It is an orb-like device with a 360-degree field of view. This means it can record data of the entire room without needing to be repositioned, allowing the viewer to travel in the room without needing to capture additional information for other angles.  By using this Light Field technology it can calculate the placement of objects on a set to create high quality digital renders of the sets in real time.

So is this video? Not exactly.  It shares similarities to video, after all cameras collect light on a sensor and play back a representation of what that sensor saw; a 2d, flat representation.   What the Immerge is able to do with Light Field is actually capture the entire volume of light in a room, allowing the viewer to step into a 3d representation of a room, with the ability to look left, right, up and down and view objects at different angles that were not intended for the audience.

Since the device captures a full view of the room there are a few things that come to mind that would be beneficial to some departments, and others needing creative solutions:

Pros:

• Equipment cannot be on set or they’ll be seen, for actors this means they can work without concern for tripping over cables or stands.
• In Order to keep crew members out of the scene the device can be controlled behind the scenes with an app on an iPad
• Scenes are more immersive to audiences
• The creation of an setting is faster and more realistic than current technologies.
• Could be used to comp characters into a scene rather than flying an entire crew to a location.

Cons:

• Traditional shots, focus marks and camera angles become obsolete.
• Fourth Wall is broken, Set Dressing and Props will need to dress entire rooms.
• Lighting cannot be on the stage, equipment will need to be better hidden.
• Boom Operator will need to be creative to collect sound.
• Footage may need some roto cleaning  by VFX.
• Expensive hardware.
• VR could lead to motion sickness and stresses our brains are not designed for.

Unanswered questions:

As with all new devices there are many questions left to be tested, for example:

• Can the device be strapped to a moving vehicle?  
• Does the device need a basic anchor point in order to collect the data?  
• How realistic is the data?
• Is there a lossless point where the camera is so far away from an object that it no longer can collect the data on the object?  
• What is the quality of the data that is collected?
• How stable is the device from a production standpoint?
• Can it shoot in a very hot environment?  How long can it run before it overheats?   
• Cold environment?

Although I cannot foresee this device completely taking over traditional filmmaking, I do think studios can use this device in other ways besides VR..  For example, it could be used to capture data from a historic building that a crew wouldn’t otherwise get access to.  You could capture the information inside the building, then shoot your characters on a studio stage to composite into the environment you recorded with the Immerge. In this way you’re able to save money by only sending a skeleton crew out to the historic location, and you get the ability to create your camera moves in post.  This means smaller crews with less equipment for b-unit work.   

In whatever way this plays out, it will be interesting to see how Immerge will help studios tell better, immersive stories, and in the end make some money. Which means riding in the Batmobile could happen sooner rather than later. 

• For Further Reading Visit: Lytro Refocuses to Create a Groundbreaking VR Camera