For almost a century, film productions have been using green screens to shoot movies in order to replace the background and make the movie seem like it was shot in another location. This sparked a revolution in the industry. Although it has been a radical technology, actors had to really push themselves to the limit to get into the character while surrounded by green walls, rather than being in the actual location. During the pandemic, a new technology started to be used in the VFX industry: the LED screens as a substitute for greens screens. This advancement helps production pre-visualize what the shot would look like and filmmakers are now able to capture both live action and CGI together once in one footage. The Mandalorian is one of the first major productions to choose LED walls over green screens, since before they had only been used in concerts.
The team production for the Mandalorian has developed a new real-time surrounding for the production of the movie, using enormous LED screens and real-time game engines. With Unreal Engine, the CGI world is rendered in real time, so the assets can be altered at any time. This allowed producers to change the set live while filming, to correct the perspective in 3D imagery, and to have pixel-accurate tracking. It also had real-time synchronization with camera movement, which made the imagery move through the panels adjusting to the camera’s field of view.
The new way of filming solves the biggest constraint in using green screens: lighting. Lightning is one of the key benefits in using the LED virtual production sets. The light coming from the LED provides real reflections and color in real props and actors. In the Mandalorian show, the main character would always use a reflective costume, so they needed the background to provide the correct reflections and highlights. When filming the old fashioned way, the green screens result in color spills that sometimes are hard to fix.
“The Volume” is the name given to the combination of LED panels used in the show. It was a circular set with a 75 feet diameter, and also had a roof composed of LED. The renders used for the backgrounds had to have high definition imagery, and compose all of the almost 360-degrees stage. In order to hide the corner point at the bottom of the screens where the panels would meet the floor, real props were placed in some of the ground areas, while other sets were built with raised ground planes. They also added grass and rocks to blend the transition from the real set to the virtual one.
All of the shots had both practical and visual sets, sometimes having part of an object, real and part of it digital. An example of this was when they shot some of the ships. The most challenging aspect was to match the colour in both sets. As the real set was lit by the LED screens, adjusting the digital colour would affect the light being provided. So, if the colour did not match, they would have to adjust the colour in the practical props, sometimes by re-painting the whole object.
The major obstacle in filming with LED screens was to reduce the moire patterns. The shots had to be shot with a shallow depth of field, so the background would be slightly blurred. This could easily be done with the correct choice of lens and light levels. The intent in this production was never to film overly sharp imagery, the director wanted the look they had when filming the first star wars decades ago.
This new technology is not yet developed to substitute the green screens forever. Even the Mandalorian used a version of green screens, provided by the LED screens. But rather than using the old fashioned way in the whole set, choose pinpointed areas to position while the rest is surrounded by LED screens. In that way, one can get the best of both worlds.