Near the end of the visit to SCADFILM’s AnimationFest, CGMagazine was able to gain more insight into the growing field of XR virtual productions. The SCAD Atlanta Digital Media Center also unveiled the university’s second XR stage for the first time—the first was on the Savannah campus. If you have seen the behind-the-scenes ILM video on The Mandalorian with Jon Favreau, then you probably have heard about XR stages/LED XR volumes.
CGMagazine was able to witness the first presentation on the stage, led by SCAD associate dean, Jud Estes, and SCAD associate chair of film and television, Quinlan Orear. Both explained how the SCAD and MEPTIK teams came together to build the stage. MEPTIK was a creative partner that helped clients like SCAD build their LED volume. Their expertise was in immersive environments for virtual and extended reality (XR) production and experiential design, real time content, and projection.
What is the difference between VR, AR and XR?
Co-founder of MEPTIK Studios, Nick Rivero explained the difference between the three Rs of virtual production in an Oz Magazine article really well, “VR: Virtual Reality you put on a headset, changes your perception, AR: Taking the digital and putting it into the physical i.e. Instagram, Pokémon Go, XR: Using the digital world to create entirely new and immersive environments.”
If you are still confused and asking, ‘so what is extended reality (XR) and why should we care about it?’ Imagine a whole film crew and cast having to meet up in Budapest for filming, then transferring all those hundreds of people to the deserts of Jordan or the United Arab Emirates. It can be tiresome on everyone, especially if a director is trying to nail a sunset scene, since sunsets usually last about 45 minutes or less.
With an XR stage, the forefront could still contain set props to match the desert ambience the actors need to stay immersed, but the XR LED volumes are able to capture those desert scenes with unreal background environments displayed on the massive screens with real-life depth. This is all accomplished with the communication of the video game engine Unreal Engine 5, the film camera and white markers placed above and in front of the XR stage called “constellations.” Each piece of equipment provides feedback on what environment is being displayed on the LED panels.
How does this all work? For any technical enthusiasts, I know you are out there: this specific setup at SCAD has a 6K RED Komodo camera with Zeiss lenses, a stYpe RedSpy tracking system and a 15-foot Jimmy Jib—with over 600 LED panels. The stage itself had a 60 x 16 ft curved LED wall with a 1.6 mm pixel pitch and a 38 x 20 ft LED ceiling with a 1.9 mm pixel pitch.
And if you have not guessed which film was shot on location in the desert like this, it was the award-winning film from Denis Villeneuve, Dune. Of course, he wanted to film on location for the realism, but imagine other directors who want to achieve that level of success but on a tighter budget.
XR technology has the ability to change the film and television game by reducing travel time and filming time. The sunset scene could all be focused on a 12-hour shoot continuously versus having to come back to the desert at sunset the next day. With the power of Unreal Engine 5, objects or buildings in the environments displayed on the screens can be changed in real-time as well. So, if a director wants to move an exploding silo for a sick action scene in the next Fast and the Furious film, they could easily do a lot of the scenes on an XR stage.
I have not even mentioned the best part: no more green screens! The environmental projections on the LED panels can be seen in real-time, allowing the actors to hone their craft on their individual performance. The environments themselves could take up to two weeks to create or less, depending on the technicality. Also, the combined equipment allows the environments to be photorealistic in depth to the camera’s eye and the human eye. So, looking off into the snowy mountains would be just as realistic as if the actor was standing at a far distance and looking at the snowy mountains.
This means if the Rebels from Star Wars: The Empire Strikes Back were trying to dodge blaster fire from the faraway AT-TE on Hoth, the Rebel actors could react accordingly. If they were able to use XR stages back then, they would be able to see where the AT-AT walker was and where the blaster shots were coming from in real time. SCAD boasted about how their university programs tailor their students to act with the XR volumes as well because this technology is expected to be the future—as green screens are becoming a thing of the past.
Okay, so film and television are great applications for XR productions. Could we use it for more? The simple answer is yes, we can. At SCADFILM’s Savannah campus, the dean (School of Animation and Motion), Dan Bartlett noted how the students have already worked on a car commercial and students have been working on making music videos, with this technology.
Fashion has also been at the forefront of SCAD’s art programs, so they have already curated fashion shows that use faceless models to showcase students’ fashion lines on a virtual runway. Why do this? Well, it would eliminate the bias of the models. But this would not be taking jobs away from models because the models would still be doing what they do best—just in a motion capture (mocap) suit.
The main takeaway of XR productions would be that it has the potential to work for a lot of many streams of studies or work, and all the environments could be changed in real-time. Orear listed off a bunch of university majors that could use this technology, and it could keep going: “Performing arts, film and television, visual effects, animation, production design, game design, immersive reality, fibers and fashion, industrial design…”
The technology itself required experts in film, TV and video games to create this concept. Thor: Love and Thunder and Obi-Wan Kenobi have been the latest big projects that use XR technology to its fullest, with many more productions continuing to enter XR virtual productions. The time and costs that could be reduced just seem like a no-brainer.
Imagine being sipping piña coladas in a yacht in the Maldives for a minute and fighting aliens on Mars the next. The production possibilities are endless! More and more of these stages are said to be opening all across the U.S. and around the world, but SCAD is the first university to teach everything XR-related to its students in theory and physical practice.
