The day of the greenscreens is over. Virtual production is the new frontier.
The way the virtual set design changes. Instead of using green screens and motion capture suits to completely film and create CG worlds, there is a new way of creating with the Unreal Engine that is revolutionary. In fact, virtual environments are ready to go on filming day.
By now we all have the behind-the-scenes photos from. seen The Mandalorian and how their stage is a huge curved screen showing the surroundings. These large screens create backgrounds that are rendered with photo-based textures and realistic lighting that connects the virtual world and the real world.
Team members from Happy Mushroom, the VFX company that was being worked on The Mandalorian, sat down with the Corridor Crew for an episode of VFX artists respond to talk about how the virtual collection came about and how it represents the future of stage design.
We would like to highlight a few points from their discussion!
Virtual production after Lucky mushroom
Happy Mushroom started using game engines as their final image in 2016 The jungle Book and put the same method in. away The Mandalorian. They used the same huge screen but no stage and a team of 12 designers to create the entire movie or show.
The reason they use these giant screens to showcase the virtual environments is because they give off the practical lighting that the DPs want. Conventional lighting used by DPs is still recommended on sets like these as it is difficult to fake shadows and lights. The virtual team will go inside during post production and add these lights to the scenes to make the surroundings look as natural as possible.
The process of creating the environments begins with a kickoff meeting where the DP and director sit down with the virtual design team and the set design team to discuss what their vision of each scene is. This process includes discussing the mood for the scenes and showing the concept art. After this first meeting, both teams go out and start building the sets.
In contrast to scouting in the real world, where a small team finds out when the right day is to shoot and where to shoot, the unreal world is designed by VR headsets and everything is designed according to the wishes of the DP. Sometimes the virtual team would take photos of boxes that the set design team had created and base the color scheme and lighting on what it looked like with those boxes.
For the second episode of The Mandalorian In Season 1, the design team created large stones that were used as a handy hiding place for the camera while filming. The virtual design team scanned these rocks and created a complete environment that perfectly matches the model rocks. The results are amazing.
Instead of having to go into post-production and create shadows, those shadows are already there. Any reflections, kicked up dirt, or really anything that can realistically be captured were made in the real world.
The goal of virtual world construction is that you can record on the camera and not have to mess around with the footage afterwards. Best of all, the luxury of making small changes is accessible. If the red boxes in the background are too annoying, the virtual team can go in and change the color of the boxes as they wish.
Being able to watch something in real time is a huge luxury that many people don’t have when filming something on a green screen. Nothing is in motion and the whole piece risks feeling too wrong.
The goal of creating these virtual sets is to do as little work as possible for the post office. Everything is ready to go, and you don’t have to spend weeks trying to get something going with agile actors. Yes, you are filming something with an entirely man-made background, but that doesn’t mean you have to ditch all of the best filmmaking practices that have been developed for 100 years.
Virtual production alone
If you don’t have the luxury of working with giant LED screens projecting your world, take a look Ian Hubert. He’s just released a full CG production that he’d been doing himself for the past three years. He relied on photorealistic copies to make the surroundings appear as real as possible, matched the lighting perfectly to his actress and the buildings in his handcrafted city, and did it all on a tiny stage.
The magic lies in Hubert’s ability to make something of the photo scans and photo-based textures. Rather than rendering an image from scratch, he had the tools to create what he wanted without putting the extra stress and time into his project. He says the hardest part of his entire project was using practical things while filming, and that’s why his project took a little over three years to create.
Finding that balance between practical and unreal is vital at this point in filmmaking. Unreal is perfect for building out of this world environments, but practical effects ground that out of this world experience into a reality that people can understand and relate to. All it takes is a lot of time in a VR headset and hit the record on the camera.
Let us know what you think of this technology in the comments below!