What is different about traditional and virtual productions?
This is part two of four in this series of articles that look at virtual production (you can read part one here). In this section we are going to look at the differences in working on a virtual production that uses LED bulk film versus a traditional production that uses chroma keying.
So you are entering a production facility that wants to use virtual production tools in their workflow instead of a traditional approach. This can mean a few different things to your preproduction and production processes. In this article, I’m going to explore how to prepare for the differences in your recording process when working on a stage with volumetric LED displays, as opposed to having composite chroma keying in the background.
LED volume displays
When you need a quick update, recording with an LED volume display means using LED panels as a backlight source that reproduces a “final pixel” image in the background of your recording, rather than piecing the background together on a green or blue screen later . It does this by tracking your camera’s movement in 3D space and encoding that information into a game engine and rendering your background environment so that you can achieve parallax (objects moving at some distance relative to each other).
This is an upgrade from standard rear projection and translites where the background wouldn’t move with this camera and the illusion could easily be ruined due to the lack of parallax.
Where do the differences begin?
It is important to note exactly where to start thinking about this methodology differently than in a traditional production environment. And that really starts in the early pre-production phase when key creative members of the team like the producer, director, cameraman and production designer begin to break down the locations and logistics for executing the script and creative vision.
I’ll give you an example using a story that could include virtual production techniques:
A young child finds a magical gem that transports them to an alien planet occupied by people who appear less than normal.
In the breakdown of the scenes and locations in the story, something stands out: “How do we do the alien planet?” There are many different options available to the discerning filmmaker, such as: B. creating the entire location on a sound stage, traveling to a remote part of the world that artistically fits, or creating the entire environment in CG and assembling the actors with the Chroma button.
Now there’s one more option that keeps things local and makes for an even more immersive shooting experience: Recording on an LED volume table.
After we have decided on an LED volume level for our film, the differences between the flows of virtual and traditional production begin.
For starters, such an environment must be present in a virtual production, instead of the conventional workflow in a chroma key phase, in which a pre-vis environment is built up from concept drawings and should be completed during or after the main photography is completed , approved and created before the camera rolls on it. This leads to more preproduction, pre-briefing, and prep briefing.
Here a very important person has to enter the picture –the VFX manufacturer or VFX supervisor. This is a title that I see may change as this new technology and its workflows become standardized. The important thing now, however, is what responsibility the VFX manufacturer bears.
You need to:
- Make sure the VFX production schedule is established and communicated to all departments so that the main photography begins on time.
- Make sure the environmental artists and lighting team speak to the director, cameraman, production designer, and producers, and that the entire creative team approves the digital environment to be created.
- Make sure the art department is in contact with the environmental artists and Virtual Art Department to ensure that all physical assets present on the stage are also available for different camera settings in the virtual environment.
- Make sure the director and cameraman have had the opportunity to explore the digital locations in advance – ideally in a VR setup for complete immersion – and that they have planned their blocking and camera settings to reflect possible changes in the environment .
As we go into production, we need to set some important things for our schedule.
- What environments are ready and available to take photos first?
- What size of LED volume do we need to achieve the recordings we have planned?
- Which recordings are not possible at an LED volume and require a chroma key background and later compositing?
Now each key role is going to experience a difference in the way they are used to work. I’ll cover some role-specific changes for department heads in the following articles, but here and now I want to introduce you to some key differences in the way the day will go on a virtual production shoot versus a traditional on-site shoot .
Setting the scene
The first thing that needs to be addressed is making the environment work with the correct orientation and size on stage. Depending on the system and the complexity of the scene, this could take anywhere from five minutes to an hour or more in a slower computing environment.
In an ideal world, you are now working on your storyboards / shot lists / pre-vis so you know which direction you will be facing on your first setup. The best way to ensure that this is communicated to the stage technicians is to make sure they have a copy of the recording list as well as the saved positions of your camera positions within the motor you are using.
Next, the camera must be aimed at its marker and the actors must be blocked accordingly. Then all important set pieces should be placed in the foreground to ensure that the size and scale match those in the background. Once all of this is done, it’s best to run the sequence once while the camera and cast are moving to see if they run into any of the following issues:
- The environment or the performers who lock the trackers on the camera.
- The entire scene blocks leaving the lighting environment of the set.
- Induce moire by getting the camera too close to the wall and focusing the panels.
It’s not uncommon for some things to need to be changed to accommodate the shot, such as:
- Rotation and orientation of sky and sun
- Shadow intensity and consistency in the background environment
- The placement of the scene within the virtual location
- Object adjustments or removal in the middle of the engine if they don’t “fit”.
Once all of these tests have been passed, the lighting can be completed as well as the final adjustment of the set.
It’s a good idea to watch the replay during rehearsal or to check first that there aren’t any technical issues that you didn’t notice before, such as:
- Background shifting or shaking due to a tracking problem
- Color differences in the LED panels (usually along the seams in the wall)
When the shot was clean and everyone is happy, you can go ahead and turn around. This is where things get interesting.
“Turning around” in volume means that the location within the virtual environment moves according to the new orientation of the camera within the scene. In contrast to conventional recordings, however, the camera and lighting do not move much. This is because you can digitally set your new scene background anywhere you want. So all you have to do is readjust it on stage to customize it and you’re good to go.
This really is the most exciting part of working in virtual production environments for any enterprising filmmaker. Because if everything went well during your production and you had little or no problems, the post production process can be very streamlined and predictable by removing much of the “fix it in post” mentality that would otherwise permeate a VFX – heavy production . The few things to keep in mind are:
- Check the largest possible monitor for moiré problems and clean them up
- Adjusting color shift issues between panels in senior class
Otherwise, you’re done – print it out and you’re good to go.
What’s next? Keep learning
For filmmakers shooting on volumetric stages, things are different and new tools are available to you. Find out what cameramen need to know to get started in virtual production. If you need a primer, we’ll cover the basics of virtual manufacturing.