Setting up PomfortVL for Foreground Grading

This article covers features which are only available in Livegrade Studio.

Note: This is documentation for features available with Livegrade Studio 6.0 or higher and the PomfortVL plugin (in Beta). Latest beta downloads for the PomfortVL plugin can be found on our virtual production workflow page.

This article documents an example setup for foreground grading in virtual production. Foreground grading uses the concept of manipulating the image of the camera with a LUT box while compensating that grade with an “inverse” grade in the background, i.e., the LED wall. This can be set up with the Pomfort Virtual LUT Box for Unreal Engine, a regular LUT box, and Livegrade Studio.

For the basic setup of the Pomfort Virtual LUT Box for Unreal Engine, please see the article Setting up Pomfort Virtual LUT Box for Unreal Engine.

Hardware Requirements

The necessary hardware setup for foreground grading is as follows:

Hardware setup for foreground grading

Hardware setup for foreground grading

 

Concept

Foreground grading aims to isolate the foreground elements (for example, an actor in the scene) from the background image and color-grade these elements independently. This is not possible directly, so the idea is to compensate the grade of the entire image with the grade of the background.

A LUT box offers the capability to grade the entire captured image (foreground and background), and the PomfortVL plugin introduces the ability to grade the background (i.e., the LED wall). Having access to both grades from Livegrade and linking the two grades, you can achieve the effect of grading the foreground while the background appears to stay unchanged.

Color pipeline for foreground grading

Color pipeline for foreground grading

Example

If the actor in the image should appear “greener” while the background should stay unchanged:

  • the LUT box is grading the entire camera image greener while
  • the grade of the LED wall gets the inverse of the warmer look, i.e. it shows a “pinker” image.
Graded foreground example

Graded foreground example

On the camera monitor, the foreground actor now appears greener (per grade of the LUT box). In the background, the LUT box’s greener look is compensated with the pinker look of the foreground. As a result, the background appears unchanged.

Setup

Unreal Engine:

  • Add the LGVL-GlobalCDL+InvGrade-Actor actor to your scene (instead of any other “global” grade actor).
  • The actor is configured with two LivegradeVL actor components, one for regular grading and one for the compensation grade. Both grades can be controller independently from Livegrade.

Livegrade Studio:

  • Set up three slots.
  • The first slot connects to PomfortVL component with ChannelName “global-cdl”.
  • The second slot connects to PomfortVL component with ChannelName “global-invcdl”.
  • The third slot connects to LUT box (camera monitoring).
  • Link the second and the third slot in the device manager.
  • Setup the third slot so that it has two CDL nodes. One will be used for global (non-compensated) grading with the LUT box, one for compensated (foreground-only) grading. Name the grading nodes accordingly (for instance, “All” for non-compensated and “FG-only” for compensated grading.
  • Make sure the CDL node “FG-only” is shared between the second and the third slot (linking checkbox is on), and the “All” CDL node is not shared.
  • Optionally set up two saturation nodes (one linked and one non-linked) the same way as the CDL nodes.

Now you can use

  • the first slot to grade the background,
  • the “FG-only” node of the second or third slot to grade the foreground, and
  • the “All” node of the third slot to grade the entire camera image.
Device manager and grade controls

Device manager and grade controls

Additional Constraints

  • Only looks with no or just a little clipping work well for inversion. So use moderate grades only for foreground grading. Remember, this is a “blending tool” or a “correction tool”, it doesn’t want to replicate the role of lighting on set.
  • The camera’s exposure and image processing must be chosen so that the camera’s captured dynamic range is (roughly) the same as the displayed dynamic range of the LED wall. This also means maintaining the LED wall’s exposure quite well balanced with the scene’s overall exposure and the relative camera exposure. For instance, if the LED wall shows a series of grey patches from black to white, the camera monitor image also needs to show the same range of grey patches (and not multiple black patches or multiple white patches). This is required so that the inversion of the camera’s grades (in the LUT box) and the look in Unreal Engine (in the virtual LUT box) cancels out.
  • The LED panels should not light the foreground objects so that the foreground objects are affected by the inverse grading applied to the background. If the object is lit by the LED wall only, the inverse look on the LED wall will also change the foreground object’s appearance (acting as the only light source). This compensates the grade of the LUT box and reduces or even cancels the desired effect. The solution is to light the foreground objects with a preferably “constant” light source. That can be achieved by limiting the inverse grading to only a smaller portion of the LED wall like the inner frustum (what the camera sees) or applying the inverse grade to the background but not the ceiling and the frontal LED panels.