Tutorial – Light fog in After Effects – Part 1

I’ve had a lot of fun tinkering around with various tricks in After Effects to see how I could integrate them into my 3D renders. One visual effect I was trying to work with recently was light fog: the idea was to have a stationary prop placed in front of a bright light, only visible by its silhouette, with light fog casing around its shape. I know how to do this in Maya, however I wanted to do it in post due as tweaking the effect in the 3D software to get the desired look can require a lot of guesswork and be slow to render. The basic look is easy enough to accomplish in post with the CC Light Burst effect, except that although the prop itself was stationary, the camera itself moves in the Maya sequence: the center point of the light burst would have to move in 3D space, and CC Light Burst is a 2D effect. I was eventually able to get this work by taking locator and camera data from Maya, tracking that locator data, and then transferring the 2D coordinates from the tracker to the CC Light Burst effect.

Confused yet? I’ll start from the beginning.

This tutorial is split into two parts: the first part will focus on rendering and compositing your Maya scene for the light fog effect described above, and the second part will walk through how to take the 3D placement data from your Maya scene and track it using After Effects.

First, you need a Maya scene. I’ve provided a very basic scene with a cube, a bright mental ray area light, a polygonal ground plane, and a camera with animation keys. (Figure 1). I’ve also set up the proper render layers for this tutorial, though to understand the workflow I’d recommend deleting the render layers and following along. This workflow should work with any scene, but be sure you have the following as they’ll be needed for tracking in After Effects:

• A camera with animation keys
• A locator point-constrained to the light source (the point constrain is only necessary if the light source itself moves.Otherwise, snap the locator so it’s placed in the exact same position as the light). If you have multiple light sources or objects you want to track, you’ll need a locator for each one.

Fig. 1

With the scene ready, it’s time to set this up for rendering. For the light fog effect, you’ll need these three render layers:

• A beauty pass
• An RGB matte pass with the cube and ground plane assigned to different colors
• An RGB shadow pass that will serve as a matte for the light fog effect. This will be used to luma mask the CC Light Burst effect so that it doesn’t show in front of the cube within its shadow.

The beauty pass is simple enough. For this scene, the default masterLayer will suffice.

For the RGB matte, select the poly cube and the ground plane and assign them to a new render layer called ‘rgb_matte’. Assign a pure red surface shader to the cube and a pure green surface shader to the ground plane. (Figure 2) We’ll use this to key the box out in After Effects.

Fig. 2

The RGB shadow pass layer is a little tricky. I’ve had difficulty using Maya’s and mental ray’s shadow pass systems, however Jeremy Birn has a very excellent tutorial on the setup and merits of making an RGB shadow pass. The best workflow I’ve found is to duplicate the light source (rename it to some variant of ‘shadow light’ to avoid confusion), hide the primary light, turn off any decay that may be present (this is why we duplicate the light; you can’t use layer overrides for decay rate), apply Jeremy Birn’s workflow for making a shadow pass, and set the value of its shadow color to a very high level (I used 100). (Figure 3) The result should have very strong contrast between black and whatever primary color you choose, though the look can be tweaked in After Effects with color correction effects when we key the background out.

Fig. 3

With everything ready, batch render the sequence and bring those renders into After Effects. Make sure the alpha settings are set properly: the RGB shadow pass will need to have its alpha ignored to be properly luma keyed later. Drop the beauty pass into its own composition and name this composition “mainComp”.

We’ll be building a separate composition specifically for the fog effect. To start this, drop in the beauty pass. (We’ll only be using it as a visual guide and this will be hidden at the very end.) Rename this comp to ‘fogComp’. Next, drop in a white solid, then drop in the rgb_matte render. We’ll need to apply a Set Matte effect to the rgb_matte layer and set its ‘Use For Matte’ to red channel. Next, select the white solid and set its track matte to alpha. At this point, your composition should look like the beauty pass with an additional white box covering the cube. (Figure 4) If everything looks lined up properly, you can hide the beauty pass layer in fogComp.

Fig. 4

Next, we need to build the light fog effect. First, create a new adjustment layer and also drag on the RGB shadow sequence, placing it above the adjustment layer. Apply a Set Channels effect to the RGB shadow layer and set all color sources to whatever color channel you set as the shadow color in Maya (in my case, green). Then, select the adjustment layer, set its track matte to Luma Inverted, and apply a CC Light Burst effect. Tweak the Intensity and Ray Length as desired. The result should have the white cube visible with light fog extending around its edges, except for where its shadow on the ground plane would be obstructing it. (Figure 5)

Fig. 5

Everything looks good so far, however the light fog should be going around the box; the box itself shouldn’t be lit. To fix this, drag on a black solid and the RGB matte. Re-apply the Set Matte effect to the RGB matte layer like we did a few minutes ago to isolate the red cube, then apply this as an alpha track matte to the black solid we just placed. There should now be a black ‘cutout’ where the box was, however there may be a thin white line going around the bottom edge of the box. To fix this, go down to the rgb matte layer that’s affecting the white solid layer and apply a Minimax effect, setting its operation to Minimum, its radius to 1, and the channel to Alpha. This should remove the thin white line. (Figure 6)

Fig. 6

From here you can tweak the look of the light fog. For starters, you can add a Levels effect to the RGB shadow layer to adjust the spread of the light fog onto the ground plane, or you can apply some Gaussian blur effects to the RGB matte layers to soften the edge of the fog.

Now to apply this light fog effect our mainComp. Go to mainComp and drop the fogComp on top of it, sending its blend mode to Screen. If you scrub through your timeline you should now have the silhouette of the cube with light fog being cast from its edges, however there’s no directionality to the light fog: it’s always being emitted from the very center of the composition. For stills or sequences with minimal camera movement you can move the Center attribute of the CC Light Burst effect and be done with it, but for complex sequences or animations where the camera moves a lot this may not be practical.

In the next part of the tutorial, we’ll go over how to take the animation data already in Maya and apply that to a 2D effect in After Effects. If you have any questions about this part of the workflow, shoot me an email at dbeach@dbeach3d.com or leave a note in the comments section. Thanks for reading!

Click here to return to the blog
Click here to go to the dbeach3d.com front page

Tags: , , , ,

Leave a Reply