Integrating CG imagery with footage from cameras using ACES
As a loose outline, it’s much better to skip the ACES complexity and focus on what is required. ACES comes with its own entire set of issues, not the least of which is obfuscating the goals and the technical needs.

It’s All About Radiometric-Like Ratios

Integration requires using a specific encoding. That is, the key here is that we seek the radiometric ratios as they existed in the scene. This is perhaps the number one aspect of integration that is, and has been, overlooked for a long, long time amongst many folks.

The key here is to understand that cameras capture scene data. However, despite all cameras doing so, they all do not encode the data in a manner that is useful. Typically, camera encodings can be described in one of two manners:


Ready to view as an aesthetic, display ready encoding
Ready to decode and work with further


The Encodings

You’ll find countless examples online where people take random footage and attempt to integrate it. Some even can look pretty remarkable, such as some of Ian Hubert’s work.

The sad part is that this isn’t the appropriate means to make life easy for themselves. If the footage “looks ok” when played back, it likely is the absolutely wrong encoding to work with, and will result in no end of massaging, sucking up countless hours of time and energy, only to never-quite-look terrific.

This should be common knowledge now for anyone who has tried to take a generic JPEG photo found online and attempted to integrate it into a Filmic based scene; it just can’t be done well. Why? Because the display-ready image has become an impossible to decode combination lock on its way to becoming pretty to look at.

The Cameras

It might shock a majority of readers to learn that you don’t need an Arri Alexa LF to get these sorts of magical encodings. Sony, Nikon, Panasonic, Fuji, Canon, and other vendors now commonly include a log-like encoding scheme in their affordable consumer grade cameras. A good number of them offer the ever-more-useful-and-important on-board 10 bit or greater bit depth recording.

While countless YouTube videos wax lyrical with misinformation about “flat” picture styles and what they are good for, the value of the documented log encodings is entirely and completely missed; log-like encodings capture and encode scene reflectance!

That is, for every log-like encoding scheme such as Sony’s SLog1-2-3, Fuji’s F-Log, Panasonic’s V-Log, Nikon’s N-Log or Canon’s C-Log1-2-3, there is a documented function available in a PDF that describes how the values are encoded with a unique transfer function.

Unlike a display’s transfer function that encodes display linear light however, the gold is in the little detail that those normalized code values can be unlocked back to scene reflectivity!

Where sRGB’s transfer function can be undone to reveal the display linear light output from 0-100%, a log-like encoding can be taken back to the scene’s ratios from some low value to some high value.

Try It Right Now

To really hammer this point home, a simple and revealing test can be done:


Take any old generic render with a reasonable dynamic range to illustrate the point.
Save the image with a sixteen bit integer TIFF using Filmic Log Encoding Base. That’s the very flat, non-aesthetic looking one.
Assert that no looks are baked into the output. We want data, not aesthetics.
Using the Image Viewer, sample around the render using the Right Mouse Button. Note the values from some of the scene energy that appear on the left side of the information bar that appears. That is, the RGB values to the left of the CM values, not the right. They should extend up arbitrarily high.
Also save the exact same image using Filmic with Base Contrast to make it look OK. Save the exact same sixteen bit TIFF.


Once saved. Open up another instance of Blender.


Load the Filmic Log Encoding Base image.
Set the colour space on the image to the corresponding Filmic Log Encoding Base.
Set up a compositor node to assert the file is promoted to float and sample the values in the Image Viewer.
Add in one more input and load the aesthetic output Filmic with Base Contrast. In this instance, the image was encoded for an sRGB display, so the transform is sRGB.


Note the values? Despite the fact that the sixteen bit integer TIFF was encoded using a device referred 0-100% range in both cases, when properly transformed via an appropriate transfer function such as Filmic Log, the values magically are transformed right back into the scene referred domain. The aesthetic “looks good” Filmic with Base Contrast? Hrm... only 0.0 to 1.0 representing the display linear aesthetically warped light, not the scene.

And guess what? Those logs for all of those awesome cameras listed above? They’ll do exactly the same thing!

This allows one to take the log-like encoded footage and seamlessly integrate it into any CGI with nearly no effort whatsoever. Use them as reflections, as emissions, and have interactive lighting on all your work.

But Wait, There’s More

There is one little gotcha with the log-like encodings; they are almost always coupled with a unique set of primaries that are different from the BT.709 assumed lights in your sRGB display, and the default set of rendering lights in Blender for Cycles and Eevee. The details of the particular encoding are unfortunately often referenced as some sort of a “gamut”. That is:


F-Log will be also encoded with F-Gamut
C-Log 1-2-3 may be encoded with a few different gamuts
N-Log will be encoded with N-Gamut
V-Log / V-Log-L will be encoded with V-Gamut
S-Log 1-2-3 may be encoded with a few different gamuts


One must take care to, after linearization with the proper transfer function, to also take the primary lights to the same primaries in the rendering / working space. This is a slightly nuanced subject, and likely requires another answer unto itself once the camera specifics are known.

No Log Camera? No Problem!

Beyond all doubt, this is a mind blowing experiment for the intrepid 3D peep to muck with, and hugely encouraged for anyone who has a log-capable camera.

But what about all of us schmucks who don’t have the releativey more recent cameras with log encoding options? There is indeed hope with a little bit of effort.

It’s entirely possible to take generic camera footage and document the transfer function manually. It’s a little elbow work, but the results will be similar to a native log, and the output will be significantly better than ignoring the core issues. Thankfully, there is an absolutely amazing set of tutorials to achieve just this via Paul Chambers’ website. After having read that amazing series, there is another little hidden gem on his site to help you use that radiometric scene reflectance too!

The key point in all of this is never let some overhyped bit of a plug-in or system prevent one from learning the core principles. No amount of any canned system can help you here; it takes a bit of elbow grease and learning to unleash a heck of a lot of amazing work. You never know... there might even be a super top secret and visually stunning project being developed in dark crustacean labs. I wonder what would happen if someone asked them...