Overlaying rendered vehicles on top of live-action video can certainly create compelling visuals for your case. To do this with Virtual CRASH, you simply need Virtual CRASH (of course), a free app called nomacs (nomacs.org), and the highly affordable Adobe After Effects. At $20 per month, we can say enough great things about After Effects.

First, you’ll need some live-action video footage over which you’ll superimpose your Virtual CRASH rendering. In this example case presented below, a DJI Mavic 2 Pro was flown at the scene. The drone photographs were processed by Pix4D, which generated the orthomosaic and point cloud data. This data was used to build the scene in Virtual CRASH 4. The Smart Alignment Tool in Virtual CRASH 4 was used to automatically align the point cloud and individual orthomosaic tiles. Here we see the orthomosaic tiles automatically aligned in our scene.

Automatic orthomosaic tile alignment uses the tfw files generated by Pix4D to determine the relative alignment between images, point cloud, and RTK GPS-based ground control points. You can learn more about automatic tile alignment here.

Below we see the scene with point cloud data and terrain surface.

Capture Live Video with Drone

The drone was flown at the shoulder of the road in order to capture live-action video from the desired perspective. Using a RTK GPS device, the position of the drone was measured in state plane coordinates (a small rope was attached the drone to measure the drone’s z-position above ground height). Here’s the drone video below used for the background.

Camera Matching Live Video to Virtual CRASH using NOMACS

A single frame of the video was captured and converted into png file format. The image was loaded into nomacs. The image was locked on top of the Windows desktop environment (Ctrl+Alt+Shift+B) and the opacity was adjusted (Ctrl+J to increase and Ctrl+Shift+J to decrease) so that the Virtual CRASH environment could be seen underneath. With the live video frame visible, the camera within the Virtual CRASH scene was adjusted until a good match was observed.

3-Step Rendering in Virtual CRASH

First pass: Full Render

With the camera angle set in Virtual CRASH, 3 renders were generated.

Here we see the full render. Note, not much time was spent “cleaning up” the scene since only the vehicles would be used in the final production. We see the final render below.

Second pass: Render with Barrier Material Type

Since visuals from a “virtual world” and the real world are being blended, a second render was done where the vehicle material types were changed from “cab” to “barrier”. Barrier is a matte material type in Virtual CRASH whose color is inherited from the color selection in the misc menu in the left-side control panel. The purpose is to overlay the two renders in After Effects, and “blend” the two videos to fine-tune the desired reflectivity of the vehicles. This render, were the vehicles have barrier material type, is shown below.

Third pass: Render the Stencil Layer

In the third pass, a “stencil” is created. This stencil layer will be used in After Effects to “mask” out everything in our renders except the vehicles and the shadows. This is done by creating a magenta filled polyshape to colorize the terrain mesh, as well as a magenta box to use as a back drop. The vehicles were set to barrier material type, and color set to green. This is shown below.

Overlay Layers

With the 3 layers finished, they can be overlaid in After Effects. First, the opacity of the matte layer is adjusted until the desired amount of reflectivity is observed. Note, the matte layer sits over the full render layer in this case.

Next, the stencil layer is set to visible. The Key Color effect is used to remove the magenta background color, leaving only the matte green vehicles, wheels, and dark magenta shadows. With the color tolerance set, the stencil layer’s mode is set to “Stencil Alpha” which effectively masks everything magenta that was selected with the Key Color option.

Here we see the final video output with vehicles and shadows over a transparent background.

Overlay and Live-Action Video and Correct Lens Distortion

With the full render, matte render, and stencil render layers completed, now the focus is shifted to better matching the live-action video and Virtual CRASH videos. Placing the rendered video and live-action video into After Effects, the Bezier Warp tool can be used on the live-action video to correct lens distortion.

Final Composition

With the videos overlaid and aligned, pixel motion blur is added. Tire marks and smoke effects are also added within After Effects. Finally, camera shake is also added - this helps mask the slight drift in the drone’s position during recording of the live video. We see the final ad video below.

Overlaying Rendered Video from Virtual CRASH on top of Live-Action Video