What is the VFX Pipeline?

What is the VFX Pipeline?

The VFX pipeline breaks down the steps of a visual effects workflow for film, television, and digital media projects. It keeps the entire VFX process organized; it allows everyone to know their role and how it fits into the production timeline. From storyboarding and reference imagery, all the way through modeling, rotoscoping, composition, and lighting (just to name a few).

In smaller productions, one VFX artist may handle the entire workflow, but most productions use teams of specialized artists.

The pipeline brings a level of sanity to a process that is usually not completely linear. Members of the VFX process often get involved during the pre-production, production, and post-production stages.

To do their best work (and to ensure they’re not asked to redo various steps), artists should understand and appreciate each step of the VFX workflow pipeline.

💡 Read more: How Stimulated Inc. Moves 6K Immersive VFX Footage for Stage Shows

VFX Pipeline Diagram

⬇️ Download PDF Version The VFX Pipeline Diagram

The VFX Workflow

When asking the question “What is VFX mean in editing?”, it’s important to understand that no two real-world VFX pipeline workflows are identical. We’ve divided these steps into the pre-production, production, and post-production phases in our VFX roadmap below, but many steps below often occur in parallel throughout the project.

Pre-Production

Much of the visual effect workflow in pre-production is planning-related, which helps keep crews informed while preparing for any technical requirements or potential execution issues. VFX artists in the planning stage can download VFX shot list templates to save time when planning which shots and scenes need visual effects.

1. Research & Development (R&D)

R&D on a video project primarily involves Technical Directors (TDs) who work with VFX supervisors to plan the technical approach and determine which shots and effects are technically feasible. Extremely VFX-heavy projects may also involve outside scientists, engineers, or mathematicians for further guidance.

TDs must ensure that all software and files used throughout the VFX pipeline are compatible and sometimes create custom software and plug-ins to improve VFX pipeline efficiency.

Most of the R&D stage is ongoing throughout a project’s lifetime as the material is tweaked and concepts evolve.

2. Storyboarding and Animatics

Storyboarding is where the VFX artist team creates visual representations of all the actions within the script. Character motions and story settings are analyzed and basic drawings are created to illustrate the desired framing on a shot-by-shot basis.

Like most planning elements, however, storyboarding isn’t final. It’s more about planting a stake in the ground and giving the VFX artists a solid idea of what the editorial team wants.

3. Pre-Visualization

Also known as previs, pre-visualization uses storyboards to create low-poly 3D models, wireframes, and scene representations to function as stand-ins for the visual effects to come. Previs typically takes place alongside other members of the creative team to determine camera angles, determine shoot locations, and plan complex scenes.

Other VFX steps in the pre-production stage can include concept and art design, which further refine artistic concepts and produces full images to define characters, settings, and props.

Layout (also known as production design), as the name implies, defines what final sets should look like and provides guidance to creators of physical or digital sets.

Production

This is when the VFX workflow really gets cracking because it’s when most of the shooting takes place, raw video files are created, and VFX dailies are submitted. But plenty of VFX tasks can be done in tandem with the production process.

4. 3D Modeling

3D modeling takes place throughout all three production phases, but in the production phase, artists transform storyboard art or low-poly 3D models into lifelike representations. Most 3D modeling is devoted to creating assets such as vehicles or buildings that either aren’t practical or cost-effective to bring on set, but 3D models are also used to create characters (to illustrate non-humans or stand in as digital doubles) and other props.

While 3D models of impractical things like spaceships or the Batmobile are the most visually interesting, 3D modelers also replace or complement physical objects shot on set that need improvements in lighting, shadowing, or texture.

3D modeling is one of the most time-consuming and labor-intensive elements of the VFX pipeline and often depends on reference photos taken during production. The 3D models that move must be rigged and animated, but more on this later.

5. Matte Painting

Matte painting is one of the oldest VFX techniques in existence and involves creating visual backgrounds that don’t exist. These days such backgrounds are often created digitally using LED panels and game engines, often as entire 3D sets for virtual production, or by chroma keying using a green or blue screen.

Years ago, matte painting was exclusively done using photographs and painted glass panels (matte paint was used because it doesn’t reflect light).

But matte painting is still used in many productions — including the Harry Potter films, Game of Thrones, and The Witcher — partly because it can save money. But it’s pretty limited in what it can do: Matte paintings can’t change their lighting, camera angles, or other elements.

6. Reference Photography

Throughout the entire production phase members of the VFX team hang out on set to take reference photos of actors, scenes, props, and anything else important. These photos are then used to rig, animate, and add texture to 3D models.

💡 Read More: Large File Transfer for Post-Production Teams

Post-Production

Post-production brings all the elements of a video production together — VFX, footage, music, and sound — into the finished product. While, as we’ve seen, the VFX team is busy throughout the production cycle, the VFX post-production workflow is the busiest phase of the entire process for the VFX team.

7. Rigging and Animating

Imagine what happens when a puppeteer pulls the strings on a marionette, and you’ve already got a pretty good idea of what rigging and animating are all about — only at a digital level. Rigging and animation breathe life into 3D models by building a system of controls that animators can use to manipulate these objects.

Rigging teams often rely on reference photographs, but motion capture cameras or suits are also often used to capture movement data to aid the rigging and animation process.

Rigging teams often get so granular that their jobs can include calculating skin weights and adding digital skeletons and muscles within 3D characters to replicate natural movement.

8. FX and Simulation

FX artists are responsible for creating concepts and scenes that move and react according to the laws of physics, such as a long shot of a raging battle at sea or in space — complete with fiery explosions, which in reality can’t exist in space, but whatever — they look cool. FX artists often work with elements such as fire, smoke, liquids, and even particles.

FX artists work alongside animators to ensure these simulated elements don’t stick out (in a bad way) while looking and feeling as natural as possible.

9. Motion Tracking/Match Moving

Motion tracking, also known as match moving, allows VFX artists (in this context, referred to as matchmove artists) to insert effects into moving scenes and live-action footage without the entire thing looking bad.

After all, inserting VFX elements into a static shot is relatively easy, all things considered — but adding the same elements to a camera move involves many more variables. That’s why motion tracking accounts for the positioning, orientation, scale, and how the object moves within the shot, including replicating physical camera moves using virtual cameras in their motion tracking software.

10. Texturing

The texturing process is pretty much as it sounds: It adds textures to the surfaces of 3D models. Texture can include anything from surface color to scaly skin on a reptile, to reflections in water, to a metallic shine or scratches on a car door. This ensures models look as realistic as possible.

11. Rotoscoping and Masking

Rotoscoping involves artists drawing around and cutting out objects or characters from frames in the original footage, to use the cutout images against a different background or context.

Rotoscoping has typically been a relatively painful and manual process, especially in the days before computerized VFX.

“One legendary example of manual rotoscoping occurred during Alfred Hitchcock’s The Birds, when crews filmed birds in nature and rotoscoped them into each shot. It took three months to rotoscope hundreds of birds, one by one, into a single shot.”

Nowadays VFX artists still perform manual rotoscoping, but new tools such as Runway, which use machine learning, have helped to accelerate the process dramatically.

The entire rotoscoping and masking process can be avoided through chroma keying, which, as we mentioned, is the process of shooting foreground subjects against an easily removable background (like a green screen). But in many cases rotoscoping is also required to create a perfect cutout.

12. Lighting and Rendering

Lighting is typically dealt with once the texture artists have done their thing. It’s the last element applied before the effect or computer-generated image (CGI) is complete. Adding and adjusting virtual lighting and shadows to match either static or live-action computer-generated scenes or characters, like texturing, helps make them look more realistic while enhancing aspects of the original shot such as color and intensity.

Just like real-life lighting, however, virtual lights must be placed strategically within a scene. Lighting artists use tools such as shader settings and lighting maps to achieve this by positioning spot, area, and directional lights to match the angles and shadows of the original footage.

Once lighting has been applied, the entire scene is handed off to compositing.

13. Compositing

Compositing, sometimes called stitching, is the final step of the post-production VFX workflow. While it is the final step in the VFX roadmap, it is also the most important because it integrates all the various VFX elements with real-life footage to create a finalized shot or scene.

A bad compositing job can ruin all your otherwise great VFX work up to this point — so it’s crucial to get it right.

The process involves a compositor gathering all the content — including live-action footage, renders, VFX plates, and matte paintings — and layering them together in preparation for the next step in the post-production pipeline (typically coloring). Some shots may require combining just a couple of elements, but others may need to layer dozens while finalizing lighting, reflections, shadowing, and atmospherics to create a seamless look and feel.

💡 Read More: Using MASV To Deliver 4K 360° Driving Plates For VFX

VFX Tools and Software

VFX is only growing in importance in the modern video production industry — indeed, the first season of Amazon’s The Rings of Power used more than 1,500 VFX artists from 20 studios.

But what kind of software and VFX systems do industry-leading VFX artists use to weave their magic?

Here are a few examples.

After Effects

After Effects is regarded as one of the best, if not the best, VFX software around. That’s partly because it integrates seamlessly with Adobe’s Premiere Pro video editing software and collaboration tools such as Frame.io, but also because it’s damn good at what it does.

After Effects also has plenty of third-party, customizable VFX templates you can download to help scale your project.

Fusion

Blackmagic Design’s Fusion is a great tool for creating immersive 360 or VR video; stereoscopic 3D effects; and the compositing of 3D models and real-life, live-action footage. It comes as part of video editing software DaVinci Resolve, and has been used to create VFX scenes in films such as Guardians of the Galaxy and the Hunger Games and even cinematics for video games such as Halo 5.

Nuke

Foundry’s Nuke is VFX and film editing software used by major industry players such as Walt Disney Animation Studios, Blizzard Entertainment, Sony Pictures Animation, and DreamWorks Animation, and has been used on productions from The Crown to Boardwalk Empire. It offers seamless review workflows and the ability to add VFX elements to dynamic editorial timelines.

Houdini

Houdini by SideFX is used in the R&D process and at other junctures to come up with customized effects. It integrates animation design, effects rendering, and character modeling and provides a host of VFX simulation modules for fluids, crowds, grains, and other elements including destruction and pyro FX. Houdini also integrates with other software such as Maya.

Maya

Autodesk’s Maya provides 3D animation and modeling, simulations, and ultra-realistic rendering. Lighting artists often use it to create and place virtual lighting and by 3D modelers to create and rig animated characters or other objects. Although it has been described as difficult to use by reviewers, the software covers various VFX pipeline elements, including dynamic simulations, texturing, and animation.

HitFilm Pro

HitFilm Pro (and its free consumer market version, HitFilm Express) is an all-in-one VFX and video editing tool that allows VFX artists to apply effects to the NLE timeline (instead of laying them). The app comes loaded with nearly 1,000 VFX templates and presets, with features such as masking, 2D and 3D motion tracking, green screen keying, and particle simulators.

Blender

Blender by Blender Foundation is powerful open-source freeware perfect for those starting in VFX who want to learn the craft without shelling out thousands of dollars on other alternatives (many mentioned above). Its 3D animation tools include a camera and object tracker that offers manual or automated tracking, camera reconstruction, and real-time previews.

Tips and Best Practices for High-Quality Visuals

• Engage in constant and ongoing communication between various creative teams; not just other VFX artists but also video editors, colorists, the on-location production team, the director, etc.
• Know the various VFX systems (tools, processes, software, etc.) of the entire VFX workflow inside and out before embarking on a months-long project.
• In virtual production, VFX artists must be aware that what looks good on workstations may not on a gigantic LED wall. Unreal Engine recommends their In-Camera VFX Production Test to see recommended configurations for virtual stages.
• Replace physical objects (fires, potholes, tire tracks, etc.) with VFX renderings to save a ton of money over replicating the same thing in the physical world, or having to go back and re-shoot certain scenes to get things perfect.
• Netflix recommends VFX artists working from home use virtual desktop solutions such as HP RGS or Teradici, a single monitor at 1920×1200 as a baseline resolution, and dedicated bandwidth and low-latency connections.
• Always follow proper file naming conventions.

Resources for Further Learning and Development

Here are a few more VFX guides to help you uncover even more insights around VFX pipelines and best practices:

• Netflix VFX Best Practices
• Unreal engine In-Camera VFX Best Practices
• How These Free or Affordable VFX Tools Can Inspire Your Storytelling
• What is Compositing? VFX Compositing Techniques Explained
• VFX: How to Use a Green Screen Like a Pro in 4 Easy Steps
• How do you integrate visual effects into your production design?