
Virtual Lighting

Virtual Lighting
Projected Lighting Effects for Complex Scenery
Virtual Lighting is a method of creating both realistic and stylistic lighting looks on stage through projection. It is able to create unique qualities of light that are either impossible or impractical to achieve with conventional lighting, but can also recreate traditional lighting looks with only a projector. Virtual Lighting is a method of creating both realistic and stylistic lighting looks on stage through projection. It is able to create unique qualities of light that are either impossible or impractical to achieve with conventional lighting, but can also recreate traditional lighting looks with only a projector.
On stage, virtual lighting allows us to create lighting looks that are not possible (or are at least impractical) to create with conventional lighting. Additionally, it is able to create these effects without requiring any clear wingspace, and with minimal rigging and equipment needs. The following examples are all created using a single projector on a preexisting theatre set. Each is designed to showcase a particular lighting effect that virtual lighting enables.
1. Sunset Through a Window
This clip shows a late afternoon sun falling across the set. This general concept is nothing new to theatrical lighting, but the use of virtual lighting here adds a layer of depth and realism beyond what conventional lighting can chive. This is easiest to see in the following three elements.
This is the shadow of collimated light – as the silhouette falls across the stage, the apparent size of the window’s shadow stays the same. As the animation progresses edges of the window panes remain parallel to one another, while the angles they meet at change to match the angle of the sun.
Because the light source itself moves, the shadows that objects on stage cast on themselves and each other shift naturally. For example, at the beginning of the animation the bookshelves cast shadows below them, and the silhouette of the lampshade on the wall falls slightly below the wall sconce.
In addition to onstage objects’ shadows shifting naturally, the tree’s silhouette moves in relation to the window’s as the angle of the sun changes and different light passes through both the tree and the window.
These elements together create a far more natural effect than can practicably be achieved with conventional lighting, particularly considering the dynamic nature of the scene.
2. Natural Daylight
Sunlight - and in particular sunrise and sunset - is deeply rooted image in theatre, but a challenging effect to portray onstage. Stylized sunrise and sunset are as (if not more) common as window in the late afternoon. The long, moving shadows of natural sunlight seen here, however, are a step further than we can recreate accurately indoors through conventional means.
This virtual lighting has plenty of challenges and restrictions, but where those can be accommodated this method is capable of simulating sunlight onstage across a wide and complex set with a much higher level of realism than conventional lighting design.
3. Water Caustics
Another excellent example of distinct but challenging lighting to recreate outside of it’s natural circumstances, “water caustics” refers to the patterns created by light passing and refracting through rippling water. This video describes some of the challenges in recreating this quality of light, with the conclusion that the most effective way to create it is to light a set through large amounts of water in inconvenient locations.
With virtual lighting, however, this effect can be created and also manipulated easily, and it wraps naturally around the geometry of the set without stretching or distorting as would a moving gobo in a stage light or an unmapped projection. It can also be applied selectively, in this case to the sides of the set, but one can imagine applying this only to particular objects, or specifically excluding them.
The final two examples recreate more realistic, natural lighting looks in ways that match the set design.
4. Car Headlights
This is a straightforward one, showing car headlights passing by a house, followed by one turning into a driveway and parking. While uncomplicated, it’s an effective tool to set the context for a scene, and impractical to recreate effectively with conventional lighting.
5. Indoor Lighting
This is an application of virtual lighting recreates natural, indoor lighting. Admittedly, this is something that conventional lighting is quite good at, and there’s no reason this effect couldn’t be achieved that way. That said, it’s significant that virtual lighting can also recreate these ordinary lighting effects, and there are certain circumstances where one might want to.
For example, if the set has projection coverage for other reasons (like any of the previous five examples), this effect can be added with no additional equipment. This saves money and/or inventory that can be used for other aspects of the production, and also saves on backstage space. This particular production of Good People would not have the backstage space to recreate this look with conventional lighting during a show, and virtual lighting might be the only way to implement an effect like this effectively, given those circumstances.
How it’s Made

The virtual lighting process consists of four general steps:

3D Scan: A high quality 3D scan of the set creates the foundation for both generating the effects and mapping them onto the set’s geometry. The above examples were achieved with photogrammetry, using a DSLR with several cross-polarized flash systems, and Agisoft’s Metashape software.
Mesh Cleanup: Basic 3D modeling is required for two tasks. First, it allows us to correct any errors in the scan of the set, notably smoothing rough surfaces and removing transparent surfaces. Second, it allows us to modify the unseen geometry of the environment in line with the design of the set, such as adding a window or removing an unseen props shelf. Third, it can be used to break the scan into several sections. This allows the game engine to treat certain elements of the set differently – e.g. lampshades, which may want to act as emissive light sources in certain scenes.
Lighting Design: In these examples the lighting is created in Unreal Engine 5, using a variety of light sources, emissive decals, and animations.
Projected Output: Accurate projection of these effects requires a carefully positioned projector rigged in the venue, along with precise calibration of the game engine to map the 3D scan of the set to the real thing. We adapt a camera in the game engine, to match its position and optics with that of the projector. This allows us to align the projection on surfaces that are too complex for traditional projection mapping.