Hyper Realistic Virtual Lighting System for Video Productions/Photography


Infocomm - Computer Simulation & Modeling
Infocomm - Augmented Reality, Virtual Reality & Computer


Having the correct lighting is an important aspect of video production, not only does it allow video creators to light the subject of the video correctly - ensuring the absence of distracting shadows or unnecessary reflections on reflective objects, it also allows content creators to convey the right artistic expression. The requirements of selecting the correct lighting temperature, strength (light intensity), and lighting effects are determined by the lighting arrangement and the way lighting is applied at the location, making it difficult to modify what has already been shot in post-production. Therefore, lighting modifications require reshoots, which require a lot of effort and cost.

For a typical video production involving a single subject, the lighting set-up is typically a three-point lighting arrangement to produce a three-dimensional (3D) lighting effect. This three-point lighting comprises the main light, fill light, and finally, background light.

This technology offer comprises a specialised lighting apparatus that constitutes a set of light emitters and light receivers, and matching/modelling software that maps the reflectance of the object and the background to allow lighting conditions to be changed post-production in a realistic manner, preserving the texture and natural lighting effect between the foreground and background. This technology makes it possible to create images as desired by the creator without the need for special equipment or knowledge of lighting, from a single specialised light.


Specialised Lighting Apparatus:

  • Array of light emitting elements (variable distribution angle and at different wavelengths) and illuminance sensors

Matching and Modelling Software:

  • Synchronises with the main camera (imaging data) to match the captured reflection map (illuminance)
  • Lighting can be freely changed post-video production
  • No prior knowledge of lighting is required
  • Enables virtual lighting effects including variable lighting intensity, temperature etc.

For video production, reflection maps and image data are recorded, and each is matched to enable virtual lighting. The procedure is as follows:

  1. Recording: Main camera and special lighting are position with respect to the subject, and image data and reflection maps are captured
  2. Matching: Algorithmically match and model the reflectance map to the camera image data
  3. Editing: Simulate any lighting placement in any lighting arrangement on editing software

For 3D modeling, reflection maps and image data of the object and background are recorded, and each is matched to enable the simulation by matching the illuminance of the object and background. The procedure is as follows:

  1. Recording: Main camera and special lighting are positioned on the object, and image data and reflection maps are captured while the object is rotated. Next, the main camera and special lighting are positioned against the background, and the image data and reflection map are captured
  2. Matching: 3D modeling is performed by matching multiple image data from the camera with multiple reflection maps using an algorithm
  3. Simulation: Place objects in any orientation while matching the illuminance of the object and the background within the software


  • In the case of video production, it enables virtual lighting effects in post-production (video and photo), reducing the labor and time required for lighting installation, and also allows the user to handle lighting effects without knowledge of lighting
  • In the case of 3D model production, the exact match between background illumination and subject illumination improves the realism of lighting effects, resulting in better reproduction of textures and natural lighting. Therefore, it can be applied to lighting of e-commerce products, lighting of people, and walk-through images of real estate, etc.

Market Trends & Opportunities

The expansion of video distribution services and social networking services has led to the diversification of video expression. There are limits to what can be expressed with a camera alone, and demand is increasing for expanded video expression through the use of lighting.

Unique Value Proposition

Compared to existing techniques that estimate illuminance from images, more accurate illumination data can be obtained by observing actual illuminance using special lighting. This allows arbitrary lighting effects to be added to the post-production of images, thus reducing the cost and man-hours required for reshoots. Additionally, no lighting expertise is required, allowing free-form lighting expression.

This technology also addresses the gap between product images that are captured for e-commerce products with the actual product. The technology's ability to naturally combine background lighting with subject lighting, combined with augmented reality (AR) technology, allows for an accurate reproduction of the product's appearance within the purchaser's environment, enabling an immersive experience. Additionally, when applied to second-hand products, surface defects such as stains, and scratches are made visible, ensuring that buyers are aware of such abnormalities when purchasing such items.

The technology owner is keen to collaborate with technology partners to manufacture the specialised lighting and develop a Software Development Kit (SDK) for the photometric stereo illuminance mapping technique that can be integrated into video editing software or ray tracing computation of three-dimensional (3D) spaces.

Intuitive and Durable Capacitive Force Sensing Technology
Physical Climate Risk Analytics
Augmented Reality Artificial Intelligence Glasses
AI-enabled Virtual Modelling for Reduction of Energy, Carbon Dioxide Emission
Low-latency Digital Twin for Industrial Applications
Conversation-aware Virtual Patient for Mixed Reality Medical Training
Computational Fluid Dynamics Suite with Innovative AI Models For Industry Application