dock10 and the University of York have joined forces to look at how AI can make virtual studios more realistic.
One of the key technical difficulties is simulating realistic light interactions in live or “as live” multi camera productions and the joint research project will examine whether Artificial Intelligence can help.
On a green screen, an actor will not automatically cast a shadow or pick up reflections when placed into a computer-generated world – these have to added in post production. That’s particularly difficult using several cameras, which need to capture multiple angles simultaneously.
“Lighting is one of the core elements of creative expression in any TV show, but the lighting options for virtual studio productions are currently extremely constrained,” explained Dr. Florian Block, R&D Lead, AI & Immersive at dock10 studios and a Reader in Digital Creativity at the University of York.
“That’s what makes this foundational research project so exciting. We are aiming to create a proof of concept that will ultimately lead to the development of a bespoke AI tool for delivering realistic lighting interactions in virtual studio productions in real-time.”
The aim of the project is to lay the foundations for the creation of innovative AI-powered compositing tools that ultimately will be applicable in real-time, making them suitable for live or ‘as live’ productions.
“dock10 is the UK leader in virtual studios production and the University of York is the UK leader for research into digital creativity,” added Richard Wormwell, dock10 Head of Production Innovation.
“We are combining our industry and academic expertise to try to crack one of the great challenges for virtual studio productions. To do so, we are exploring innovative, AI driven solutions that have the potential to radically improve virtual production workflows. We are hugely excited to see where this research project might lead.”
The R&D grant is from XR Network+ and will be led by XR Stories at the University of York.
