Emerging tech like virtual production is evolving rapidly. Is Hong Kong ready?

This article appeared originally in SCMP on 14 April, 2023.
Authors: Yolanda Lam, Researcher at Our Hong Kong Foundation.

The cultural and creative industries have for a long time used innovative technologies to create content. From artificial intelligence to virtual reality, the use of such technology in the industry evolved even more rapidly during the Covid-19 pandemic.

With the return of live audiences and countries reopening their borders, some technologies have faded into the background. However, others have not only stayed but continue to disrupt traditional workflows, ushering us into a new era of content creation.

Virtual production is one such technology. It has gained a strong foothold in film and television in the past few years, as seen in hits such as Top Gun: Maverick and House of the Dragon, as well as the screen industries in general.

Virtual production is when creators use real-time game engines to create interactive virtual backgrounds and put them on large LED volumes – a system of linked high-end LED panels used to display video footage or 3D content – to create a realistic film set.

Special effects that are normally added in post-production are instead brought forward to pre-production. In essence, virtual production enables physical – that is, actors and props – and digital elements, such as special effects, to come together in real time.

Compared with traditional methods of filmmaking, this can be more time- and cost-effective. The workflow allows for on-the-spot iteration and modification of scenes without posing too much risk to the production timeline or budget. Lighting, colouring and other background visualisation can all be altered in real time. A sunset could become a sunrise in an instant, for example. This can shorten week-long, multi-location shoots into several days in a studio.

Virtual production also creates a safer and more sustainable working environment, which enables greater flexibility and creative freedom for storytellers. Cast and crew members do not need to venture into precarious and unpredictable locations for filming, as those environments can now be recreated and manipulated on set in the studio.

More importantly, this has given rise to libraries of digital assets and virtual environments. These can be reused as long as they are compatible with the game engine.

Although virtual production is unlikely to replace existing workflows entirely, it could still represent the future of content creation for film, television, music videos, advertisements, live events and the like. It is no wonder that its market size is growing steadily. In 2022, the global virtual production market was valued at US$1.82 billion, and this number is expected to grow to US$6.79 billion by 2030.

Large virtual production studios are also popping up worldwide. The world’s oldest large-scale film studio, Studio Babelsberg, unveiled its own Dark Bay Virtual Production Studio in 2021. It remains one of Europe’s largest virtual production studios, and it was where Netflix’s sci-fi series 1899 chose to film.

More recently, Los Angeles-based NantStudios launched the world’s largest permanent LED volume at their virtual production studio in Australia. Similar set-ups have been built by mainland China’s Versatile Media and Amazon Studios in the United States, as well as in South Korea and Thailand.

Given its rapid rise, virtual production has garnered government support. To build capacity for local content creators, Singapore’s Infocomm Media Development Authority introduced a Virtual Production Innovation Fund in partnership with the UK’s National Film and Television School last year. The fund supports the use of virtual production technologies in short-form content such as music videos, short films, commercials and live streaming.

To advance research and development in this area, Innovate UK also funded the virtual production R&D of disguise, a leading software company in the field, when the technology started garnering widespread attention in 2021.

In Hong Kong, the industry is also keen to explore virtual production. Shaw Studios and the Hong Kong Society of Cinematographers organised an on-set virtual production film test in 2021, inviting local filmmakers to learn about the process. Last year, the Science Park welcomed its first virtual production company, Votion Studios. Other multinational live-event companies with Hong Kong offices are also equipped with virtual production facilities, such as Creative Technology.

Developing virtual production capabilities requires significant training and expensive hardware, and testing space. Qualified experts are in high demand, but they remain scarce.

The government has promoted art tech in recent years, prompting several higher education institutions to offer relevant courses. The Hong Kong Design Institute is one such institution, teaching virtual production through its Higher Diploma in Arts Technology programme, covering skills such as pre-production and previsualisation, real-time 3D visualisation and cinematic scene modelling. It is also the first tertiary institution in Hong Kong to bring a full set of LED volume into the classroom to teach virtual production.

This prompts deliberation about the importance of building bridges between higher education and the industry so future creative talent can learn the right skill sets and keep pace with the latest advancements. It is therefore a good time for the government to explore ways to fund and support capacity building and R&D in virtual production and other emerging technologies.

Virtual production is innovating the way stories are told and forging new careers paths. Can Hong Kong’s policymakers keep up?