When AI moves into the camera
What happens when generative AI moves closer to the moment of filming? AP University's Immersive Lab took this research upon them and developed PAIC, a promptable AI camera. The system places an AI-layer between the camera and the output so visuals can be generated live while the camera is rolling.
From clip-based AI to real-time video
Most AI video tools are sequence based: they analyse frames in batches so they can offer more temporal stability. This works well for consistent results, but it also slows down the process. Trying a different background or visual style usually means generating a whole new version and waiting for the results.
PAIC works differently. It processes video frame by frame, changing each image and sending it directly to the output. A creator can type a prompt and immediately see the scene change in the live feed. A background, effect or character can be changed or finetuned while filming, a visual style can be adjusted mid-shot and a face swap can be done instantly.
To make this possible, the team developed a GPU (Graphics Processing Unit) acceleration method that allows the system to run dozens of AI processes every second. It runs several times faster per GPU than standard implementations. In another demo, the team tested a real-time face swap using the face of Belgian rapper Zwangere Guy. This shows that the pipeline is flexible, allowing different AI models to be integrated or replaced depending on the user’s needs. As new models become available, they can be added without changing the overall setup.
Key takeaways
- Enhancing virtual studio workflows
The technology can be complementary to studio environments such as green key or virtual production studios by adding an AI layer to the live camera feed. This allows visual changes to be applied directly during filming. - Testing visual styles during production
Because visual changes appear instantly, production teams can compare and iterate on different styles, backgrounds or effects while the camera is rolling, without needing to generate multiple versions afterwards. - Real-time visual adjustments for creators
Earlier experiments also explored how creators could adjust the look of their content while recording, by prompting visual changes directly on the video feed. In that sense, what you want is what you get: the result becomes immediately visible in the live output.
Conclusion
Film production traditionally moves step by step. Preparation comes first, filming follows and post-production comes last. Technologies such as virtual production have already blurred those lines by letting producers preview environments during the shoot. PAIC pushes that idea further by allowing visual changes to happen during filming itself.
The AP Immersive Lab is currently testing the system with partners to see where it works best in real productions and is looking for additional partners interested in testing the technology in practice. Systems like PAIC could shift part of the creative process from post-production to the moment of filming.
