Bernie Su, the first person to win an Emmy for his genre-busting new forms of video, is at it again, testing his latest project, Whispers, with an elite crowd of Southern California tech-entertainment leaders Wednesday night at one of the movie industry’s great redoubts, the Sony Pictures Entertainment lot in Los Angeles suburb Culver City.
Su’s project with digital production company Pickford is one of two exploring how AI tools can create a unique interactive experience for audiences. Whispers follows Marcus Kent, a private investigator trying to solve crimes with help from “whispers,” AI prompts and clues submitted by cellphone-wielding audience members.
Det. Marcus Kent’s behavior in ‘Whispers’ is informed by the mobile-phone prompts of audience members, streaming alongside the main screen during the experience.
(Image by David Bloom)
The AI generating the experience incorporates the whispers from across the length of the performance, and creates a one-of-kind experience for that specific audience, the generated animation using cloned voices created (by permission) from a set of SAG-AFTRA actors.
The Whispers experience unfurled down the street from AI on the Lot, the suddenly massive conference focused on using AI for filmmaking held partly at the sprawling home for sponsor Amazon MGM Studios and Amazon Music.
Su joined digital production startup Pickford almost exactly a year ago, he said, to bring to fruition his ideas for a new kind of theatrical experience that was intensely interactive, using AI to create something well beyond old-school choose-your-own-adventure games and books.
Indeed, when it’s working well, Whispers suggests a new direction for struggling theater chains or even connected TVs to offer audiences new kinds of fun experiences. Importantly, the transformation would require relatively little new technology or physical enhancement or restructuring for theater operators.
That said, an experience with Whispers or anything similar to it will need to come with a couple of cautions, if Wednesday night’s experience with about 100 of the most prominent (and puckishly contrary) minds in Hollywood technology is in any way representative.
Afterward, among those watching was Renard Jenkins, a former PBS and Warner executive who is president of industry tech group HPA. He suggested most theaters likely would need to beef up their WiFi connectivity, especially in a larger theater where as many as a few hundred people might be simultaneously the show.
The hour-long screening generated several hundred “whispers” to Marcus, including a remarkably dedicated set of contributions from a small group apparently fixated on the general idea of Taco Bell chalupas, i.e., something ridiculous that had virtually nothing to do with the noir-ish crime mystery.
It’s not hard to imagine, for instance, sets of teen boys trying to monkey wrench a theatrical presentation with wildly inappropriate prompts, though the AI could be used to screen the worst of such attempts, just as it is when similar systems moderate language in videogame chatrooms such as Call of Duty.
The other challenge is reducing the latency of the AI itself in synthesizing all the whispers as it prepares to render the next scene. Buffering for up to a few seconds happened several times as the AI engine.
But regardless, there’s much to intrigue with Whispers and the opportunities it suggests.
Su won Emmys for achievement in interactive media in 2013 (Lizzie Bennet Diaries), 2015 (Emma Approved), and 2019 (Artificial). Each of those projects pushed older generations of technology.
Whispers now moves to the Alamo Drafthouse theater in downtown Los Angeles, among other Alamo outposts around the country, Pickford executives said.