Restoring the Voice of an Opera Legend: Respeecher gives Maria Callas a Synthetic Opera Treatment
Earlier this year, Respeecher contributed to the 2023 Accelerator project "Synthetic Humans for Entertainment & Accessibility." An objective of the project was to blur the lines between reality and synthetic media using advanced technologies to create lifelike synthetic humans.
To meet the demands of the challenge, Respeecher was going to need a two-pronged attack. The first requirement was to create a synthetic rendition of Maria Callas, the renowned Greek opera singer. We haven’t been focused on Maria’s signing performance, but on her speech, as we were responsible for a part where Maria handles press conference questions. This meant not only replicating her voice but also capturing the nuances and emotive qualities inherent in her speech.
The second requirement revolved around developing a photorealistic sign language interpreter capable of satisfying the demands of accessibility in broadcasting. To deliver on these objectives, every partner involved in the project was going to have to up their game within the voice and motion capture technology space, as well as in creating photorealistic 3D models. If the final result delivered anything less than enhancing the inclusivity and diversity of the media content, the project would be deemed a failure. So, while Respeecher went to work on voice cloning, the other companies spun up their video production departments.
Respeecher, Signly, Pluxbox, and 4DR Studios worked on making this vision a reality. Throughout the process, we put multiple motion capture technologies and synthetic human creation toolkits through their paces. Respeecher specifically contributed by cloning voices, notably recreating Maria Callas' voice for interactive interview segments. We meticulously analyzed and processed audio recordings from her archive before using our AI algorithms to create the most accurate voice model of Callas. The other teams' expertise in lip-sync technology and real-time facial expression capture played a crucial role in making the synthetic Callas both sound and appear more realistic.
When all was said and done, the project’s final result was due to the collaborative success of every team involved. The synthetic version of Maria Callas not only represented the technical advancement of the synthetic media space but also created an entirely new emotional and nostalgic experience for audiences.
Likewise, the sign language interpreter model showed off its potential for expanded use in broadcasting. Overall, the project demonstrated the potential for synthetic humans to create inclusive, emotionally resonant, and visually appealing content. The published technical paper, as part of the Final POC Results, details the key points of success and lists policy guidelines that emphasize the project's contribution to AI and sustainable production in synthetic media.
LinkedIn Co-founder, Microsoft Board Member, Investor, Podcaster
TV presenter and adventurer
Head of Post Production at Blumhouse
Co-Director In Event of Moon Disaster
President and co-founder of Veritone
Media, Communication & Film Studies, Muhlenberg College
Sound Designer and Supervisor, Actress
Emmy Award Winning Animation Producer
Joseph López Osorio
Marketing Technology Director, DDB Latina
President & CEO, DDB Puerto Rico
Producer & President at Italus Films
Director, 141 Productions
EVP, Legion Creative
Producer, Renegade Animation, Inc.
Line Producer, Religion of Sports
Voice Over Actor & Coach
Subscribe to our newsletter
Sign up to receive email updates on exclusive content and new product announcements.