Respeecher Revives_Communication_Capabilities_for_a_Patient_with_Friedreichs_Ataxia_Respeecher_Voice_Cloning_Technology

Respeecher Revives Communication Capabilities for a Patient with Friedreich's Ataxia

   

Speech is a uniquely human ability that we often take for granted. When delivering healthcare solutions for individuals with speech disabilities, we meet a lot of extraordinary people who have had to re-evaluate how they live their lives without the ability to speak.

This time, we met Joseph Boon, a New Zealander. He was diagnosed with Friedreich's Ataxia along with his brother and sister. Despite the very unpleasant prognosis, Joseph and his family wanted to find ways to improve their quality of life. After some searching, Joseph came upon Respeecher. And that’s when our collaboration began.

 

The Challenge

 

Friedreich's Ataxia is a rare genetic disorder that affects some of the body's nerves. Symptoms often begin in late childhood and can include trouble walking, fatigue, changes in sensation, and slowed speech. The symptoms of Friedreich's ataxia usually get worse as time passes. The disorder affects Joseph’s coordination, hearing, and speech.

Before turning to Respeecher, he used special technology to help him hear clearly. “Although my ears pick up sound well enough, the transmission of that audio information to my brain is like heavy traffic going over a bridge in a hurricane. It takes time and effort to wade through the mess of sound and perceive what I want or need. Of course, this applies to the sound I make too”, explains Joseph.

Due to hearing issues, he can’t perceive his voice clearly. Many years ago he started using a small directional microphone connected to headphones to follow conversations in noisy environments and to hear his own voice more clearly.

Joseph’s speech began to noticeably degrade a decade ago. Today, he can still be understood but he cannot sing in tune, or hold a tune at all.

“I work hard to maintain the ability to communicate because it’s more important to me than almost anything else, even the ability to walk. But, just like walking, my voice will keep being challenged until I run out of ways to adapt. That is what brought me to Respeecher.”

Like other patients that lose their ability to speak, Joseph decided to try voice cloning technologies and figure out which ones work best for him. However, he was always concerned that it means that he has to use someone else’s voice. With Respeecher he finally found a chance to recreate his own voice.

Respeecher Revives_Communication_Capabilities_for_a_Patient_with_Friedreichs_Ataxia_Respeecher_Voice_Cloning_Technology-1

 

The Solution

 

In order to compile an accurate model of Joseph’s voice, we usually require around 30 minutes of recorded speech. However, in the case of patients, we ask them to provide any samples they already had prior to the deterioration of their speech. Luckily, Joseph was a presenter at a local radio station and had some recordings of himself back when he was live on the air. We then used these recordings to clone his voice.

Once we had the samples, we began working on creating an AI model of Joseph’s target voice.

Respeecher Revives_Communication_Capabilities_for_a_Patient_with_Friedreichs_Ataxia_Respeecher_Voice_Cloning_Technology-2

 

The Result

 

As a result, Joseph was presented with a voice model that was indistinguishable from the speech he had before his original voice began to fade. He uses this model for his vocal exercises.

“I recorded an old school friend and converted his voice to mine. I got him to read a selection of literature I like, but find it a challenge to read aloud myself. He is a trained actor, his accent is close to mine, and I like the rhythm of his speech. The result is me. Well, what I would sound like without Friedrich’s Ataxia. I listen, and recite the same words, trying to match the rhythm, pace, pitch etc. I do this repeatedly.”, says Joseph.

Such training is helping Joseph maintain his voice. “Doing this through Respeecher I am trying to sound more like myself rather than trying to sound like my friend, or someone else.”, he adds.

So currently, Joseph is using our Text-to-speech solution as Speech-to-speech can't help him fix his speech impairments. However, he is going to leverage STS as well, for his creative projects with the help of voice actors.

Having this tool is a significant emotional boost for Joseph as it allows him to reclaim his vocal creativity. Today, he is part of a creative community and is planning to record voiceovers for his radio screenplays using scripts he’s written.

We have added Joseph’s voice to the Respeecher voice marketplace. In this way, Joseph’s voice will be able to take on many incarnations, giving life to a number of different new projects.

As technology develops, Joseph hopes to use his voice model for direct communication.

Respeecher has created voice solutions for different healthcare cases, such as giving voice to Michael York and helping patients that went through larynx removal. Aside from laryngeal cancer, other disorders that could be candidates for Respeecher’s voice revival include Parkinson’s disease, amyotrophic lateral sclerosis, multiple sclerosis, vocal fold paralysis, pharyngeal cancer, and dysarthria.

We are honored to be able to work with such strong people like Joseph. Thanks to people like him, we are able to continue improving our technology and help more people who are suffering from voice disabilities.

 

Reviews

review_avatar

Halsey Burgund

Co-Director In Event of Moon Disaster

review_avatar

Ryan Steelberg

President and co-founder of Veritone

review_avatar

David Romberg

Media, Communication & Film Studies, Muhlenberg College

review_avatar

Abigail Savage

Sound Designer and Supervisor, Actress

review_avatar

Michael York

Actor, Producer

review_avatar

Tim Jones

Emmy Award Winning Animation Producer

review_avatar

Joseph López Osorio

Marketing Technology Director, DDB Latina

review_avatar

Edgardo Rivera

President & CEO, DDB Puerto Rico

review_avatar

Joshua Madoff

Composer, Actor

review_avatar

James Tumminia

Producer & President at Italus Films

review_avatar

141 Productions

Production & Distribution Group

We worked with Respeecher on a film called ‘In Event of Moon Disaster’ first shown at the Amsterdam Documentary Film Festival 2019. They helped us create a synthetic voice of Richard Nixon to bring to life a never-read contingency speech in case the Apollo 11 mission went badly. We created a highly realistic film, in a large part due to their work. ‘In Event of Moon Disaster’ shows the creative possibility of voice replacement technology as well as highlighting just how realistic deepfake technologies can be, acting as a civic engagement project for the public along with contextual and educational resources.
For Veritone, the most important thing is ensuring our clients get access to technology without compromising on the ability to protect, the delivery speed and quality, and in this case, of the generated content. Respeecher's technology meets both requirements and has a proven track record of working with tier one projects.
When I approached the team at Respeecher with a very sensitive and personal documentary project, I didn't know what to expect. I’m happy to say that it has been an amazing experience. They are extremely professional, ethically minded and are genuinely passionate about bringing their innovative technology to the world. Their approach to the voice cloning process is highly personal and transparent. As a documentarian, it was important to me that the approach was ethical and human focused. The Team at Respeecher delivered on their promises and I could not be happier with the results.
Respeecher is a remarkable tool for Sound Editors. It delivers very high-fidelity recreations of a target voice, with transparent performance-matching of its source. It blows text-to-speech out of the water! The effect is uncanny and incredibly effective and I can imagine a whole slew of uses going forward. I am very excited to have discovered Respeecher, and it will be my go-to for voice recreation in the future, without question.
I am very grateful to Respeecher for giving my words and work a whole new lease on life. The technique involved is as ingenious as it is amazing, and promises a great future for all those involved in voice work. With Respeecher, the quality of the re-worked passages is indistinguishable from the original. History comes almost miraculously alive again. I heartily endorse this product.
Respeecher's source/target conversion of Mr. York's speaking voice demonstrably opens doors for the entertainment, education, and healthcare fields. Suffice it to say, it was an absolute honor to collaborate with them. Their brilliant accomplishment and timely delivery have allowed our humble project to be a success, exceeding our expectations for what's even possible.
Discovering Respeecher was the highlight of our project. They agreed to work with the task even when our training data was not of the best quality. The result was fantastic, and the level of detail was just outstanding.
Working with Reespecher was a unique experience. Their attentiveness to detail and dedication to this project allowed us to reconnect with one of our lost narration idols in Puerto Rico "Manuel Rivera Morales" on a new level. It gave his family and PR a glimpse of what could have been if he was still alive and narrating all sorts of Olympic sports games. It was truly magical to see this project come alive.
Working with Respeecher has been incredible. Their team was meticulous and patient, walking us through the process of developing an accurate model... and the final result was nothing short of miraculous. It really sounds like we’re hearing our (deceased) subject speaking again. Something I never would have thought was possible.
From the introduction to Respeecher’s innovative and state-of-the-art products to the thorough and quick technical support, I immediately felt that I found the perfect partner and technology for my projects’ needs. During our discussions, the team offered other creative options that assured me we were on the same page and had my project’s success as a top priority. The Respeecher team members are consummate professionals.
Making a film which was so sensitive we needed a company who understood the sensibilities and ethical issues we were dealing with. From the first meeting with them, Respeecher acted with respect and professionalism, delivered what we wanted brilliantly, efficiently and to spec.
review_avatar
review_avatar
review_avatar
review_avatar
review_avatar
review_avatar
review_avatar
review_avatar
review_avatar
review_avatar
review_avatar

Ready to Replicate Voices?

Want to see our technology in action? Get a demo today.

REQUEST A DEMO