• There are no suggestions because the search field is empty.
Respeecher_Collaborates_with_a_Young_Researcher_to_Advance_Voice_Capabilities_for_Laryngectomy Patients_Respeecher_Voice_Cloning_Software-2

Respeecher Collaborates with a Young Researcher to Advance Voice Capabilities for Laryngectomy Patients

 

Voice conversion is a new technology that is capable of drastically changing the way people with speech disabilities function in their everyday lives. 

The Respeecher team is constantly looking for new ways to advance its technology. On our journey, we met with so many extraordinary people. One of them was a young scientist named Konrad Zieliński, a Ph.D. student at the University of Warsaw who had lost his voice due to laryngectomy. His need for novel solutions for people in similar situations was the inspiration for the topic of his thesis, the main focus of his company, Uhura Bionics, and the beginning of our fruitful collaboration.

A laryngectomy is the surgical removal of the larynx (or voice box). It is usually done to treat severe or advanced-stage cases of laryngeal cancer. As a result of the surgery, patients lose their ability to speak and have to rely on voice-assistance technologies such as an electrolarynx or tracheoesophageal voice prosthesis (TEP).


However, there are certain challenges regarding these two ways of “fixing” voice disabilities.


Today, patients must use various noise-reduction techniques and filtering systems to improve speech intelligibility with an electrolarynx. At present, the technology responsible for improving electrolaryngeal speech intelligibility still has a long way to go. And while TEP speech is more natural than electrolarynx speech, it is still much less natural sounding than normal laryngeal speech.


Both of these methods seek to restore a patient's ability to speak. The only issue is the low quality of voice that these technologies produce. Communication difficulties affect patients in their jobs, personal relationships, and social gatherings. 

Konrad is looking for different ways to restore patients’ original voices after having their larynx removed, and that’s why he decided to try Respeecher.

We interviewed Konrad to learn more about his research plans, his experience using Respeecher, and his future plans regarding solutions for voice disorders.

Respeecher: Can you tell us a little bit about the work you are doing right now?

Konrad: I am a PhD Student at the University of Warsaw, I work in the Human Interactivity and Language Lab (HILL). Right now I am working with a grant that has allows me to evaluate various bionic systems for laryngectomees.

I recently had been published at two prestigious conferences on human-computer interaction and speech technology: ACM CHI 2022 in New Orleans, US and INTERSPEECH in Icheon, Korea. 

Currently, I am developing my own company Uhura Bionics with technical co-founder Marek Grzelec. We are participating in the EIT Health Patient Innovation Bootcamp, distributing voice amplifying speakers for people with voice disorders. We are also working on a novel electronic larynx that generates more elegant speech with a more natural sound.

We are natural partners for Respeecher and we feel like we have a lot to do together.

R: What is the eventual goal you are trying to accomplish with Respeecher? Is it real-time voice conversion?

K: Yes. We are striving to achieve voice conversion in 50 millisecond, so that it is imperceptible to the human ear. However, it is a very ambitious goal, and there are still many technical challenges ahead of us.

R: Speaking of now, with the current level of the technology Respeecher has, how are you going to use it for your recent projects?

K: I consider two scenarios.

One is as an assistive tech for content production. So that people with voice disabilities could produce different types of audio and video content, such as lectures, voiceovers, advertisements, and many more.

Another scenario is related to one of the biggest issues people with voice disabilities have - communicating with someone directly or by phone. Devices designed to aid communication (as electrolarynx or TEP) usually make speech sound robotic and unrecognizable, especially when talking to someone by phone.  

R: Do you consider this technology to be available for the wide scope of people affected by voice disabilities? 

K: When the technology is more developed, I would say in 5-10 years, then of course, it will become more available for a larger number of patients with voice problems. The combination of the devices we are developing right now and the Respeecher technology will make the lives of people suffering from speech problems much easier. 

R: What are your thoughts after trying Respeecher?

K: Respeecher created a software that allows me to change my voice with electrolarynx to sound more human-like. They even utilized recordings from before my laryngectomy surgery (4 years ago) to build a dedicated voice conversion system that resembles my old voice! I can’t describe how excited I am about the current results and our future work together!

How does Respeecher's Technology for Patients with Voice Disabilities Work

 

In order to help laryngectomy patients achieve a higher quality of life, Respeecher is exploring the use of its technology on voice samples of the electrolarynx and tracheoesophageal voice. The solution will offer real-time intelligible voice replacement to improve support specifically for individuals who have undergone a laryngectomy.

Respeecher’s voice-changing technology transforms the sound of electrolarygeal and TEP speech into clearer, more articulated, and more intelligible audio. In particular, the technology dampens the mechanical hum of the electrolarynx and TEP voices while accentuating the natural tonal inflections. This makes it much easier to communicate, both for the patient and their interlocutors.

The technology could be deployable using a modified phone. Patients can communicate live through a speaker or use the technology for phone calls and other electronic voice communications.

Respeecher_Collaborates_with_a_Young_Researcher_to_Advance_Voice_Capabilities_for_Laryngectomy Patients_Respeecher_Voice_Cloning_Software

The Result and Future Steps

 

Today, Respeecher’s voice cloning algorithms deliver critical benefits when employed as an assistive technology to those who have lost their natural ability to speak.  

Konrad provided the samples of his voice for Respeecher to create his voice model and started testing the Voice Marketplace in real-time and offline voice conversions. The results of the collaboration create an example of how laryngectomy patients can use Voice Marketplace independently. 

 

With the help of voice cloning, as the partnership with Konrad showed, the patients are able to communicate in a more natural manner as well as produce different types of audio and video content, such as lectures, voiceovers, advertisements, and many more.

Reviews

review_avatar

Reid Hoffman

LinkedIn Co-founder, Microsoft Board Member, Investor, Podcaster

review_avatar

Bear Grylls

TV presenter and adventurer

review_avatar

Jennifer Trent

Head of Post Production at Blumhouse

review_avatar

Halsey Burgund

Co-Director In Event of Moon Disaster

review_avatar

Ryan Steelberg

President and co-founder of Veritone

review_avatar

David Romberg

Media, Communication & Film Studies, Muhlenberg College

review_avatar

Abigail Savage

Sound Designer and Supervisor, Actress

review_avatar

Tim Jones

Emmy Award Winning Animation Producer

review_avatar

Michael York

Actor, Producer

review_avatar

Joseph López Osorio

Marketing Technology Director, DDB Latina

review_avatar

Edgardo Rivera

President & CEO, DDB Puerto Rico

review_avatar

Joshua Madoff

Composer, Actor

review_avatar

James Tumminia

Producer & President at Italus Films

review_avatar

Sophie Robinson

Director, 141 Productions

review_avatar

Steve Isaacs

EVP, Legion Creative

review_avatar

Derek Kedell

Producer, Renegade Animation, Inc.

review_avatar

Jacob Mosler

Line Producer, Religion of Sports

review_avatar

Bev Standing

Voice Over Actor & Coach

Respeecher also matched my voice to Scott’s professional reading of the Impromptu text. The end result is a nearly perfect audio artifact with amazing intonation. It sounds flawlessly human—though a bit less of a match to “my” voice than the Vall-E recording—and most importantly, similarly didn’t require multiple days of my time to record. If we hadn’t prioritized releasing an early version to show folks what that sounded like, the Respeecher team was confident that with more iterations we could have gotten it near-perfect.
We worked with Respeecher to recreate my voice in Ukrainian to as part of our education initiative ‘BecomingX Ukraine’. The technology is phenomenal and the result is astounding – I never thought I’d hear myself speaking fluent Ukrainian! Well done and thank to all the team for all your incredible work you are doing. We’re forever grateful and proud to partner with you.
We used Respeecher to de-age the voice of a well-known actor in one of our movies. We were nervous that it wouldn’t work very well since the actor is in her 80s and we needed to de-age her down to mid-forties. Even though there was such a huge age jump, the end result from Respeecher was amazing, and added a level of authenticity to our movie. Everyone from our director through our entire post team was excited to work with Repeecher. Hopefully we will be able to do so many more times in the future.
We worked with Respeecher on a film called ‘In Event of Moon Disaster’ first shown at the Amsterdam Documentary Film Festival 2019. They helped us create a synthetic voice of Richard Nixon to bring to life a never-read contingency speech in case the Apollo 11 mission went badly. We created a highly realistic film, in a large part due to their work. ‘In Event of Moon Disaster’ shows the creative possibility of voice replacement technology as well as highlighting just how realistic deepfake technologies can be, acting as a civic engagement project for the public along with contextual and educational resources.
For Veritone, the most important thing is ensuring our clients get access to technology without compromising on the ability to protect, the delivery speed and quality, and in this case, of the generated content. Respeecher's technology meets both requirements and has a proven track record of working with tier one projects.
When I approached the team at Respeecher with a very sensitive and personal documentary project, I didn't know what to expect. I’m happy to say that it has been an amazing experience. They are extremely professional, ethically minded and are genuinely passionate about bringing their innovative technology to the world. Their approach to the voice cloning process is highly personal and transparent. As a documentarian, it was important to me that the approach was ethical and human focused. The Team at Respeecher delivered on their promises and I could not be happier with the results.
Respeecher is a remarkable tool for Sound Editors. It delivers very high-fidelity recreations of a target voice, with transparent performance-matching of its source. It blows text-to-speech out of the water! The effect is uncanny and incredibly effective and I can imagine a whole slew of uses going forward. I am very excited to have discovered Respeecher, and it will be my go-to for voice recreation in the future, without question.
Respeecher's source/target conversion of Mr. York's speaking voice demonstrably opens doors for the entertainment, education, and healthcare fields. Suffice it to say, it was an absolute honor to collaborate with them. Their brilliant accomplishment and timely delivery have allowed our humble project to be a success, exceeding our expectations for what's even possible.
I am very grateful to Respeecher for giving my words and work a whole new lease on life. The technique involved is as ingenious as it is amazing, and promises a great future for all those involved in voice work. With Respeecher, the quality of the re-worked passages is indistinguishable from the original. History comes almost miraculously alive again. I heartily endorse this product.
Discovering Respeecher was the highlight of our project. They agreed to work with the task even when our training data was not of the best quality. The result was fantastic, and the level of detail was just outstanding.
Working with Reespecher was a unique experience. Their attentiveness to detail and dedication to this project allowed us to reconnect with one of our lost narration idols in Puerto Rico "Manuel Rivera Morales" on a new level. It gave his family and PR a glimpse of what could have been if he was still alive and narrating all sorts of Olympic sports games. It was truly magical to see this project come alive.
Working with Respeecher has been incredible. Their team was meticulous and patient, walking us through the process of developing an accurate model... and the final result was nothing short of miraculous. It really sounds like we’re hearing our (deceased) subject speaking again. Something I never would have thought was possible.
From the introduction to Respeecher’s innovative and state-of-the-art products to the thorough and quick technical support, I immediately felt that I found the perfect partner and technology for my projects’ needs. During our discussions, the team offered other creative options that assured me we were on the same page and had my project’s success as a top priority. The Respeecher team members are consummate professionals.
Making a film which was so sensitive we needed a company who understood the sensibilities and ethical issues we were dealing with. From the first meeting with them, Respeecher acted with respect and professionalism, delivered what we wanted brilliantly, efficiently and to spec.
When Legion Creative took on the challenge of bringing MTV’s Kurt Loder back to his mid ’90s glory for the Showtime original Yellowjackets, we needed a resource to bring his voice in line with his de-aged visual appearance. Respeecher worked quickly, with clear communication and helped us achieve the time-warping magic we were looking for.
Using the Respeecher technology as a powerful filter for our recorded dialogue allowed us to quickly solve a voice-over challenge that could have completely derailed our production timeline and set us back several months. Their easy-to-use online web interface allowed us to resolve our problem in a matter of weeks and with very minimal impact on our overall schedule. Their friendly staff were extremely responsive to our feedback and provided excellent guidance on the use of the amazing technology.
Respeecher brought the voice of WILT CHAMBERLAIN sonically back to life allowing Wilt to posthumously narrate his own documentary series GOLIATH [Showtime & Paramount+]. Our experience working with their team was a pleasure from start to finish. They embraced the creative challenge of building a practically perfect replication of Wilt's voice from hours of archival to carefully match the narration led by our voiceover actor's performance of Wilt's own words selected from his books. Their technicians paid fastidious attention to detail and contineud to work with our directors feedback to adjust the nuance of Wilt's voice until it was seemingly flawless. The Estate of Wilt Chamberlain authorized our team to create a 3-hour series documenting his extraordinary life; so naturally it was a huge relief when we shared the final WILT converted VO narration and received a unanimous approval from his family. Thank you again to Respeecher! Your professionalism, commitment to detail and excellence surpassed our expectations. Your nuanced work brought Wilt's voice back to life will be cherished in GOLIATH eternally.
Working with Respeecher has provided me to control of where my voice is used, the opportunity to provide consent to my voice being used and receveing compensation.
review_avatar
review_avatar
review_avatar
review_avatar
review_avatar
review_avatar
review_avatar
review_avatar
review_avatar
review_avatar
review_avatar
review_avatar
review_avatar
review_avatar
review_avatar
review_avatar
review_avatar
review_avatar

Ready to Replicate Voices?

Want to see our technology in action? Get a demo today.

REQUEST A DEMO