Respeecher's AI Voice Lab is a team of amazing, daring, and creative sound specialists who go beyond basic voice conversions. We asked them to share - and demonstrate - some handy tips.
In this article, our Synthetic Speech Artist Ihor Isakov will show 3 different ways of denoising audio for a better voice conversion using Resemble.ai, REAPER, and Respeecher Marketplace.
Why denoise the audio before voice conversion?
In a perfect world, the source voice for a high-quality speech-to-speech conversion is recorded in a professional studio and there's nothing in the audio quality that compromises the output.
This, however, isn't always the case. When background noise, hum, or audio artifacts are mixed into your audio, the AI can't cleanly separate these important vocal features from the unwanted sounds - that is, of course, if you don't follow our tips on how to record good source audio.
It's almost like trying to trace a drawing through dirty glass - the noise acts like smudges and scratches that make it hard to see the actual lines you want to copy. The AI might end up trying to convert the noise along with the voice, leading to poor-quality results - and that's something we at Respeecher don't find acceptable.
The importance of balance
So, the less noise, the better, right? Well, not really.
Excessive denoising removes essential voice details that the AI needs for accurate conversion. When you strip away too much audio information, you lose subtle vocal characteristics like natural breathiness, micro-fluctuations in pitch, and harmonic overtones that make voices sound human and unique.
Imagine that you are editing a photo and choose to over-sharpen it. You end up removing the texture and creating artificial-looking results. The AI relies on these fine details to create realistic voice mappings, so excessive denoising produces robotic or unnatural conversions.
The sweet spot is clean but not sterile - you want to remove interference while preserving the natural imperfections that make voices authentic.
3 affordable ways to denoise audio
Now that we've established that we want our source audio clean - but not too clean - let's explore some amazing software that can help us achieve this result.
1. Respeecher Marketplace
First up, the easiest way: use the denoise input audio feature on Respeecher Voice Marketplace:
- Sign in to Marketplace
- Click on the Settings icon
- Under Audio Settings, toggle the Denoise Input Audio on.
2. Resemble.ai
Next, we'll explore an AI-powered audio enhancer. This example uses Resemble Enhance in denoising mode:
- Go to resemble.ai/enhance
- Scroll down to Try it yourself! section.
It supports audio files up to 30 seconds long. To enhance larger files, you'll need to sign up. - Drag & drop or upload your file in the Input Audio section.
- Click Submit.
- To save the cleaned audio, click the Download icon in the Output Denoised Audio section.
Here's how the original version compares with the denoised one:
3. REAPER
For the final method, we'll take a more hands-on approach. This involves using ReaFir - a built-in plugin inside REAPER, a digital audio workstation available with a free evaluation license.
- Drag the audio file into REAPER - it will automatically create a new track.
- Click the FX button to explore the plugins.
- In the window that opens, type ReaFir in the Filter box.
- Click on the line that says VST: ReaFir and click Add.
ReaFir functions as an EQ and dynamics processor. In this case, it will be used as a denoiser. - Define what you consider unwanted noise by building a noise profile. Highlight a section of audio that only contains background noise - no voice, breathing, or clicks.
- Change the Mode to Subtract.
- Enable the checkbox labeled Automatically build noise profile.
- Press Play.
The plugin will analyze the noise and subtract it from the entire track. - Once the profile is built, uncheck the box from Step 7 to lock it in.
Let's compare the original and clean versions to hear the difference:
