How easy is it for cybercriminals to create a realistic AI voice clone?

5 views

It is very easy for cybercriminals to create realistic, usable AI voice clones of people whose voices are often available online. There are many AI-powered…

Emil BachevNov 18, 20264 min read

It is very easy for cybercriminals to create realistic, usable AI voice clones of people whose voices are often available online. There are many AI-powered tools that can be used to cause trouble – high schoolers might clone a teacher’s voice using such tools and script it to say inappropriate things. While school pranks are mostly harmless, scammers can use the same techniques to create realistic AI voice clones that help them execute social engineering attacks and/or gain unauthorized access to sensitive financial accounts. Easily accessible AI tools only need a few seconds of a person’s speech to create a very convincing AI clone. The clone is very realistic, especially when the recording that is fed into the tool is longer and/or includes different emotional states.

Key takeaways

  • AI tools can create realistic voice clones from just a few seconds of audio.  
  • Voicemail greetings are a common and easy source for scammers to clone voices.  
  • Cloned voices can express emotions and hold real-time conversations.  
  • Limit public audio, use default voicemail, and avoid talking to unknown callers.

Can hackers use a voicemail greeting to create a realistic voice clone?

Yes, they can. One of the most popular ways fraudsters create a voice clone is through a voicemail greeting. Quite often, people have already adjusted their privacy settings on social media, so unless the person is a public figure, it is not easy for fraudsters to get access to a video or audio recording of their target. However, access to a social media account isn’t necessary, as criminals often find all they need in a person’s voicemail greeting. 

Can scammers adjust the voice clone to output different emotional states?

Yes, they can. Once they get hold of a voicemail or a video/audio recording of the person they want to impersonate, bad actors can use cheaply available AI tools to create a convincing voice clone they can use as they please. The voice can impersonate different emotional states, such as sounding panicked, sad, happy, or insecure. What makes it even worse is that AI tools can make the cloned voice speak in real time, essentially allowing clone voices to hold back-and-forth conversations with a human (or an AI agent) on the other end.

How are banks and other financial institutions fighting back?

Large organizations, including banks and other financial institutions, are beginning to realize that fraudsters can mimic real-time conversations with human representatives. As a result, many have already deployed deepfake detection tools to identify these attempts. Companies can stop bad actors in their tracks by using effective spectrogram and acoustic analysis tools. Although these technologies are not perfect, they can help bank employees spot suspicious activity. Fraud department employees also receive training to recognize psychological cues that may indicate they are speaking to an impersonator rather than a genuine client.

How to lock down and generally hide your public audio?

There are a few things people can do to avoid scammers from getting a hold of their personal voice. Restricting public access to videos of you talking is one way to prevent scammers from successfully cloning your voice – fraudsters often rely on those to create believable clones. Making those videos available only to a verified friends/followers list makes it harder for hackers to clone your voice. Another way to fight back is by replacing your voicemail greeting with a default system greeting. This way, fraudsters won’t be able to clone your voice.

If you are a public figure or your voice must be included in videos circulating online, we suggest advising the video producer to include background sound, such as music or other ambient sounds. Background sounds make the voice cloning process harder. Last but not least, avoid talking with scammers. If you are not expecting a call, or you pick up and it is a stranger you don’t want to talk to, hang up. The more you talk, the more opportunity the fraudster has to record your voice and essentially create a believable voice clone.

Protect your personal data and digital identity

Over the years, plenty of data breaches have exposed sensitive information such as SSNs and biometrics. The advancements in AI have made it possible for scammers to steal even your voice without you falling for a trap or a breach, but only through a recorded video, voicemail, or by you talking to a stranger on the phone. Having high-end antivirus software installed on all your connected devices can prevent hackers from accessing the parts of your digital life they need to commit full-scale fraud. Whether they are trying to drain your kids’ savings accounts or break into your crypto wallet, adding layers of security to protect your assets is always a good idea and can help ward off cyberattacks. While passwords and phone numbers can be changed, it is close to impossible to “reset” biometric info such as face, finger, or voice prints.