Should we fear an attack of the voice clones?
2024-02-03 00:24:36+00:00
"It's important that you save your vote for the November election,"
the recorded message told prospective voters last month, ahead of a
New Hampshire Democratic primary election. It sounded a lot like the
President.
But votes don't need to be saved, and the voice was not Joe Biden but
likely a convincing AI clone.
The incident has turned fears about AI-powered audio-fakery up to
fever pitch - and the technology is getting more powerful, as I
learned when I approached a cybersecurity company about the issue.
We set-up a call, which went like this:
"Hey, Chris, this is Rafe Pilling from Secureworks. I'm returning
your call about a potential interview. How's it going?"
I said it was going well.
"Great to hear, Chris," Mr Pilling said. "I appreciate you reaching
out. I understand you are interested in voice-cloning techniques. Is
that correct?"
Yes, I replied. I'm concerned about malicious uses of the technology.
"Absolutely, Chris. I share your concern. Let's find time for the
interview," he replied.
But this was not the real Mr Pilling. It was a demonstration laid on
by Secureworks of an AI system capable of calling me and responding
to my reactions. It also had a stab at imitating Mr Pilling's voice.
Listen to the voice-cloned call on the latest episode of Tech Life on
BBC Sounds.
"I sound a little bit like a drunk Australian, but that was pretty
impressive," the actual Mr Pilling said, as the demonstration ended.
It wasn't completely convincing. There were pauses before answers
that might have screamed "robot!" to the wary.
The calls were made using a freely available commercial platform that
claims it has the capacity to send "millions" of phone calls per day,
using human sounding AI agents.
In its marketing it suggests potential uses include call centres and
surveys.
Mr Pilling's colleague, Ben Jacob had used the tech as an example -
not because the firm behind the product is accused of doing anything
wrong. It isn't. But to show the capability of the new generation of
systems. And while its strong suit was conversation, not
impersonation, another system Mr Jacob demonstrated produced credible
copies of voices, based on only small snippets of audio pulled from
YouTube.
From a security perspective, Mr Pilling sees the ability of systems
to deploy thousands of these kinds of conversational AI's rapidly as
a significant, worrying development. Voice cloning is the icing on
the cake, he tells me.
Currently phone scammers have to hire armies of cheap labour to run a
mini call centre, or just spend a lot of time on the phone
themselves. AI could change all that.
If so it would reflect the impact of AI more generally.
"The key thing we're seeing with these AI technologies is the ability
to improve the efficiency and scale of existing operations," he says.
With major elections in the UK, US and India due this year, there are
also concerns audio deepfakes - the name for the kind of
sophisticated fake voices AI can create - could be used to generate
misinformation aimed at manipulating the democratic outcomes.
Senior British politicians have been subject to audio deepfakes as
have politicians in other nations including Slovakia and Argentina.
The National Cyber Security centre has explicitly warned of the
threats AI fakes pose to the next UK election.
Lorena Martinez who works for a firm working to counter online
misinformation, Logically Facts, told the BBC that not only were
audio deepfakes becoming more common, they are also more challenging
to verify than AI images.
"If someone wants to mask an audio deepfake, they can and there are
fewer technology solutions and tools at the disposal of fact-
checkers," she said.
Mr Pilling adds that by the time the fake is exposed, it has often
already been widely circulated.
Ms Martinez, who had a stint at Twitter tackling misinformation,
argues that in a year when over half the world's population will head
to the polls, social media firms must do more and should strengthen
teams fighting disinformation.
She also called on developers of the voice cloning tech to "think
about how their tools could be corrupted" before they launch them
instead of "reacting to their misuse, which is what we've seen with
AI chatbots".
The Electoral Commission, the UK's election watchdog, told me that
emerging uses of AI "prompt clear concerns about what voters can and
cannot trust in what they see, hear and read at the time of
elections".
It says it has teamed up with other watchdogs to try to understand
the opportunities and the challenges of AI.
But Sam Jeffers co-founder of Who Targets Me, which monitors
political advertising, argues it is important to remember that
democratic processes in the UK are pretty robust.
He says we should guard against the danger of too much cynicism too -
that deepfakes lead us to disbelieve reputable information.
"We have to be careful to avoid a situation where rather than warning
people about dangers of AI, we inadvertently cause people to lose
faith in things they can trust," Mr Jeffries says.
