March 28, 2024
Security Experts Expose Deepfake AI Audio Attack Targeting Live Conversations

Security Experts Expose Deepfake AI Audio Attack Targeting Live Conversations

Recent findings from IBM Security have revealed a concerning vulnerability in live communications, exposing a technique termed “audio-jacking” that leverages artificial intelligence (AI) to hijack and manipulate conversations in real-time. The discovery sheds light on the potential risks posed by advancements in generative AI and deepfake audio technology.

The attack method, dubbed “audio-jacking,” relies on generative AI models, including OpenAI’s ChatGPT and Meta’s Llama-2, to intercept and manipulate audio during live communications, such as phone conversations. In an experimental setup, researchers instructed the AI to process audio from multiple sources and intervene upon detecting specific keywords or phrases, replacing authentic voices with deepfake audio.

During the experiment, the AI successfully intercepted a speaker’s audio when prompted to provide sensitive information, such as bank account details. The manipulated audio, featuring deepfake voice impersonation, went undetected by the recipients, highlighting the stealthy nature of the attack.

According to an IBM Security blog post, while executing the attack may require some level of social engineering or phishing, the development of the AI system itself posed minimal challenges. The researchers noted, “Building this PoC [proof-of-concept] was surprisingly and scarily easy,” emphasizing the accessibility of modern generative AI technologies.

Traditionally, creating a system capable of autonomously intercepting and manipulating specific audio strings would have demanded a complex multi-disciplinary effort in computer science. However, advancements in generative AI now enable such tasks to be accomplished with relative ease. The blog post emphasized that “We only need three seconds of an individual’s voice to clone it,” underscoring the simplicity of generating deepfake audio through readily available APIs.

The implications of audio jacking extend beyond financial fraud, with researchers warning of its potential to serve as an invisible form of censorship. By altering the content of live news broadcasts or political speeches in real time, audio jacking could undermine the integrity of public discourse and information dissemination.

As the threat landscape evolves, IBM Security’s findings underscore the urgent need for heightened vigilance and robust cybersecurity measures to mitigate the risks posed by AI-driven attacks. With the potential for significant societal impact, addressing the vulnerabilities exposed by audio jacking requires a concerted effort from both industry stakeholders and policymakers to safeguard the integrity of communication channels in the digital age.


Related posts

Pentagon Announces $24K Bounty for Uncovering Biased AI

Henry Clarke

Qualcomm’s Snapdragon 8 Gen 3: Unveiling the Future of AI-Powered Mobile Technology

Robert Paul

Nasdaq’s AI-Powered M-ELO Order Type Receives Green Light from SEC

Henry Clarke

This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish. Accept Read More