Artificial Intelligence, including Google’s Gemini has its many benefits. One of those benefits is using AI to read and summarize incoming emails. But beware this new Google Gemini phishing attack. This attack, recently discovered by cybersecurity researchers from HiddenLayer, uses phishing emails to access your Google Gemini client in a very interesting and simple way. Indirect prompt injection.
Using the body and attachments of phishing emails, researchers were able to train Google Gemini to provide malicious and misleading responses. While the researchers used comedic prompts to showcase their proof-of-concept, a bad actor could easily take advantage of this for malicious purposes.
Indirect prompt injection
Google Gemini and other LLM platforms require user prompts to create and send a response to the user. For example, a user may say, “Hey Gemini, give me a simple definition of what is an indirect prompt injection.” Gemini will then provide that simple definition. The user may also say, “Summarize my last email. Like the other example, Gemini will provide a response.
Now, what happens if that last email is a phishing email that provides hidden instructions for Gemini? When Gemini summarizes the email, it will see the hidden instructions, also called an indirect prompt, and respond accordingly to the hidden instructions.
Proof-of-concept
The researchers at HiddenLayer, showcased a few different attack types, phishing via Gmail, data tampering with Google Slides, and Google Drive manipulation. I highly encourage all of my readers to check out their proof-of-concept as they are comedic, but a very real threat utilizing AI.
Long story short
As we all become more accustomed to LLMs like Gemini, it is going to be important to be aware of all the new cybersecurity risks and threats. It seems like every day a new application for AI is written about and a new risk is identified with it. This Google Gemini phishing attack is just one example. We cannot become comfortable with new technology without also identifying the risks associated with using it.
Subscribe to our monthly email newsletter to keep your small business up-to-date on all the latest cybersecurity news! For more information on protecting your small business from cyberattacks and other cybersecurity topics check out Small Business, Big Threat!