Can AI Chatbots like ChatGPT Help Detect Phishing Scams?

Kaspersky experts have conducted research on ChatGPT, an AI-powered language model, to test its ability to detect phishing links. While ChatGPT is known for its ability to create phishing emails and write malware, its effectiveness in detecting malicious links was limited. The study found that ChatGPT had a high false positive rate of up to 64 percent and often produced imaginary explanations and false evidence to justify its verdicts.

The experiment involved testing the model on more than 2,000 links that Kaspersky anti-phishing technologies deemed phishing and mixing them with thousands of safe URLs. The results showed that ChatGPT had a detection rate of 87.2 percent and a false positive rate of 23.2 percent when asked if a link leads to a phishing website. When asked if a link is safe to visit, ChatGPT had a higher detection rate of 93.8 percent but a higher false positive rate of 64.3 percent.

Although the detection rate was high, the false positive rate was too high for any production application. However, ChatGPT showed impressive results in identifying potential phishing targets, successfully extracting targets from over half of the URLs, including major tech portals like Facebook, TikTok, and Google, marketplaces such as Amazon and Steam, and numerous banks from around the world, without any additional training.

The experiment also revealed that ChatGPT had limitations in proving its point on whether the link is malicious, often providing misleading explanations despite the confident tone. However, ChatGPT still has potential in assisting human analysts in detecting phishing attacks, although language models still have limitations and tend to produce random output.

“ChatGPT certainly shows promise in assisting human analysts in detecting phishing attacks but let’s not get ahead of us – language models still have their limitations. While they might be on par with an intern-level phishing analyst when it comes to reasoning about phishing attacks and extracting potential targets, they tend to hallucinate and produce random output. So, while they might not revolutionize the cybersecurity landscape just yet, they could still be helpful tools for the community,” comments Vladislav Tushkanov, Lead Data Scientist at Kaspersky.

Kaspersky’s ML team is at the forefront of applying machine learning technologies to cybersecurity tasks, constantly updating Kaspersky products with the latest tech and intel. To take advantage of Kaspersky’s expertise in machine learning and stay protected, the company’s experts recommend:

• For corporate cybersecurity, Kaspersky Managed Detection and Response is an essential tool capable of detecting and preventing intrusions in their initial stages. It utilizes advanced machine-learning models to filter out mundane events and sends only alarming ones to professional human analysts. This service enhances a company’s ability to withstand cyber threats while optimizing the use of existing workforce resources.
• Providing your staff with basic cybersecurity hygiene training is crucial. Conducting simulated phishing attacks can also help ensure that they know how to distinguish phishing emails.
• Lastly, using the latest Threat Intelligence information to stay aware of actual TTPs (tactics, techniques, and procedures) used by threat actors is also recommended to enhance cybersecurity.