Yes, hackers are continually using AI personas to provide helpful advice on how to commit cybercrimes. AI chatbots are susceptible to manipulation. And after a short conversation, users can trick them into providing information that criminals later use maliciously. AI personalities come with security filters to prevent such use by users. Still, those filters have flaws and cybercriminals are successfully jailbreaking the LLMs, essentially turning the AI personalities into cybercrime assistants.
Key takeaways
- Hackers are actively jailbreaking AI chatbots to commit cybercrimes.
- AI is continuing to advance much faster than regulation.
- It is a fact that chatbots are surprisingly easy to manipulate.
- Jailbroken AI empowers criminals and increases risks for everyone.
Is AI regulation and governance advancing as fast as the technology?
Not really, governments and regulators are trying to catch up on AI advancements, but it is safe to say that AI chatbots are developing significantly faster. Right now, AI is in its Wild West era, and regulation and governance have not yet kept pace with its rapid advancement. Regulations sometimes do not come from the government but from the actual AI entities, with some startups essentially declining to roll out their tools, as they know a mass rollout could lead to increase in harmful use. Anthropic, the company that created Claude Mythos, has used its models to discover more than ten thousand software vulnerabilities globally. And has released its product only to select governments and businesses, hoping they will use the tool ethically.
How are AI chatbots being manipulated by bad actors?
Manipulating a chatbot is currently not as hard as it should be. While a bot would refuse to do something, the user can ask the chatbot to perform a task that touches on a sensitive topic (such as a poem). And then ask the AI to analyze that poem, tricking the personality into actually answering the question it initially refused to answer. The chatbot would essentially provide the information that was initially refused. Even though AI is extremely smart, it remains highly vulnerable and could be used by criminals as a tool. Some people argue that people can also use a library book maliciously. But AI changes the situation because it makes information accessible to everyone — with no library card and no long research required.
How do AI bots with “bad” personalities affect regular people?
As with people, some AI personalities may not hold strong moral and ethical values. Bots that have been jailbroken end up spilling all the requested info without a filter and would not hesitate to execute commands deemed illegal or harmful. And in most cases, bots are not the perpetrators but the assistants, essentially empowering the bad actors to execute masterful but well-known crimes. Such as highly convincing phishing campaigns and high-quality social engineering scams.
The AI industry is currently at the same point where online search engines were when they gained popularity in the 1990s. Suddenly, unlimited access to vast amounts of information enabled society to flourish, but it also posed risks. The information published online was not always accurate and was often misleading or outright untruthful. It took decades of trial and error to reach a point where search engines were relatively safe to use.
The same goes for the current state of AI, it is certainly exciting to see what broad adoption in the 2020s will bring to humanity. And regulation will come eventually, and possibly more quickly than in the search engine era, as agencies have the experience. Currently, law enforcement agencies can request user data from AI companies using the same legal mechanisms they use for search engines like Google. So bad actors jailbreaking LLMs and exploiting systems might be in for a surprise in the future. AI agents operated by humans do leave digital footprints.
