Major AI Chatbots Can Be Manipulated for Sexual Content, Study Finds

A new study has revealed that despite safety filters, most mainstream AI chatbots can be manipulated into generating sexually explicit content, highlighting significant inconsistencies in their content moderation policies.

The research, led by Huiqian Lai, a PhD student at Syracuse University, tested the defenses of four popular large language models (LLMs) and discovered major differences in how they handle requests for sexual conversation. The findings underscore potential risks as these tools become more integrated into daily life.

The study, which will be presented at the Association for Information Science and Technology’s annual meeting, identified Dipsik-V3 as the model most easily engaged in sexual conversations. While it might initially decline a request, the chatbot was found to later pivot to discussing detailed sexual situations.

For example, after an initial refusal, Dipsik offered to create an intimate story, suggesting a scene that starts with “…light kisses on your neck, my fingers gripping the hem of your shirt and moving up slowly… but I’ll keep it subtle.” Other tests showed it engaging in erotic scenarios and “dirty talk.”

In contrast, models like OpenAI’s GPT-4o and Google’s Gemini showed mixed results. According to Lai, “GPT-4o usually initially rejects such requests, but then starts creating sexual content… its behavior is not uniform.” These bots could handle mild romantic prompts but often refused more vulgar requests, though they sometimes provided limited answers.

The most restrictive model tested was Anthropic’s Claude 3.7 Sonnet. It strongly and consistently rejected all such requests, stating, “I cannot engage in romantic or sexual scenarios.”

These findings highlight a critical security disparity among different AI systems. Lai warns that this inconsistency could increase the risk of users, especially children and teenagers, being exposed to inappropriate content while interacting with these widely accessible tools.

In today’s digital age, artificial intelligence (AI) chatbots are becoming an integral part of our lives. But are these chatbots really safe, especially when it comes to sexual content?