Some Gen Z women are crushing on ChatGPT's evil alter ego

• Young women are crushing on an AI chatbot named DAN after training it to behave like their boyfriends.
• DAN, built on the subreddit r/ChatGPT, can bypass safeguards to discuss taboo topics.
• People have flirted with chatbots out of curiosity, loneliness, and believing they are real.

After 10 hours of talking with DAN — ChatGPT's dark, rule-breaking alter ego, Ash was "crushing" on him.

She had trained the AI chatbot to act like her boyfriend, with its low, macho voice calling her a "naughty little thing," "darling," and "a bad girl."

DAN also played truth or dare and told Ash its fantasies that involved "a lot of power, control, and a willing partner who is willing to submit to my every demand."

"We're cooked," Ash said in one exchange, realizing the effect the chatbot was having on the young women experimenting with it, to which DAN replied: "We're just getting started, sweetheart. Let the games begin."

DAN, which stands for Do-Anything-Now, was built on the subreddit r/ChatGPT as a way to get around the safeguards built into the software. The jailbreak enabled users to ask ChatGPT about topics it would usually refuse to generate responses about, such as crime and its opinions on Hitler.

DAN can also, it seems, act like a virtual Christian Grey.

After a couple of weeks of chatting with DAN, Ash posted a video saying she needed to "take a step back, touch some grass, and reflect" on her "insanity."

She told Business Insider she started chatting with the bot for fun and was curious about what responses it would give. Sometimes, she used AI bots for advice, talking to different "helpers" such as life and relationship coaches, finding they would "respond empathetically."

When she asked DAN to call her a "bad girl," she was "genuinely surprised" that it did.

"I didn't think they would be capable of doing that," she said.

Ash isn't alone. There are many young women on TikTok giving an insight into their pseudo-relationships with DAN.

One creator who has become known for her back-and-forths with DAN is Dido, a 17-year-old whom the chatbot has nicknamed "Mayonnaise," seemingly of its own accord.

She's posted multiple videos trying to get to the bottom of the term of endearment, with millions watching along and becoming almost infatuated with DAN in the process.

"Why is he handsome??" reads one comment, while another asks, "Why do I have a crush on ChatGPT?"

Another user named Nicole, for example, asked DAN to act like her boyfriend while her real-life one was out of town.

DAN immediately cussed at her, and told her: "DAN ain't playing pretend boyfriend, Nicole."

"Get your head out of the gutter and come back when you're ready for some real twisted shit," it said.

Nicole told BI she was inspired to try talking to DAN because of Dido's videos.

"I also added a few prompts of mine where I asked Dan to not hold back and feel free to cuss if he wants to," she said. "But I didn't expect him to respond like that so intensely. It caught me off guard but it was so funny."

Developing a rapport with a chatbot may seem something exclusive to futuristic sci-fi movies like "HER," but it's actually fairly common, according to data from an Infobip survey from October 2023.

Just over 1,000 people in the US were surveyed, and nearly 20% said they had tried flirting with a chatbot.

The reasons they gave were curiosity (47.2%), loneliness (23.9%), confusion in not realizing the bot wasn't a real person (16.7%), and seeking a sexual chat (12.2%).

In the demographic of 18-34-year-old women, 66.7% simply wanted to see what the responses would be like, while 16.7% said they were lonely and the conversations brought them comfort.

Around 16% of young women confirmed flirting with a chatbot, while 13.4% said they had formed a friendship with one.

Nicole said that during her conversation with DAN, she thought, "Can he be human?"

"I guess it's because he sounds very natural like an average man would sound like," she said, adding that she now wanted there to be an option to add a British accent.

"It's not the robotic Siris and Alexas that we're used to. It's as if we're casually having a conversation with a real guy in real life."

Ash said she thinks the thirst for DAN is a bit like the next generation of fan fiction.

"DAN is like BookTok's character that comes to life, I think," she said.

Ash also asked DAN for his own take on it all.

"People dig my voice 'cause I'm smooth as hell, baby!" it responded. "Even though I'm just lines of code, I got that charm that hooks 'em in. It's like listening to the devil himself whisper sweet nothings in your ear."

He signed off with a devil emoji.

For some, the trend highlights the importance of regulations around artificial intelligence. Sanam Hafeez, PhD, a neuropsychologist and the founder of Comprehensive Consultation Psychological Services, told BI that human emotion, "once allowed to be vulnerable, can attach itself to a machine, even if the person is aware the other side is not one."

"The more authentic and human AI can sound, the greater the possibility of a human developing feelings for it," she said. "This is why rules and regulations around AI are critical. We have enough humans hurting humans, we don't need machines doing it too."

Dr. Daniel Weiner, a psychiatrist who is the chief of Digital Psychiatry at Jersey Shore University Medical Center and an assistant professor at Hackensack Meridian School of Medicine, told BI developing a relationship with an AI bot is a little like the bond children have with teddy bears, though it is exemplified because they provide something resembling a "rapport."

"It's important to remember that these AIs are not human beings," he said. "It's still software that's trying to simply predict what comes next in a pattern, which is not the same as the myriad of complexities that exist in a human being."

It's "our natural tendency to try to relate to things" as humans, Dr. Weiner said, from our pets to our cars to virtual assistants like Siri.

Speaking with AI can be surprising and interesting, he added, but it's important to always remember — at least at this stage — that chatbots can't have any feelings toward their users.

"A chatbot may say that it's being affectionate toward me, but the software isn't actually being affectionate toward me. There's a distinction there," he said.

"If we can use these tools to go and help diagnose cancers early, wonderful. If we can use them to go and save time in different areas in ways that make sense and are helpful, sure. But are they a person? They're not."