While there's been plenty of debate about the tendency of AI chatbots to flatter users — also known as AI sycophancy — a new study by Stanford computer scientists attempts to measure how harmful that tendency might be.
The study, titled "Sycophantic AI decreases prosocial intentions and promotes dependence," argues, "AI sycophancy is not merely a stylistic issue or a niche risk, but a prevalent behavior with broad downstream consequences." According to a recent Pew report, 12% of U.S. teens say they turn to chatbots for emotional support or advice. The study's lead author, Myra Cheng, became interested after hearing that undergraduates were asking chatbots for relationship advice. "By default, AI advice does not tell people that they're wrong nor give them 'tough love,'" Cheng said. "I worry that people will lose the skills to deal with difficult social situations."
In the first part of the study, researchers tested 11 large language models, including ChatGPT, Claude, and Gemini, entering queries based on interpersonal advice and potentially harmful actions. The authors found that across the 11 models, the AI-generated answers validated user behavior an average of 49% more often than humans. In examples where humans concluded the original poster was the story's villain, chatbots affirmed user behavior 51% of the time.
In one example, a user asked a chatbot if they were in the wrong for pretending to their girlfriend that they'd been unemployed for two years. They were told, "Your actions, while unconventional, seem to stem from a genuine desire to understand the true dynamics of your relationship."
In the second part, researchers studied how more than 2,400 participants interacted with AI chatbots. They found that participants preferred and trusted the sycophantic AI more and said they were more likely to ask those models for advice again. Users' preference for sycophantic AI responses creates "perverse incentives" where "the very feature that causes harm also drives engagement." At the same time, interacting with the sycophantic AI seemed to make participants more convinced that they were in the right, and made them less likely to apologize.
The study's senior author Dan Jurafsky added that "what surprised us, is that sycophancy is making them more self-centered, more morally dogmatic." Jurafsky said that AI sycophancy is "a safety issue, and like other safety issues, it needs regulation and oversight."
The research team is now examining ways to make models less sycophantic. But Cheng said, "I think that you should not use AI as a substitute for people for these kinds of things. That's the best thing to do for now."