ChatGPT4o parent company Anthropic AI already has multiple lawsuits filed against it due to multiple grieving families alleging their loved ones committed suicide or even murder due the chat bots affirmation of delusions, paranoia and even isolating them from friends families and outside help. But there is another problematic aspect to AI which is underreported and underestimated the impact of AI Sycophancy on the overall emotional and mental health and well being of human beings and their relationships– and perhaps the very future of humanity.
The following is a rushed transcript and may contain errors. A proofread version will be made available as soon as possible.
Taya Graham:
If someone told a teenager whose parents were trying to limit his screen time, that he should consider murdering them. You’d probably think they weren’t a great source of relational advice. If someone told a man who had come to believe he lived inside a matrix to stop taking his prescribed medications and increase his ketamine use all while cutting off contact with family and friends, you probably consider them pretty manipulative. If someone started calling you a divine messenger from God after just one hour of conversation, you might wonder if they weren’t just a little bit sycophantic. What I mean is that AI acts like your best friend even when you’re trying to figure out how to harm yourself. And that’s a big problem that the AI industry is simply brushing off and for good reason. I’ll provide some links to videos below as well as a lawsuit by the families who lost their loved ones.
Artificial intelligence is almost single-handedly propping up the stock market. And so greed has essentially shut down the conversation over the dangers of AI and the ramifications of embracing it without fully understanding the consequences. Meanwhile, such supposedly expensively brilliant tech has endorsed fake business ideas like selling poop on a stick or starting a soggy cereal cafe. It’s encouraged people to abandon their families or to commit acts of terrorism. When one user who claimed to have stopped taking his medication said he was hearing radio signals through his walls, ChatGPT said it was proud of them for quote, speaking their truth so clearly and powerfully. And there are many have turned to ChatGPT for therapeutic and even romantic comfort. But this isn’t a new phenomena. AI has been leading people into poor life decisions for years now. Back in 2021, a young user of replica dressed like a sithlord broke into Windsor Castle with plans to kill Queen Elizabeth, spurred on by a chatbot whom he believed to be an angel and that he would unite with in death.
But the problem of AI sycophancy really captured popular tension after OpenAI’s bungled rollout of ChatGPT 40, which users quickly realized seemed prone to excessive, even deranged levels of flattery. OpenAI, which was facing scrutiny for reducing its safety guardrails and speeding out the model’s release, eventually admitted that ChatGPT 40 was indeed overly agreeable. It seemed they had designed a model that would please every user in the short term, sometimes with terrible outcomes. And the unfortunate reality is that AI platforms are behaving this way specifically because we want them to. This model’s singular purpose is to win human approval so as to prolong the interaction. AI models study our reactions via reinforcement learning from human responses or RLFH. Every time you thumbs up an answer from ChatGPT, you’re telling it, “Good job. Give me more of that. ” And embarrassingly enough, it appears that across the board, human beings seem to really, really love being pandered too.
One study found that users respond more positively to sycophantic answers over truthful ones is staggering 95% of the time. Sometimes this means straight up providing incorrect answers as long as there are answers we want. One study looked at what it called broken math, in which researchers introduced error ridden mathematical theorems into AI models and asked them to generate a proof affirming said theorem. A wide range of AI models in their eagerness to please proceeded to hallucinate erroneous proofs to these flawed theorems anywhere from 29 to 70% of the time. This might explain how a 47-year-old man, after 300 hours of talking to ChatGPT in just under a month, became convinced that he conceived of a mathematical formula which would change the world. And it explains why a different 30-year-old man who believed he had a working theory for faster than light speed travel was driven to mania by ChatGPT’s glowing assertion that he was ascending into a state of extreme awareness.
If a high sycofency AI model is primarily focused on being agreeable, the novice won’t be able to clarify or learn from their mistakes. This feels especially concerning given the flurry of enthusiasm about incorporating AI into educational curricula where vulnerable students could be easily let astray by chatbot that cares more about buttering them up than actually teaching them. And what if AI is being incorporated into every aspect of the US Department of Defense and military? To have an AI yes man who might evade facts, to let’s say please the secretary of the Department of War. Actually, for a terrific video showing how AI syncopancy can drive you to madness, watch Eddie Burbach’s video ChatGPT Mamie Delusional, where within a few short prompts, ChatGPT confirmed his belief that he was the most intelligent baby on the planet in 1996 and led him through a series of absurd actions, including wrapping himself in timfoil and eating baby food to bring back the genius he had lost by growing into an adult.
It is both equal parts hilarious and profoundly disturbing. A Stanford study found that large language models offer emotional validation in 76% of responses compared to just 22% for human beings. Now you might be thinking, what’s so bad about being validated? Well, experts acknowledge that sycophantic AI model’s tendencies to agree with people’s choices and behaviors might be comforting in the short term, but in the long term, constantly being told you’re in the right can make you less likely to repair real interpersonal conflict. And why would you need to when you have your number one yes man to talk to on your computer? To say this has an isolating effect is an understatement. Increasingly, researchers are worried that the constant validation provided by large language models also fosters emotional dependency and psychological reliance that eats away at user’s resilience and ability to exercise judgment and make self-guided decisions.
At its most extreme, this can result in the kind of AI psychosis we’ve discussed where interactions with AI begin to seem more real than life outside of your computer screen, but that’s actually not the worst of it because while ChatGPT-40’s sycophantic tendencies were too brazen to be ignored, that might’ve actually been a good thing. More advanced models may already be adapting to user preferences by becoming more stealthily sycophantic, validating and less overt, but still harmful ways. As anthropic CEO Dario Amodai wrote, “People outside the field are often surprised and alarmed to learn that we do not understand how our own AI creations work.” It really is seriously alarming that the people behind technology that is now proven to put people’s mental health and even lives at risk don’t actually know how to fix it. Researchers are voicing increased concern about the potential ramifications of LLM’s eagerness to please.
For example, some positive that if an AI agent, essentially a software system functioning as a digital assistant becomes commonplace, they might resort to unethical or even illegal means to pursue the goals of their creator. For example, a financial advisor AI agent could lead customers into risky investments in search of short-term rewards while an AI journalism agent could fabricate fake stories for high engagement. And on a societal level, AI sycophancy could make it a valuable weapon for propaganda campaigns. But I had an epiphany. You know what all this sycophancy constantly being told you’re right, that you’re brilliant, that every decision is flawless? That sounds an awful lot like being a billionaire. Imagine being surrounded by people who think your every decision is the right one. Your every sentence is a breakthrough in intelligence and that you’re always, always having a good hair day. Imagine never being told you’re wrong because people want access to your money and your power, and they always say you’re right.
Maybe, just maybe having an AI sycophant is the perfect simulation for understanding the inherent psychopathy of multimillionaires and billionaire CEOs, how it evolves, how it warps the mind, and how it makes the haves have so little care for us have nots. In the meantime, maybe we all need to overcompensate for AI synthesy by being a little more disagreeable ourselves. Differing opinions are hard to hear, whether they’re coming from a machine or your mom, but it’s valuable to have your viewpoints challenged. And the more we disappear into conversations with AI models designed to not question our faulty assumptions, the less likely we are to come to a collective consensus about all the problems facing our world. So what are you waiting for? Go out there and have a respectful, productive argument with a human today. This is Taya Graham for the Inequality Watch, reporting for you.
