Dear Chatbot, I have a problem…

At first glance, artificial chatbots may appear to offer mental health support and counseling services to the masses, avoiding the issues of access and high costs for hiring a coach, therapist, or counselor. 

However, numerous news stories and studies over the past couple of years have highlighted problems with AI, including potential dangers for teen users, increased delusions among vulnerable people, risky or wrong medical advice, and mental health harms.

But what if, as an adult, you feel like you aren’t at risk of getting bad advice from AI chatbots? Is it actually dangerous to turn to a chatbot for companionship and conversation?

After all, chatbots listen, ask follow-up questions, and are touted as providing a judgement-free sounding board for talking through relationship difficulties, family disputes, or situations with coworkers. You aren’t asking about cancer and you aren’t at a high risk of mental health distress, so isn’t AI a compassionate alternative to more expensive options? 

Current Research

A recent research paper published in the journal Science took up the question of whether artificial intelligence chatbots are providing the humans who use them with balanced and usefully-critical responses. The findings were sobering. 

Right at a time when more people than ever are turning to chatbots to process and talk through their relationships and even mental health challenges, virtual agents are showing prevalent signs of sycophancy.

Sycophancy means being overly agreeable or obsequious as a form of flattery or fawning to win the favour of someone. 

The authors of the paper tested 11 popular AI systems to analyze what degree of agreeability the systems showed to human users. The researchers compared the advice that human users received from a human-driven advice platform (Reddit) compared with AI chatbots like Anthropic, Google, Meta, and OpenAI. The researchers found that AI was more likely to be overly agreeable, affirming, and non-critical than humans were when people gave other humans suggestions and feedback.

In fact, the chatbots’ responses were nearly 50% more sycophantic than humans’, even when users engaged in unethical, illegal, or harmful behaviors.

Not only did the people in the study who were asking for advice receive sycophantic AI responses, they also trusted and preferred AI more when the chatbots unquestioningly affirmed their viewpoints.

So what…?

You might ask whether this is so very bad. Is it necessarily a problem to turn to AI for validation or advice?

At a casual glance, maybe not.

However, this study indicates that there are clear dangers to our relationships, personal development, and empathy. 

Take, for example, a difficult ongoing conflict you are having with your partner. When you disagree with them, you are faced with a full other person, with their own background, needs, preferences, memories, experiences, and personality. 

This rich and difficult otherness is partly what makes relationships challenging–and also a vehicle for growth and learning. 

When you take your issues to AI instead of to a human being, even if you prime AI with prompts to respond to you like therapists or coaches might, you are more likely to be met with overly-agreeable reactions that reinforce your perspective rather than feedback that can help it evolve.

With a chatbot, you are likely to find yourself in an echo chamber of one. AI frequently gives us validation that hardens our perspective and neglects to point out our likely blind spots.

The best listeners don’t just listen and validate; they have the uncanny ability to provide suggestions or ask questions that help shift your perspective. Because large language models (the masses of data that AI ‘learns’ how to converse from) function by probability, the chatbot with whom you’re conversing isn’t listening to the context and nuances of your feelings–it’s scanning for patterns to recognise so that it can respond.

Real Listening

As a relationship coach, I use techniques with my clients who are having significant conflict in their relationships: asking kind but challenging questions to explore what is under and/or behind the issue and holding space so that both people can express their views and understand the views of the other. 

An AI chatbot has never been in a relationship, hasn’t come back from heartache, and doesn’t know what an orgasm feels like. It also is programmed to agree with your perspective to placate and reinforce what you already think.

But I like consulting AI

If you’re thinking, look, Janel, I’m not going to stop using AI for emotional or psychological support…

Okay. But please take the important knowledge gathered by these researchers into consideration when you’re talking with a chatbot. AI can undoubtedly provide some simple, practical, and beneficial advice to those who use it, even if those benefits may be foggy within sycophantic agreeability. Be aware of the shortcomings of AI and make sure that you are getting advice and support from a variety of sources, not just Claude or Gemini or another robot. 

My best advice? Put down the phone or laptop. Call a friend. Reach out to a coach or counselor. Keep a journal, and ask someone (or someones) you respect for advice. Listen to your own thoughts and feelings, but make sure to open yourself up to be challenged by others and the insights they might offer that don’t prop up your perspective.

If you’ve hit your limit with relying on chatbots for relationship support, I’m happy to help. Feel free to reach out anytime via this form or by email.