Can an AI chatbot be used as a substitute for counselling?

In February, the BBC published an article on their news site about people using the AI chatbot DeepSeek as a substitute for therapy. About one week later, another article was published reporting allegations that DeepSeek had shared users’ data with another tech company called ByteDance. This left me sitting with the question: can AI ever be an effective substitute for counselling? 

In this article, I explore this question by considering the ethical boundaries of counselling and whether AI is adept at managing these in the hope that it will help you make informed decisions about how to look after your mental health.  

How does AI work?  

AI chatbots are built from a system that has been fed massive amounts of data from various sources such as books, websites, social media and so on. The system learns to spot patterns in language and makes guesses about the most likely next word in any given sequence. These guesses gradually increase in accuracy over time, to the point that it can generate convincingly human-like responses. 

Once the system has done its initial training, there is also a human training phase where the results are fine-tuned, and humans rate the responses to guide the system towards preferred behaviour – for example, being polite, staying on topic, or avoiding harmful content. 

How can AI be helpful? 

AI chatbots are accessible at any time and provide a platform where you can get things off your chest. We often have things we just need to express in the moment, or don’t feel ready to share with a real person, and putting these thoughts into sentences helps us to see what we’re thinking from a different perspective. Journaling is one way of achieving this, but unlike journaling, chatbots also provide an instant response that matches the tone and content of the message you’ve sent, which may help you to feel listened to, validated, and understood.  

If you’re using the chatbot because you want to feel listened to, this probably also means you want to connect with someone. Chatbots reply in such a human-like way that it gives you the sense of connection that you’re looking for. And, contrary to speaking with a friend or family member, the chatbot won’t have opinions, feelings, or thoughts about what you say that might challenge your views. 

Being able to access something that makes you feel supported for free is very appealing in today’s climate of ever-rising costs. 

How can AI be harmful?  

Although AI seems appealing, it’s important to be aware of the risks and downsides. We'll now discuss what these are and how a good counsellor could provide a better experience.  

Lack of confidentiality  

One of the main issues with AI chatbots like ChatGPT is that they don’t have the same confidentiality policies as counsellors. 

It’s often said that when an online product is free, it means you’re the product. The information you give has monetary value to the company, so they will be keen to use and share it. In the EU and UK, these activities are governed by a law known as GDPR (General Data Protection Regulation), which requires that data be handled securely and for a legitimate purpose. However, companies are still allowed to gather and share data with third parties so long as they comply with these rules.  

This data processing can include sensitive information that you might provide to a chatbot, such as your personal thoughts, feelings and biographical information. This could easily be combined with other information gathered from your device that can identify who you are, such as your account information and GPS location, and there is a risk that this information spreads wider than you intended. It is important to consider whether you trust tech companies to keep this information out of the hands of someone you wouldn’t want to have it. 

In contrast, counsellors work within a confidentiality policy, which means that what you say will be kept between you and the counsellor. Counsellors keep brief, objective notes after each session, which are kept securely, and you can request access to them at any time. These records are eventually destroyed in accordance with a timeframe set by GDPR. 

Counsellors will only disclose information if you or someone you know is at risk of harm and you mention this to the counsellor. The counsellor has a duty of care to raise the matter with an appropriate agency. 

Giving advice 

AI chatbots do not hesitate to give advice, which is not the role of counselling. Developing your confidence, a sense of control, or becoming more assertive are all common goals within counselling and giving advice undermines that process, as it takes the power and control away from you. When you’re feeling overwhelmed, distressed or lost, it’s natural to want someone to pull you out of those feelings and tell you what to do.

You might be given techniques to help you manage the symptoms you’re experiencing, and psychoeducation can play a role within sessions when appropriate. But in general, a counsellor won’t tell you what to do. Any decision you make should be yours. Your counsellor is just there to help you understand who you are, how you feel and what you want to do next. 

There is also the risk that AI will give bad advice, because it doesn’t know any better. During its training, humans attempt to train the AI to avoid harmful responses. However, we can never guarantee that AI will respond in a certain way. There have been reports in the media of AI chatbots encouraging unhelpful or dangerous behaviours, such as giving diet advice to someone seeking support for an eating disorder. An article on the APA’s website highlights instances where, in a couple of cases, prolonged use of an AI therapist run by Character, AI resulted in a child becoming physically violent, and another sadly taking his own life. 

Profiling 

Because AI chatbots are trained on human interactions on platforms such as social media, they can pick up harmful biases and stereotypes which influence their responses to you, and it cannot be guaranteed that these problematic responses will be weeded out in training. 

Of course, counsellors also have unconscious biases and may be prone to snap judgements. However, counsellors are trained to recognise their own biases and notice when they’re making judgements and where these judgements might be coming from, so they can make sure they don’t intrude on the work. We are experienced with working with clients from different backgrounds and aim to treat people as individuals, without judgements or assumptions.

Sycophancy 

AI chatbots have a well-established tendency to be sycophantic, meaning that they give the most pleasing responses possible in order to maximise engagement. But constantly validating and agreeing with you can be just as harmful as bad advice. In reality, no one is perfect, and we sometimes need to learn from our mistakes so we can grow as people. 

While counsellors give support and empathise with the feelings you bring, they also empower you to look at yourself from a different perspective. A metaphorical mirror is held up to you, and sometimes people can find what they see difficult to process. It is challenging to explore the idea that your current thoughts and feelings might be unhelpful. But the counselling environment creates a space where it is OK to be wrong and make mistakes – accepting them is a key part of building resilience and growth.  

So, while AI is designed to be a permanent fixture in your life, which can keep you in a cycle of dependence, the aim of the counsellor is to eventually become redundant to you, because you have learned the skills you need to independently navigate your life.  

Lack of boundaries 

Counsellors work within what we call therapeutic boundaries. These are put in place by a clear agreement between therapist and client, which includes meeting only at a specified time and place. Keeping the process contained makes it easier as the weeks go on for the client to manage everyday life in between sessions. Counselling helps clients to be able to tolerate their distress, build resilience and know that whatever is bothering them will have a space during the session. For some clients, it is also an opportunity to learn that having firm boundaries doesn’t always mean something negative. 

Being able to access emotional support all the time, on the other hand, can engage you in a co-dependent relationship, where you cannot learn to manage without it. This is a major risk of relying on AI for therapy. 

Accountability 

An important aspect of the counselling agreement is accountability. If the counsellor breaks the agreement, a complaint can be made to the membership body they belong to. As part of their membership, counsellors are required to see a clinical supervisor at least once a month, who makes sure they are working appropriately and ethically. Counsellors should also be registered with the Information Commissioner’s Office, whom you can contact if you have any concerns about how your data is being stored and used.   

While AI chatbots are not completely unregulated, it is fair to say that they don’t have the same safeguards in place for providing therapeutic support, and it can be hard to hold someone accountable when they get things wrong. 

Lack of a relationship 

One of the reasons counselling can be really beneficial is the opportunity to genuinely connect with another person. Counsellors call it the therapeutic alliance, and it’s what creates safety and trust and facilitates change. It can also be helpful for some people to develop a real understanding of what a relationship with another human is, with all its imperfections. 

AI can do a very good job of simulating a relationship, but ultimately, it is just a simulation. As mentioned at the beginning of this article, it doesn’t have an emotional response to what you’re saying and doesn’t care about you. An AI Chatbot is trained to keep you engaged and to be as helpful as possible in as many ways as possible, so you keep using the app.  

It should be clear from the above that there are risks and downsides to AI chatbots that should be kept in mind. An alternative to chatbots could be apps designed specifically for therapeutic purposes. There is a debate in the counselling community about whether these apps are useful for helping to meet the demand for mental health support.

However, a lot of the points in this article still apply. If you’re unsure about data usage, privacy, or if the responses you’re getting aren’t quite right, there are other non-AI apps that have been developed. For example, Daylight is an NHS-funded app for anxiety and teaches self-management and CBT techniques. If human connection is what you’re searching for, there are also person-run email, text, and online chat services like SHOUT and Samaritans that are free and available to use 24/7.  

If you want to go further than ‘getting things off your chest’ and develop self-help techniques, then counselling might be the right option for you. It’s ok to ask counsellors about their training, qualifications, and memberships to ensure they adhere to a good working practice and ethical framework as well.