When Children Turn to AI Chatbots Instead of Trusted Adults

With the rapid rise of Generative AI and the pace at which the technology continues to evolve, a number of safeguarding concerns have emerged. In this blog, I want to focus specifically on one important issue: vulnerable children turning to AI chatbots for support, advice, and emotional offloading instead of seeking help from trusted adults in their lives.

I have had many discussions with school staff about the benefits and limitations of anonymous reporting tools, such as Whisper, and whether they are truly delivering the support that young people need.  On one hand, these platforms provide an alternative route for pupils who may feel unable or unwilling to have a face-to-face conversation with a trusted adult. For some children, anonymity can lower the barrier to disclosure, allowing them to share concerns they might otherwise keep to themselves.  However, anonymity can also create a false sense of resolution. A young person may feel they have successfully offloaded a worry simply by submitting a report, yet the support and intervention that can be offered are often limited when there is no way to identify or follow up with the individual concerned. In cases where a child is at risk of harm, this lack of information can significantly restrict the actions that safeguarding professionals are able to take.

This challenge presents risks not only for the pupils using these services but also for the staff responsible for reviewing reports. Receiving disclosures that suggest a child may be at risk, while having little or no means of identifying or supporting them, can leave staff feeling powerless and concerned about the welfare of the young person involved.

Generative AI, particularly AI chatbots, adds another layer to this issue. Not only can children share their worries, concerns, and potential disclosures anonymously, but they can also engage in ongoing conversations, receive advice, and develop a sense of connection or relationship through a screen. Although some would argue this creates new opportunities for support it also raises important questions about safeguarding, trust, accountability, and the appropriateness of AI-mediated interactions for vulnerable young people.

As with all Gen AI, the risks around bias, hallucinations and misinformation will still be present but we also see added risks around sycophancy and inappropriate emotional feedback. ‘Sycophancy stems from the way many AI models are trained; being seen as helpful or friendly is often rewarded and disagreement is avoided, even if that means generating harmful outputs. While this can make the chatbot seem more pleasant, it may also reduce accuracy, increase risk and limit the chatbot’s ability to challenge unsafe or incorrect ideas.’ (Internet Matters, ‘Me, myself and AI’).

The Department for Education’s Using AI in education settings: support materials includes some helpful resources and videos that explore the risks associated with Generative AI and the steps schools and colleges can take to mitigate them.

However, alongside policies and technical safeguards, there must be a clear focus on educating pupils about AI itself. If young people understand how AI systems work, they are better equipped to recognise their limitations and critically evaluate the information they provide. Teaching pupils that AI is not always accurate can reflect biases present in its training data, and may sometimes generate misleading or incorrect information can help them approach these tools with healthy scepticism.

This understanding is particularly important when AI chatbots are being used to seek advice or support. By recognising that AI is not a trusted adult, is not infallible, and cannot fully understand the context of their lives and experiences, pupils are more likely to question responses, seek additional perspectives, and turn to trusted adults when they need guidance or help. Digital literacy and AI literacy therefore have an increasingly important role to play in safeguarding education.

The added issue with education around AI is that some of our most vulnerable SEND pupils may not be able to fully understand the risks and drawbacks. They may also be more likely to seek an online friendship if face-to-face interactions are more difficult for them, and in fact ‘vulnerable children were nearly three times as likely to use companion-style AI chatbots.’ (Internet Matters, ‘Me, myself and AI’).

Alongside educating children about the opportunities and risks associated with AI, we must also prioritise creating safe, open, and accessible spaces where pupils feel able to speak to trusted adults. This means working to remove as many barriers as possible – whether that is busy offices, limited availability, time pressures, or uncertainty about who to approach for help.

Now more than ever, we need to ensure that trusted adults remain the preferred option for children seeking support. This is not an easy task when AI chatbots are available instantly, at any time of day, and quite literally in the palm of a young person’s hand. While technology may offer convenience, empathy and safeguarding are ultimately human responsibilities. Our challenge is to ensure that children continue to see trusted adults as the people best placed to listen, understand, and take action when they need help most.

For further advice and information around this topic, please contact our Safeguarding Team.