Listening to Shout user feedback at scale

Shout is a free, volunteer-led text service providing 24/7 anonymous crisis support across the UK. Since our launch in 2018, we have carried out more than 3 million conversations with people in need, 1.2 million of which were taken in the last two years alone. With such high levels of demand, we are committed to providing and maintaining the highest level of service quality.

To understand what Shout texters appreciate about the service and identify any areas for improvement, we listen to their experience in a number of ways. This includes consulting with the Shout service User Voice Group and partnering with academic institutions to analyse the thoughts of particular demographic groups. In these ways, we can explore individual user experiences and needs in great detail; however, the greater the depth, the smaller the number of users we can include.

Therefore, to ensure that we also have a broader view of our user population, we collect anonymous feedback through an optional post-conversation survey. Through the survey, Shout texters can rate whether they found their recent conversation helpful and provide anonymous detailed feedback in the form of free text.

This report describes the insights gained from analysing close to 40,000 pieces of anonymous free-text feedback collected in 2024 and 2025.

The majority of texters found their conversation helpful and attributed it to:

• the kindness and compassion of their volunteer
• having the space to process their thoughts and feelings
• receiving prompt replies and useful advice

Those dissatisfied with the service often mentioned:

• their conversation ending abruptly
• conversations feeling robotic
• long wait times
• slow replies
• the quality or lack of advice

To understand what volunteer behaviours gave rise to these complaints, we analysed the contents of the affected conversations. The insights were then used to both improve the Shout volunteer guidance as well as build automatic monitoring systems that prioritise volunteers for additional feedback from the Volunteer Support team.

Most commonly used words in feedback notes associated with conversations rated as helpful (left) and not helpful (right).

Given the high volume of texter feedback we receive, analysing it in a systematic way can prove challenging.

One approach – used by our qualitative researchers – is to read through randomly chosen examples to identify the most common themes. Though very powerful, this type of analysis can also be subjective and involves only a small subset of feedback, which risks missing less frequently mentioned yet significant themes.

To make sure that we ‘listen’ to the experiences of as many texters as possible, we also apply machine learning methods to all collected feedback.

This involves:

• using language models pre-trained on vast amounts of text data to produce numerical representations of text that capture its meaning
• automatically grouping reviews together based on their similarities

In this way, we can extract and explore the common themes discussed in tens of thousands of feedback notes.

The remainder of the feedback touched on other themes, including remarks on the need for more funding dedicated to mental health services. Interestingly, despite having rated the conversation helpful, a small proportion (<1%) of texters reported it as feeling robotic or AI-driven – we discuss this theme in more detail in the next section.

Other common sources of complaints referenced finding the overall service generally unhelpful (14%), sometimes attributing it directly to a lack of, or the quality of advice received.

A similar proportion of texters stated the conversation was unhelpful specifically in the context of a crisis, suicide, or self-harm (12%). Others quoted long wait times and slow replies (6%) or their volunteer being dismissive or condescending (2%).

Around 13% of texters shared thoughts about a bad conversation outcome, mentioning that it made them feel worse, more alone, or unheard.

Surprisingly, 15% of texters chose to leave a positive or partially positive review, even though they rated the conversation as not helpful. This category contained thank you notes for the volunteers – sometimes combined with tips for improvements – and acknowledgements that the volunteer tried their best to help but the user ultimately needed a different type of support.

The remainder of reviews touched on a variety of topics such as expressing disapproval at Shout being exclusively a text-based service – sometimes adding that it was too basic or limited to address complex mental health issues or problems – or frustration at having been brushed off by other mental health services, including the NHS.

The majority of Shout texters who shared their feedback found the conversation helpful, often crediting:

• the kindness and empathy of the volunteer
• having the space to share and work through their thoughts and feelings
• prompt replies
• useful advice

In contrast, those who were not satisfied with their conversation often cited:

• abrupt conversation endings
• robotic conversation style
• long wait times
• slow replies
• the quality or lack of advice given

To address the major areas for service improvement, we carried out follow-up investigations, updated the Shout volunteer guidelines, and developed automatic conversation quality monitoring systems – this work is described in more detail in the case study panels below.

Analysing anonymous survey responses allows us to investigate themes in positive and negative user feedback at a large scale. However, it is important to note that survey responses might be biased - for example, texters who were satisfied with their conversation might be more likely to leave feedback. Whilst such bias is difficult to eliminate completely, using a range of different approaches to explore user experiences can help to mitigate it.

In support of the findings described here, the insights gained from analysing texter feedback echo the themes identified through other channels.

For example, researchers from University of Glasgow conducted interviews with young people who have previously contacted Shout for help with suicidal thoughts and found that the key aspects of the experience that mattered to them were:

• service availability
• a person-centered approach involving compassion and curiosity
• solution-focused support

These themes overlap with the comments regarding:

• conversation wait and reply times
• positive volunteer qualities (as well as roboticism and lack of personalisation)
• the provision of advice and resources

Similarly, the Shout service User Voice Group – a group of previous service users spanning different age and ethnic groups, experiences of mental ill-health, and access needs – has been closely involved in shaping both research directions and improvements to the Shout service.

Amongst their suggestions were improvements to the messages texters receive when waiting for the conversation in the queue, intended to address some of the concerns related to long wait times – a theme discussed in the survey feedback for not helpful conversations.

The group has also drawn from their lived experience to:

• provide valuable input regarding what kind of volunteer responses make them feel listened to
• help shape additional questions asked in the texter survey related to conversation outcomes beyond overall helpfulness, such as feeling heard and understood

These insights have since been incorporated into volunteer training.

In summary, listening to the experience of Shout texters – through analysing thousands of anonymous survey responses and more in-depth consultations with smaller user groups – allows us to build a comprehensive view of what users appreciate about the service and pinpoint areas for improvement.

The most prevailing themes highlight the power of human compassion, curiosity, and collaboration in making those who reach out feel heard, empowered, and adequately supported in their time of need.