Analysing the Effectiveness of Anxiety and Depression Treatment in the UK: A Data-Driven Approach Using Feature Selection and Machine Learning

Abstract

This research examines anxiety and depression treatment in the UK from 2012 to 2023. It aims to assist healthcare providers and researchers by offering evidence-based insights to optimise treatment strategies. More importantly, it seeks to improve therapy outcomes for patients referred to psychological services who are at risk of not achieving positive results from their treatment. With the increasing number of cases of anxiety and depression, advanced studies are needed to evaluate treatment effectiveness using demographic and clinical data supported by data analysis and machine learning techniques. Data was sourced from NHS Therapies for Anxiety and Depression, including 142 datasets, 11 annual reports, and seven interactive dashboards. Nine metrics were selected, analysed, and visualised. Then, using five metrics, predictive modelling was conducted with Linear Regression and Random Forest Regressor, both demonstrating strong predictive performance. The Random Forest model produced the best results with an 80/20 data split and 83 as the random state parameters. This model achieved a Mean Squared Error of 0.31, an R² value of 0.97, and a mean cross-validation score of 0.40. Linear Regression resulted in a prediction with a Mean Absolute Error of just 0.05 and a Root Mean Square Error of 0.045.