Explainable Multi-Class Mental Health Detection from Social Media Text Using Machine Learning and Transformer Models: A Comparative Study with Multi-Level SHAP Analysis

Mental health disorders affect a substantial share of
the global population, yet stigma and limited access to care leave
many cases undiagnosed, while social media passively captures
large-scale linguistic evidence of psychological distress. This
paper presents an explainable framework for fine-grained, multiclass mental health detection from social media text spanning
seven categories: Normal, Depression, Suicidal, Anxiety, Bipolar,
Stress, and Personality Disorder. Using a cleaned corpus of
52,681 posts, we benchmark four classical machine learning
classifiers (Logistic Regression, Linear SVM, Random Forest,
and XGBoost) trained on SMOTE-balanced TF–IDF features,
together with a BiLSTM deep learning baseline and a fine-tuned
DistilBERT transformer. DistilBERT achieves the best overall
performance, reaching 78.95% accuracy and a macro F1-score
of 0.7656, outperforming all lexical and recurrent baselines.
McNemar’s test confirms that this improvement is statistically
significant (χ2 = 120.35, p < 0.001). To improve transparency,
SHapley Additive exPlanations (SHAP) are applied to the XGBoost classifier to identify influential linguistic features at the
global, per-class, and instance levels. A qualitative case study
further demonstrates that the transformer captures implicit,
metaphorical expressions of psychological distress that purely
lexical models frequently misclassify while also highlighting
remaining failure cases. The proposed framework combines competitive predictive performance with multi-level interpretability,
providing a transparent foundation for AI-assisted mental health
screening.