Privacy-Preserving Sentiment Analysis using PATE-based Federated Learning

Due to the proliferation of user-generated text data
on personal devices, sentiment analysis has become more important for understanding public opinion, business performance,
and mental health indicators. However, existing methods require
the raw private text data to be sent to a central server, a
practice in direct violation of data protection policies such as
GDPR and HIPAA. In this paper, a new framework called PATE-FL, integrating Federated Learning and the Private Aggregation
of Teacher Ensembles, is introduced to achieve strong (ϵ, δ)-
differential privacy and maintain competitive task performance.
Eight teacher models are trained on three heterogeneous clients, with all inter-client communications being AES-GCM encrypted. Through knowledge distillation, a single student model is then trained on noisy soft label data generated by aggregating the votes of all teacher models on approximately 3,500 unlabeled public text samples, thus eliminating any direct exposure of the final model to raw private text data. Performance evaluation on a balanced 50,000-sample text corpus results in 85.94% ± 0.30% accuracy and macro F1, achieved at a strong level of privacy characterized by ϵ ≈ 3.5 and δ = 10^−5 . These results demonstrate a significant advancement of existing privacy-aware methods in terms of performance and privacy guarantees.