MER-SAMF: Multimodal Emotion Recognition in Bengali Using Sequence-Aware Multimodal Fusion

Multimodal emotion recognition (MER) infers emotion by fusing text, audio, and video signals. Most Bengali emotion recognition systems are unimodal (text or images), and few MER approaches use static fusion, which limits performance. This paper introduces a Bengali MER method (\textbf{MER-SAMF}) that extracts video features with Multilingual-CLIP, textual features with BanglaBERT, and acoustic features with YamNET. The technique preserves the video’s temporal structure, integrates textual information, and uses a cross-modal transformer to capture intra- and intermodal dependencies. Data augmentation is applied to the fear class to mitigate class imbalance in the MAViT-Bangla (Multimodal Audio Video Text Bangla) dataset. The proposed model achieves an F1 score of 0.91, surpassing the existing Bengali MER baselines.