A Synchronization-Aware Deep Reinforcement Learning Framework for QoE-Optimized Mulsemedia Streaming

Immersive smart environments and Metaverse-oriented applications increasingly rely on the synchronized delivery of heterogeneous multimedia streams, particularly video and haptic feedback, over bandwidth-varying networks. Existing Dynamic Adaptive Streaming over HTTP (DASH) and Adaptive Bitrate (ABR) algorithms are primarily designed for single-stream video and generally lack mechanisms to explicitly address cross-modal synchronization drift, which can significantly degrade temporal coherence despite high video quality. This paper investigates a joint video–haptic adaptation framework to maximize overall Quality of Experience (QoE) while preserving synchronization between modalities. The adaptation problem is formulated as a Markov Decision Process (MDP) and optimized using a Proximal Policy Optimization (PPO)-based reinforcement learning policy that selects paired video and haptic representations for each streaming segment. A unified reward function is designed to maximize bitrate utility while penalizing rebuffering events, quality fluctuations, and synchronization drift. Performance is evaluated through trace-driven DASH simulations using real-world network throughput traces and compared with established baseline algorithms, including BOLA and Model Predictive Control (MPC). Experimental results demonstrate that the proposed approach effectively balances visual quality and temporal coherence, achieving an average video bitrate of approximately 2054 kbps while reducing synchronization drift to 22.57 ms, outperforming conventional video-centric adaptation strategies. These findings highlight the importance of drift-aware adaptive streaming and demonstrate the potential of reinforcement learning for delivering synchronized multisensory experiences in future immersive communication systems.