Q-Learning-Based Degradation-Aware BESS Management for Agricultural Microgrids

Battery Energy Storage System (BESS) integration
improves reliability and sustainability in agricultural microgrids,
but inefficient control accelerates degradation and can itself
compromise reliability. We propose a reliability-constrained,
degradation-aware BESS management strategy using Q-learning.
During development we found that an unconstrained action space
lets a Q-learning agent inflate its apparent cycling reduction
by withholding discharge from demand– a reward-hacking
failure mode leaving 4.7–14.9 percentage points more demand
unserved than the baseline. We resolve this by guaranteeing
discharge whenever the battery can physically supply it, and
training the agent only to control charging intensity during
solar surplus. Evaluated on one year of real 15-minute field
data from an agricultural microgrid in Jashore, Bangladesh,
across three agricultural seasons with a chronological, leakage
free 80/20 train/test split and validated across 3 independent
training seeds, the controller matches a same-dataset greedy
baseline’s service reliability to within 0.03% of total demand
while reducing cumulative battery cycling by 44.4–54.1% and
cumulative degradation damage by 66.2–69.3% (mean ± std) on
held-out data. A companion ablation confirms the unconstrained
variant inflates its apparent reduction to 77.8–95.2% only by
sacrificing reliability. A complementary multi-objective (NSGA
II) analysis and a benchmark of four supervised SoC/RUL fore
casting models (best R2=0.915, LSTM) are also reported. Results
indicate reliability-first reward design is essential for trustworthy
degradation-aware reinforcement learning in agricultural BESS
applications.