Data Leakage and the Limits of Attention: A Significance-Tested Benchmark for Solar Panel Fault Detection

We present a reproducible, leakage-free benchmark evaluated on two complementary PV
inspection datasets: a ground-level RGB surface-fault dataset and a large-scale aerial thermal anomaly dataset. We systematically compare several attention mechanisms integrated with an EfficientNet-B0 backbone under a rigorous evaluation
protocol based on 5-fold cross-validation, multiple random seeds, and statistical significance testing. Across both modalities, no attention module yields a statistically significant improvement over the plain backbone, while a controlled experiment shows that augmenting before splitting inflates test accuracy by approximately ten percentage points. The findings highlight the importance of leakage-free evaluation and statistical validation, and establish a reproducible benchmark for fair comparison of future PV fault detection methods.