Evaluation Blind Spots in IEC 60870-5-104 Intrusion Detection: Beyond Attack-Only Metrics

We demonstrate that attack-only metrics under family-disjoint evaluation of IEC 60870-5-104 IDS can conceal significant false-positive rates: a 40% benign holdout reveals 5.8–10.5% FPR despite attack F1 > 0.99. We further show that 48–50% of false positives are shared across RF, XGB, and CNN, suggesting these errors stem from feature-space difficulty rather than classifier-specific overfitting.