Crop Yield Prediction Using Explainable Machine Learning on Multi-State Agricultural Data

The research encourages a data-driven approach to forecasting yields with agronomic statistics that are readily available: historical data on crop type, growing season, area cultivated, production, annual rainfall, fertilizer use and pesticide use in several states and years. This work fills the gaps in the previous studies with
six main innovations:
1) Formal Data-Quality Audit: Documentation of missingness, outliers, temporal coverage prior to modelling. All papers reviewed omitted this, as well.
2) Target-Leakage Detection and Correction: We expose a previously unflagged target-leakage issue in Sharma et al. A model trained on area and production reached
R2 = 0.9950 compared to 0.0781 without production, inflating reported accuracy by 0.9169 R2 points. This likely explains the 98.96% claimed accuracy. We exclude production and all proxies from the feature set.
3) Variance-Decomposition Audit: We demonstrate via within-cell variance analysis that fertilizer and pesticide columns encode state-year intensity rates, not crop
specific dosages. We document this structural dependence explicitly.
4) Multi-Split Generalization Testing: Beyond standard random splits, we evaluate under temporal (train less than or equal to 2015) and spatial (6 unseen states)
schemes. A 0.1251 R2 generalization gap between random and spatial splits confirms and quantifies the caution in prior work that standard splits overstate real
world performance.
5) Rigorous Uncertainty Quantification: Split-conformal prediction intervals are validated for empirical coverage (94.17% at 95% target). This validation step is absent from all reviewed papers. Practitioners receive calibrated confidence bounds, not point estimates alone.
6) Explainability and Transferability: TreeSHAP with interaction values reveals that prior-year yield and 3 year rolling mean dominate (0.377 and 0.314 impor
tance). This extends single-feature attribution to explicit interaction effects. Leave-One-Crop-Out analysis shows R2 ranges from 0.879 for wheat to negative 0.346 for sugarcane. This surfaces a previously invisible unseen crop transfer gap, a limit no single-crop or few-crop study could uncover.