How Sample Size Determines Whether the Best Model Is Also the Best Decision in Budget-Constrained Churn Targeting

This study shows that the model selected as best by AUC may not provide the most stable or profitable customer targeting decisions when data are limited. Using the same evaluation process at two sample sizes, we find that this divergence appears at 875 customers but disappears with the full 7,032-customer dataset. The findings highlight the importance of checking whether model selection results remain consistent as more data become available.