IJCNN 2026 (WCCI, Maastricht)
Accepted Mar 2026
Simple Averaging vs. Learned Stacking in K-Fold Ensembles
A simple question with a surprising answer: when you build a K-fold ensemble, should you learn how to combine models, or just average them? Using a leakage-free GroupKFold setup (no well appears in both training and validation, so out-of-fold R squared ranks models without ever touching ground truth), I show that simple averaging consistently beats a learned Ridge meta-learner, and the gap widens as you add more architecturally diverse models, because learned weighting discards diversity that matters under distribution shift. A Top-K ablation shows monotonic gains across 5 diverse architectures. The 4-model ensemble reaches R squared of 0.89 on 4 fully blind test wells, 7.16 points better than the best single model. Data: 188,631 training samples (17 wells) and 84,973 blind test samples (4 wells), Potwar Basin.
0.89
R² on 4 blind wells
+7.16 pts
Over best single model
188,631
Training samples