Combining Ensembles of Multi-Input Multi-Output Subnetworks and Data Augmentation Does not Harm Your Calibration

Ensembling neural network models is a common practice to increase model calibration and robustness. Likewise, data augmentation is a set of techniques used to enhance model calibration and robustness by introducing invariant feature transformations. However, the total effect of combining two methods is not well researched. There are contradicting results presented in the literature showing that combining some ensembling methods and data augmentation can result miss-calibrated models. In this paper, we aim to show that data augmentation does not degrade model calibration for ensembles of multi-input-multi-output subnetworks. We find that combining ensembles of multi-input multi-output subnetworks with data augmentation increases accuracy without harming model calibration. Moreover, combining subnetwork ensembles with data augmentation also helps to achieve better uncertainty estimates. We designed and performed a factorial experiment consisting of 3 factors; data sets (Cifar-10, Cifar-100, Tiny ImageNet), ensembling frameworks (MIMO, Linear-MixMo, and Cut-MixMo), and data augmentation methods (MixUp and CutMix).