An Ensemble Deep Learning Approach for Breast Cancer Classification Using Vision Transformers and CNNs

Breast cancer is still one of the most prevalent causes of cancer death globally, indicating that there is a critical need for early and precise diagnostic methods. While histopathological examination remains best method for breast cancer diagnosis, its accuracy and reliability depend significantly on the experience and subjective interpretations of pathologists. This research examined three state-of-the-art deep neural network architectures - Inception-v4, EfficientNet-B0, and Vision Trans former (ViT) - for the binary classification of breast histology images into benign and malignant categories. This research utilized the BreakHis dataset containing 7909 histopathological images as the training dataset, in addition, an external validation BACH dataset was used consisting of 400 images. Diagnostic performance on the external test set was further increased using a novel stacking ensemble method that combined predictions from individual models via Random Forest as a meta-classifier. The stacking ensemble method significantly outperformed individual models, achieving an accuracy of 96.0% and ROC-AUC of 0.99 on the external BACH dataset. Robustness analysis was also conducted to evaluate performance against common imaging artifacts, and visual interpretability was provided through Grad-CAM analyses, enhancing the clinical relevance of the models. The study also checked how the models performed when there were common image issues or distortions. Grad-CAM visualizations were used to see which areas in the image the models relied on for making predictions. In this work, CNNs were used to capture fine details from the images, while the Vision Transformer helped to recognize the overall structure. Bringing both models together improved the accuracy and made the results clearer for medical use.