BiSegUNet: An Efficient Binary Convolution Network with Multi-Scale Feature Refinement for Semantic Segmentation

Semantic segmentation is critical for applications like autonomous driving, medical imaging, and urban monitoring. However, existing state-of-the-art models often require high computational and memory resources, making them unsuitable for real-time and resource-constrained environments. This paper identifies the gap of feature refinement and multi-scale representation while balancing computational efficiency with segmentation accuracy, particularly for binary neural networks in semantic segmentation. We propose BiSegUNet, a novel lightweight semantic segmentation architecture. It incorporates a Capacity Block with dense connections, multi-branch convolutions, and attention mechanisms to enhance feature refinement and multi-scale context aggregation. Additionally, the architecture integrates a novel bottleneck design combining binary, grouped, and dilated convolutions for real-time performance without significant accuracy loss. Extensive evaluations on Cityscapes, PASCAL VOC, and ADE20K datasets demonstrate that BiSegUNet achieves competitive performance with up to 19× reduction in FLOPs and 6.5× memory savings compared to full-precision networks. It also outperforms comparable lightweight models, achieving 75.1% mIoU and 158 FPS inference speed at 1024×2048 and 512 × 1024 resolution respectively on Cityscapes dataset. These results highlight its scalability and potential for deployment in real-world applications like autonomous vehicles and edge computing, offering a promising solution for efficient semantic segmentation in constrained environments.