Investigating Fairness in Facial Verification with Siamese Neural Networks

This study investigates fairness in facial verification using a Siamese neural network, which is designed to compare two facial images by learning their similarity. The network's identical Siamese subnetworks process input images to produce feature embeddings, which are then compared using a distance metric to determine if the images belong to the same person. To evaluate the network's ability to distinguish between similar and dissimilar faces, triplet loss was employed during training. The labelled faces in the wild (LFW) and Olivetti datasets were used for both training and evaluation, with transfer learning incorporated through a pre-trained VGG19 model. During training, the model's triplet loss decreased significantly from 95.8 to 0.39. However, despite these positive training measures, the model achieved only 57.14% accuracy when evaluated on unseen data, revealing a substantial gap between initial apparent training success and real-world application. These findings underscore the complexities of ensuring fairness in facial verification systems and emphasize the importance of ongoing research. In addition, this paper employed grad-cam to provide interpretability, offering additional insights into the network's decision-making process.