Towards an Accurate Domain-Specific ASR: Transcription for Pathology

A known problem with Automatic Speech Recognition (ASR) systems is their struggle to recognise specific medical terms. Biopsy medical terms are not only rare, but also require knowing how to pronounce them correctly, which is challenging for both medical and non-medical people. Additionally, because of the sensitivity of the content and the preservation of privacy, it is preferable to utilise ASR systems that work on a hospital’s internal infrastructure. This study evaluated state-of-the-art open-source ASR systems using anonymised audio recordings of biopsy examinations conducted by pathologists. We assessed the performance of models suitable for deployment within hospital infrastructure, including various sizes of OpenAI’s Whisper models, and compared them with Meta’s Wav2vec 2.0 model. Additionally, we investigated two approaches for adapting these models under limited data conditions: providing contextual input to Whisper and fine-tuning both Whisper and Wav2vec 2.0. Finally, we examined the models’ ability to recognise medical terminology used in pathology reports, focusing on two categories: anatomical and pathology terms. Our findings indicate that providing contextual information to Whisper models significantly improves both the overall average word error rate (WER) and the term error rate (TER), with reductions ranging from 17% to 48% compared to default and fine-tuned models. The best overall performance was demonstrated by the Whisper large-v2 model, which achieved an average WER of 0.06.