A Vocalisation-based Drawing Interface for Disabled Children

In our work with disabled children at Ireland’s National Rehabilitation Hospital, a problem we have experienced in the facilitation of art activities is that traditional art materials and standard computer drawing programs sometimes prove inaccessible. In this paper, an original system, called “PaintMyVoice” is presented which facilitates the creation of two or three-dimensional images using a variety of novel input modalities. In particular, vocalisations can be used to create original images of a variety of objects, including trees, flowers and landscape elements. Additional input to the system can optionally be provided via mouse, keyboard, switch interface or digital camera depending on the abilities of the user. Here, the program’s user interface is described, with an emphasis on accessibility features. The signal processing techniques used to measure various vocal characteristics including intensity, pitch and other spectral characteristics are outlined. The means of translation from vocalisation to visual representation is also explained for each type of object discussed. This technology facilitates artistic expression by all children, but especially those with severe physical and/or intellectual disabilities. Furthermore, in certain cases, it may be used to provide motivation in therapeutic vocal exercises. Finally, the results of initial user trials are presented.