Comparing Computer Versus Human Data Collection Methods for Public Usability Evaluations of a Tactile-Audio Display

We present a public usability study that provides preliminary results on the effectiveness of a universally designed system that conveys music and other sounds into tactile sensations. The system was displayed at a public science museum as part of a larger multimedia exhibit aimed at presenting a youths’ perspective on global warming and the environment. We compare two approaches to gathering user feedback about the system in a study that we conducted to assess user responses to the inclusion of a tactile display within the larger audio-visual exhibit; in one version, a human researcher administered the study and in the other version a touch screen computer was used to obtain responses. Both approaches were used to explore the public’s basic understanding of the tactile display within the context of the larger exhibit.

The two methods yielded very similar responses from participants; however, our comparison of the two techniques revealed that there were subtle differences overall. In this paper, we compare the two study techniques for their value in providing access to public usability data for assessing universally designed interactive systems. We present both sets of results, with a cost benefit analysis of using each in the context of public usability tests for universal design.