Nowadays, the concept of 'interaction' has been emphasized in the area of design of consumer products, communication devices, and computer systems. Users can communicate or use a product through various interactions between the system and users. The multi-modal functions of a product are providing diverse ways of interactions including visual, auditory and tactile modalities. However, since the design guidelines for creating good interface design haven't been developed sufficiently, it is difficult for designers to collect the information of multi-modal resources, and to construct the way of integrating multi-modalities for a specific product. There is, therefore, a need for developing a systematic study on the interaction design in order to understand the effectiveness of multi-modal interfaces and to maximize the interactivity between the system and users.
The goal of this study is to establish guidelines of developing multi-modal interface design systematically, and to propose a new interface design process by two approaches: cognitive and experimental. Through a cognitive approach, this study investigated the characteristics of human-system interactions and the information processing model. Through the literature review, the multiple-resource theory and the modality compatibility have been emphasized. The former reveals that the spatial process and verbal one in the working memory do not interfere each other. The latter suggests that the composition of stimulus-response are dependent on the task. On the basis of these background studies, the taxonomy of modality and guidelines for effective interaction design are proposed.
For an experimental approach, a simulation tool called 'MIDS(Multi-modal Interface Design Simulator)' was developed. It consists of 'object library', 'modality composition module', 'entity arrangement module', and 'test module'. The concept of MIDS is to make virtual rapid prototypes by selecting and editing interface designs from the library, and developing creative ideas. By using MIDS, designers can make several multi-modal prototypes and choose better one by simulating the whole interactions. By using the object-library which is extracted from the currently used interfaces within products, designers can develop interface design prototypes easily without programming.
