Modality Effects on Memory for Multimedia Messages

Student researchers

Eric Jones & Sriram Kalyanaraman (PhD Students)

Faculty Supervisor

Dr. S. Shyam Sundar

For a complete report of this research, see:

Sundar, S. S., Kalyanaraman, S., & Jones, E. (2000, July). Modality effects on memory for multimedia messages. Paper presented to the Sociology and Social Psychology Division at the 23rd General Assembly and Scientific Conference of the International Association for Media and Communication Research (IAMCR), Singapore.

Introduction

The growing importance of the World Wide Web (WWW) offers interesting possibilities for exporing differences between different modalities. While traditional communication research has largely examined the differences between radio and TV, such research has also confounded modality with medium. The computer, on the other hand, offers the unique advantage of being able to combine different modalities such as text, audio, and video within a single medium. Drawing on the Dual Coding Theory (DCT), which suggests that different modalities are processed in separate sub-systems of the brain with the conclusion that two modalities are more memorable than one, this study explored the effects of two modalities (text-audio) versus a single modality (text only) on memory.

Hypotheses

Based on the DCT, it was hypothesized that people who were exposed to a message with two modalities would have higher memory scores than people who were exposed to the same message with only a single modality.

Method

Thirty-one participants were randomly assigned to one of two conditions (text, text-audio) in a between-participants experiment. Participants were exposed to a motivational message, and equivalence was maintained across both the experimental conditions except that participants in the text-audio condition listened to the message while simultaneously reading it, whereas participants in the text condition read only the message on a computer screen. After message exposure, participants were asked to fill out a questionnaire containing measures of memory.

Results

Hypothesis: Supported. An analysis of the data revealed that there was a significant difference in memory between the two experimental conditions. Specifically, participants in the text-audio condition remembered significantly more aspects of the message than did participants in the text-only condition. This can be seen from the graph below, where TA refers to text-audio, and TO refers to text-only.

Conclusions

An important implication of this study is that the computer can be used to control for medium differences and at the same time explore modality differences. Also, the results indicate that two modalities are preferable to a single modality. This is of enormous practical significance to Web designers as they make decisions regarding the optimal use of multimedia features offered by the new medium.

For more details regarding the study contact

Dr. S. Shyam Sundar by e-mail at sss12@psu.edu or by telephone at (814) 865-2173

>> Return to abstract listings page
>> Return to the main research page

Media Effects Research Lab at College of Communications, Penn State University