Researchers work together to improve communication-aiding technologies

UNIVERSITY PARK, Pa. — Deciding which hand to use for communication-aiding technology is a cognitive task that can slow the user down, potentially inhibiting communication and performance, according to a Penn State study.

More than four million Americans who have such complex disabilities that they cannot meet their communication needs through their own speech can benefit from the use of aided Augmentative and Alternative Communication, or AAC, which is the use of computers, tablets or mobile devices to supplement or replace speech or writing. Because many individuals access these devices through hand reaches, it is necessary to consider hand or arm movements in design of AAC tools and systems.

"Crossing midline, or reaching an arm over the body, is more 'costly' — uses more time and effort," said Robert Sainburg, professor of kinesiology and neurology, and co-author of a study that recently appeared in the journal Neuroscience.

The researchers' findings suggest that cognitive processes are normally engaged to avoid costly actions, and that choosing the best (lowest-cost) response takes cognitive effort, even though one does not normally associate such a seemingly easy choice with cognition.

Previous studies suggest that selecting which hand to use for a reaching task appears to be modulated by a factor described as "task difficulty." However, what features of a task might contribute to greater or lesser "difficulty" in the context of hand-selection decisions has yet to be determined.

In the current study, the researchers found that visual display complexity affects reaction time, and cognitive-perceptual load. Additionally, cognitive-perceptual load interferes with hand-selection decisions and increases the frequency of contralateral — across body — reaches.

"We hypothesized that this reaction for hand-selection should recruit cognitive resources and thus should be influenced by cognitive-perceptual loading," Sainburg said.

Researchers tested this hypothesis by adapting a visual search task that presented participants with visual displays that varied in visual-perceptual characteristics. The task required participants to select a just-prompted target stimulus from an array of 16 symbols.

The symbols consisted of four items worn on the feet (sandals, boots), four items worn on the torso (t-shirt, dress shirt), four summertime items (bathing suits, sunglasses), and four inclement weather items (raincoat, warm hat).

Each set of items had loose categorical relations to one another, in addition to sharing color. Their physical shapes were not similar; for instance, the baseball cap and the swimsuits were both red, but quite different in shape, said Krista Wilkinson, professor of communication sciences and disorders, and co-leader of the study.

Jiali Liang, a doctoral student in the Department of Communication Sciences and Disorders, was also an investigator in the study.

Researchers examined how and whether the visual-perceptual characteristics of the different displays influenced motor behavior during the selection of the target.

Participants were 11 right-handed young adults, who were enrolled in college or who had gained a college degree, with an average age of 24 years.

The researchers' data suggested an influence of cognitive-perceptual load on hand selection, such that poor decisions that required more time and energy were made, as cognitive load increased. This included making more cross-midline reaches. In other words, participants chose the "wrong hand" more often when their attention and cognition was diverted to other aspects of the task.

"These findings indicate that choosing which hand we use to reach to something is a cognitive choice that can suffer when our cognition is focused on other things," Sainburg said. "So, seemingly simple motor tasks can suffer when our attentional resources are diverted. In everyday experience, this is one reason why smart phone use during walking or driving can be so dangerous: It not only diverts important visual attention resources, but can also result in bad, possibly dangerous, motor choices too."

This work was supported by the National Institutes of Health and a Penn State Social Science Research Institute Award.