ASL Fingerspelling: Can Looking at Hands Help? An Eye-Tracking Study

This research consists of an eye-tracking study examining the efficacy of eye gaze indexing (EGIX) in manipulating viewer eye gaze and enhancing second language (L2) fingerspelling comprehension in American Sign Language (ASL) through a controlled laboratory experiment. The study consisted of two groups and two conditions, EGIX+/EGIX- to test the effect of EGIX on participant eye gaze (EG) behaviors and fingerspelling comprehension using eye-tracking software and a comprehension quiz. The results indicate that participant EG was the same regardless of whether the signer used EGIX.The results also indicated that participant comprehension scores were the same regardless of whether the model used EGIX. Several statistical analyses of comprehension and EG metrics found that as hand fixation duration, mouth fixation duration, number of hand fixations, and number of mouth revisits increased, comprehension performance decreased. On the other hand, the Area of Interest metrics did not affect performance or only revealed weak trends. The decreases in comprehension may highlight that the students who struggled with comprehension looked more and longer at the mouth and hand as coping strategies to try to glean additional information from mouth grammar or that they were struggling to identify each letter handshape, rather than a causal relationship with EGIX. Word length effect on comprehension was statistically significant, though varied by word length. Importantly, constraints from name origin may have played a role in the distribution of the comprehension decrease since words of Greek origins cause greater statistically significant reductions in performance. The qualitative results show that students have a keen awareness of where they look while viewing signed videos. Noticing and perceived helpfulness did not show statistically significant impacts on performance universally. However, some students who reported noticing and reported the EGIX+ as helpful increased their score by 10% or higher on the mean comprehension of EGIX+ words. The pedagogical takeaway is that the benefits of using EGIX to help novice to intermediate signers with fingerspelling comprehension are inconclusive. EGIX+ may provide substantial benefits for some individuals, but the effects are not generalizable.