A Comparison of Hand and Computer Coding of Expressions of Emotion

Hand-coding systems of measuring facial expressions were developed to study and analyze human emotions, but they are time-intensive and thus seldom used. As technology has advanced, new computer software programs, such as Affectiva, were developed to code facial expressions automatically using artificial intelligence and machine learning. Since this technology is still new, Affectiva and its validity remain understudied, and no psychological research has been conducted to compare Affectiva computer coding and hand coding of children’s emotions. The purpose of this study was to compare hand and computer coding of children’s expressions of emotion during a videotaped parent-child interaction. The study answered the following questions: 1) Do hand and computer coding agree?; and 2) Are hand and computer coding in higher agreement for some emotions than others? The sample included 25 pairs of twins from the Arizona Twin Project. Facial expressions were coded from videotape by a trained and reliable human coder and using the software Affectiva. The results showed that hand and computer coded emotion were in agreement for positive, but not negative emotions. Changing the context of the interaction to elicit more negative emotion, and using the same indicators of each emotion in computer and hand coding are suggested to improve the comparison of computer and hand coding.