Prosodic and Semantic Effects on the Perception of Mixed Emotions

Abstract

In two experiments, we examined the perception of mixed happy-sad emotions elicited by a combination of prosodic voice cues (pitch and tempo), and sentence content (semantics) in speech. In Experiment 1, participants rated sentences with happy, sad, and neutral semantic content spoken by a female talker on a 6-point Likert scale measuring happiness and sadness. Prosodic cues that characterize happy (high pitch, fast tempo) and sad (low pitch, slow tempo) emotions elicited the highest ratings when these cues were congruent with the semantic content of sentences. When the prosodic cues were incongruent with the sentence meaning, happiness and sadness ratings declined. Emotion ratings for mixed emotions (high pitch, slow tempo; and low pitch, fast tempo) received intermediate ratings in comparison to those signalling pure happy and sad emotions and were less affected by the semantic content of sentences. In Experiment 2, we used eye-tracking to examine the pattern of attention directed toward dynamic facial information in talker’s expressions of happy, sad, and mixed emotions. An analysis of the mean number of eye gaze fixations and the duration of eye gaze responses indicated that participants attended primarily to the eye regions and less to the mouth regions of the face. Furthermore, attention towards the eye region was affected by semantic content and prosodic pitch-tempo combinations, but attention towards the mouth region was not. The findings from the current study are expected to extend our knowledge on the mechanisms affecting the perception of basic and mixed emotions in normal populations and in special populations with social-emotional deficits.

Discipline: Psychology (Honours)

Faculty Mentor: Dr. Tara Vongpaisal