The Prosody-Semantics Tango: Insights from 15 Years of Exploring Emotional Speech Processing Across Populations

ABSTRACT The Test for Rating Emotions in Speech (T-RES) has been developed in order to assess the complex interplay of prosody (tone of speech) and semantics (verbal content) in the processing of emotions in spoken language. Listeners are asked to rate the extent of a predefined emotion in spoken sentences, which include emotional content (anger, happiness and sadness) in both semantics and prosody. To date, English, German and Hebrew versions have been developed. The T-RES has been tested in people with 1) Tinnitus, 2) cochlear implants, 3) forensic and non-forensic schizophrenia, 4) Autism Spectrum Disorders, 5) Intellectual and developmental disabilities, 5) early and late onset of blindness and of 6) deafness. The test has also been tested with older adults and elementary school children. This presentation will provide an overview of the tool, as well as its main findings across languages and populations. First, prosody and semantics are separate but not separable dimensions, with similar trends across languages. Second, performance for young typically developed adults, across languages, indicates prosodic dominance, i.e. prosodic information plays a greater role in processing emotional speech than semantics. For some groups, prosodic dominance is diminished, possibly as a compensation mechanism for auditory and cognitive difficulties. Third, it appears that early exposure to intact spoken language (visual and auditory) is important for the development of spoken-emotion processing. Fourth, auditory and cognitive difficulties appear to impair spoken emotion processing. Finally, group-differences in prosody-semantics interactions can lead to communication breakdowns, with implications on social interactions and health services.