More than one gesture but less than two? Inter-stroke dependencies in form and meaning

Schuyler Laparle (1) , Gaëlle Ferré (2) , Merel Scholman (3)

1 Tilburg University [Netherlands]
2 FoReLLIS [Poitiers] - Formes et représentations en linguistique, littérature et dans les arts de l’image et de la scène
3 Universiteit Utrecht / Utrecht University [Utrecht]

In gesture analysis, one must segment the gesture stream into discrete units to be analyzed for meaning and alignment with accompanying speech. This requires defining what constitutes a single “gesture”. Following the influential work of Kendon (2004) and McNeill (2005), this nearly always involves identifying the “stroke”, the gesture’s meaningful core. Innovations in the understanding of gestural meaning complicate this procedure, as there is increasing evidence that gestures are both internally complex and compositional. In the present work, we discuss the ways in which this complexity in gestural meaning may be addressed by including the notion of “gesture sequences”. Using data from American television talk shows and TED talks, we demonstrate two types of interstroke dependencies in pragmatic gestures where one movement’s meaning and form is dependent on that of a previous movement: (i) “closing gestures” signal the closure of a discourse topic and are dependent on a preceding presentation gesture; (ii) “contrast sequences” signal a contrast discourse relation and are dependent on adjacent presentation gestures occurring in different regions of gesture space. These findings have important implications for both how we annotate the gesture stream and how we simulate gesture in virtual agents.

Mots clés: Discourse; Coherence; contrast; topic closure; Gesture

Domaines: Sciences de l'Homme et Société

Schuyler Laparle, Gaëlle Ferré, Merel Scholman. More than one gesture but less than two? Inter-stroke dependencies in form and meaning. 26th International Conference on Human-Computer Interaction, Jun 2024, Washington DC, United States. ⟨hal-04433349⟩