Degrees of freedom of facial movements in face-to-face conversational speech

Gérard Bailly (1) , Frédéric Elisei (1) , Pierre Badin (1) , Christophe Savariaux (1)

1 ICP - Institut de la communication parlée

In this paper we analyze the degrees of freedom (DoF) of facial movements in face-to-face conversation. We propose here a method for automatically selecting expressive frames in a large fine-grained motion capture corpus that best complement an initial shape model built using neutral speech. Using conversational data from one speaker, we extract 11 DoF that reconstruct facial deformations with a average precision less than a millimeter. Gestural scores are then built that gather movements and discursive labels. This modeling framework offers a productive analysis of conversational speech that seeks in the multimodal signals the rendering of given communicative functions and linguistic events.

Mots clés: Facial movements; model-based face tracking; expressive audiovisual speech

Domaines: Sciences de l'information et de la communication; Traitement du signal et de l'image [eess.SP]; Traitement du signal et de l'image [eess.SP]

Gérard Bailly, Frédéric Elisei, Pierre Badin, Christophe Savariaux. Degrees of freedom of facial movements in face-to-face conversational speech. International Workshop on Multimodal Corpora, 2006, Genoa, Italy. pp.33-36. ⟨hal-00195551⟩