Visual lip information supports auditory word segmentation

Antje Strauß 1, * Christophe Savariaux 2, 3 Sonia Kandel 4 Jean-Luc Schwartz 1
* Corresponding author
1 GIPSA-PCMD - PCMD
GIPSA-DPC - Département Parole et Cognition
2 GIPSA-Services - GIPSA-Services
GIPSA-lab - Grenoble Images Parole Signal Automatique
3 GIPSA-CRISSP - CRISSP
GIPSA-DPC - Département Parole et Cognition
4 GIPSA-VSLD - VSLD
GIPSA-DPC - Département Parole et Cognition
Abstract : Word segmentation is one of the initial processes that needs to be solved when acquiring the first or learning a second language. Acoustic cues like the fundamental frequency and segment durations have been shown to facilitate the detection of word boundaries. The role of visual speech and in particular of lip movements in word segmentation is still rather unknown. In French, liaisons, e.g. between determiner and noun, often pose a problem of several segmentation possibilities (e.g., the sequence /lafiS/ with liaison ("l’affiche") means the poster whereas without liaison ("la fiche") it means the file.). Here, we use 17 ambiguous French sequences with and without liaison. They were presented in carrier sentences either with clear acoustic cues for the first or the second segmentation possibility or with ambiguous acoustic cues. The three audio conditions were combined with lip movements hyper-articulating either the first or the second segmentation possibility in order to observe the influence of visual information on segmentation. Theparticipants had to indicate as quickly as possible which of the two versions they understood (e.g., "l’affiche" or "la fiche"?). Results show that lip information indeed biases the word segmentation decision. These data provide important implications for audiovisual integration processes.
Complete list of metadatas

https://halshs.archives-ouvertes.fr/halshs-01298351
Contributor : Antje Strauss <>
Submitted on : Tuesday, April 5, 2016 - 5:49:12 PM
Last modification on : Tuesday, June 11, 2019 - 4:29:31 PM
Long-term archiving on : Wednesday, July 6, 2016 - 3:40:19 PM

File

Strauss_AVSP2015.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : halshs-01298351, version 1

Citation

Antje Strauß, Christophe Savariaux, Sonia Kandel, Jean-Luc Schwartz. Visual lip information supports auditory word segmentation. 1st Joint Conference on Facial Analysis, Animation and Auditory-Visual Speech Processing (FAAVSP 2015), Sep 2015, Vienna, Austria. ⟨halshs-01298351⟩

Share

Metrics

Record views

591

Files downloads

162