Visual Salience and Perceptual Grouping in Multimodal Interactivity

Frédéric Landragin 1 Nadia Bellalem 1 Laurent Romary 1
1 LANGUE ET DIALOGUE - Human-machine dialogue with a significant language component
INRIA Lorraine, LORIA - Laboratoire Lorrain de Recherche en Informatique et ses Applications
Abstract : This paper deals with the pragmatic interpretation of multimodal referring expressions in man-machine dialogue systems. We show the importance of building up a structure of the visual context at a semantic level, in order to enrich the significant possibilities of interpretations and to make possible the fusion of this structure with the ones obtained from the linguistic and gesture semantic analyses. Visual salience and perceptual grouping are two notions that guide such a structuring. We thus propose a hierarchy of salience criteria linked to an algorithm that detects salient objects, as well as guidelines for grouping algorithms. We show how the integration of the results of all these algorithms is a complex problem. We propose simple heuristics to reduce this complexity and we conclude on the usability of such heuristics in actual systems.
Type de document :
Communication dans un congrès
First International Workshop on Information Presentation and Natural Multimodal Dialogue, 2001, pp.151-155, 2001
Liste complète des métadonnées

Littérature citée [9 références]  Voir  Masquer  Télécharger

https://halshs.archives-ouvertes.fr/inria-00100576
Contributeur : Frédéric Landragin <>
Soumis le : mardi 7 novembre 2006 - 16:47:52
Dernière modification le : jeudi 11 janvier 2018 - 06:19:48
Document(s) archivé(s) le : mardi 6 avril 2010 - 21:48:04

Fichiers

Identifiants

  • HAL Id : inria-00100576, version 2

Collections

Citation

Frédéric Landragin, Nadia Bellalem, Laurent Romary. Visual Salience and Perceptual Grouping in Multimodal Interactivity. First International Workshop on Information Presentation and Natural Multimodal Dialogue, 2001, pp.151-155, 2001. 〈inria-00100576v2〉

Partager

Métriques

Consultations de la notice

537

Téléchargements de fichiers

306