Are Transformers a Modern Version of ELIZA? Observations on French Object Verb Agreement - HAL-SHS - Sciences de l'Homme et de la Société Accéder directement au contenu
Communication Dans Un Congrès Année : 2021

Are Transformers a Modern Version of ELIZA? Observations on French Object Verb Agreement

Bingzhi Li
  • Fonction : Auteur
  • PersonId : 1095862
Benoît Crabbé

Résumé

Many recent works have demonstrated that unsupervised sentence representations of neural networks encode syntactic information by observing that neural language models are able to predict the agreement between a verb and its subject. We take a critical look at this line of research by showing that it is possible to achieve high accuracy on this agreement task with simple surface heuristics, indicating a possible flaw in our assessment of neural networks' syntactic ability. Our fine-grained analyses of results on the long-range French objectverb agreement show that contrary to LSTMs, Transformers are able to capture a non-trivial amount of grammatical structure.
Fichier principal
Vignette du fichier
main.pdf (196.75 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

halshs-03755089 , version 1 (21-08-2022)

Identifiants

Citer

Bingzhi Li, Guillaume Wisniewski, Benoît Crabbé. Are Transformers a Modern Version of ELIZA? Observations on French Object Verb Agreement. 2021 Conference on Empirical Methods in Natural Language Processing, Nov 2021, Punta Cana, Dominican Republic. ⟨10.18653/v1/2021.emnlp-main.377⟩. ⟨halshs-03755089⟩
42 Consultations
97 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More