UNIGE document Chapitre d'actes
previous document  unige:105543  next document
add to browser collection
Title

Colorless green recurrent networks dream hierarchically

Authors
Bojanowski, Piotr
Grave, Edouard
Linzen, Tal
Baroni, Marco
Published in Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018. New Orleans, Louisiana - June 2018 - Association for Computational Linguistics. 2018, p. 1195–1205
Abstract Recurrent neural networks (RNNs) have achieved impressive results in a variety of linguistic processing tasks, suggesting that they can induce non-trivial properties of language. We investigate here to what extent RNNs learn to track abstract hierarchical syntactic structure. We test whether RNNs trained with a generic language modeling objective in four languages (Italian, English, Hebrew, Russian) can predict long-distance number agreement in various constructions. We include in our evaluation nonsensical sentences where RNNs cannot rely on semantic or lexical cues ("The colorless green ideas I ate with the chair sleep furiously"), and, for Italian, we compare model performance to human intuitions. Our language-model-trained RNNs make reliable predictions about long-distance agreement, and do not lag much behind human performance. We thus bring support to the hypothesis that RNNs are not just shallow-pattern extractors, but they also acquire deeper grammatical competence.
Keywords Computation and Language
Identifiers
Full text
Structures
Research group Laboratoire d'Analyse et de Traitement du Langage (LATL)
Citation
(ISO format)
GULORDAVA, Kristina et al. Colorless green recurrent networks dream hierarchically. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018. New Orleans, Louisiana. [s.l.] : Association for Computational Linguistics, 2018. p. 1195–1205. https://archive-ouverte.unige.ch/unige:105543

17 hits

3 downloads

Update

Deposited on : 2018-06-13

Export document
Format :
Citation style :