Towards incremental learning of word embeddings using context informativeness

Kabbach, Alexandre; Gulordava, Kristina; Herbelot, Aurélie

doi:10.18653/v1/P19-2022

Proceedings chapter

English

Towards incremental learning of word embeddings using context informativeness

Contributors

Kabbach, Alexandre

; Gulordava, Kristina; Herbelot, Aurélie

Presented at

Florence (Italy), July 2019

Published in

Alva-Manchego, F., Choi, E. & Khashabi, D. (Ed.), Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics : Student Research Workshop, p. 162-168

Publisher

Association for Computational Linguistics

Publication date

2019

Abstract

In this paper, we investigate the task of learning word embeddings from very sparse data in an incremental, cognitively-plausible way. We focus on the notion of ‘informativeness', that is, the idea that some content is more valuable to the learning process than other. We further highlight the challenges of online learning and argue that previous systems fall short of implementing incrementality. Concretely, we incorporate informativeness in a previously proposed model of nonce learning, using it for context selection and learning rate modulation. We test our system on the task of learning new words from definitions, as well as on the task of learning new words from potentially uninformative contexts. We demonstrate that informativeness is crucial to obtaining state-of-the-art performance in a truly incremental setup.

UNIGE affiliation entities

Faculté des lettres / Département de linguistique

Citation (ISO format)

KABBACH, Alexandre, GULORDAVA, Kristina, HERBELOT, Aurélie. Towards incremental learning of word embeddings using context informativeness. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics : Student Research Workshop. Alva-Manchego, F., Choi, E. & Khashabi, D. (Ed.). Florence (Italy). [s.l.] : Association for Computational Linguistics, 2019. p. 162–168. doi: 10.18653/v1/P19-2022

Proceedings chapter (Published version)

CC BY

Identifiers

PID : unige:152014
DOI : 10.18653/v1/P19-2022

Additional URL for this publication

https://www.aclweb.org/anthology/P19-2022

297views

213downloads

Creation

01/06/2021 14:15:00

First validation

01/06/2021 14:15:00

Update

16/03/2023 00:40:46

Status update

16/03/2023 00:40:46

Last indexation

31/10/2024 22:13:27

Towards incremental learning of word embeddings using context informativeness

Contributors

Presented at

Published in

Publisher

Publication date

Abstract

UNIGE affiliation entities

Citation (ISO format)

Main files (1)

Identifiers

Additional URL for this publication

Technical informations

Creation

First validation

Update

Status update

Last indexation