en
Proceedings chapter
Open access
English

Butterfly effects in frame semantic parsing : impact of data processing on model ranking

Published inProceedings of the 27th International Conference on Computational Linguistics, Editors Bender, E.B., Derczynski, L. & Isabelle, P., p. 12
Presented at Santa Fe, New Mexico (USA), August 2018
Publication date2018
Abstract

Knowing the state-of-the-art for a particular task is an essential component of any computational linguistics investigation. But can we be truly confident that the current state-of-the-art is indeed the best performing model? In this paper, we study the case of frame semantic parsing, a well- established task with multiple shared datasets. We show that in spite of all the care taken to provide a standard evaluation resource, small variations in data processing can have dramatic consequences for ranking parser performance. This leads us to propose an open-source standardized processing pipeline, which can be shared and reused for robust model comparison.

Citation (ISO format)
KABBACH, Alexandre, RIBEYRE, Corentin, HERBELOT, Aurélie. Butterfly effects in frame semantic parsing : impact of data processing on model ranking. In: Proceedings of the 27th International Conference on Computational Linguistics. Santa Fe, New Mexico (USA). [s.l.] : [s.n.], 2018. p. 12.
Main files (1)
Proceedings chapter (Published version)
Identifiers
  • PID : unige:152534
107views
12downloads

Technical informations

Creation06/22/2021 8:35:00 AM
First validation06/22/2021 8:35:00 AM
Update time03/16/2023 12:47:00 AM
Status update03/16/2023 12:46:59 AM
Last indexation05/06/2024 7:44:59 AM
All rights reserved by Archive ouverte UNIGE and the University of GenevaunigeBlack