• Home
  1. Home
  2. Online Evaluation of Coreference Resolution
Proceedings chapter
OA Policy
English

Online Evaluation of Coreference Resolution

Contributors

Popescu-Belis, Andréi; Rigouste, Loïs; Salmon-Alt, Susanne; Romary, Laurent

Presented at

Lisbon (Portugal)

Published in

Maria Teresa Lino, Maria Francisca Xavier, Fátima Ferreira, Rute Costa, Raquel Silva (Ed.), LREC 2004 (Fourth International Conference on Language Resources and Evaluation), p. 1507-1510

Publisher

ELRA - European Language Ressources Association

Publication date

2004

Abstract

This paper presents the design of an online evaluation service for coreference resolution in texts. We argue that coreference, as an equivalence relation between referring expressions (RE) in texts, should be properly distinguished from anaphora and has therefore to be evaluated separately. The annotation model for coreference is based on links between REs. The program presented in this article compares two such annotations, which may be the output of coreference resolution tools or of human judgement. In order to evaluate the agreement between the two annotations, the evaluator first converts the input annotation format into a pivot format, then abstracts equivalence classes from the links and provides five scores representing in different ways the similarity between the two partitions: MUC, B3, Kappa, Core-discourse-entity, and Mutual-information. Although we consider that the identification of REs (i.e. the elements of the partition) should not be part of coreference resolution properly speaking, we propose several solutions for the frequent case when the input files do not agree on the elements of the text to consider as REs.

UNIGE affiliation entities

UNIGE research groups

Citation (ISO format)

POPESCU-BELIS, Andréi et al. Online Evaluation of Coreference Resolution. In: LREC 2004 (Fourth International Conference on Language Resources and Evaluation). Maria Teresa Lino, Maria Francisca Xavier, Fátima Ferreira, Rute Costa, Raquel Silva (Ed.). Lisbon (Portugal). [s.l.] : ELRA - European Language Ressources Association, 2004. p. 1507–1510.

Main files (1)

Proceedings chapter (Published version)

Identifiers

  • PID : unige:2262
742views
447downloads

Technical informations

Creation

29/07/2009 10:52:00

First validation

29/07/2009 10:52:00

Update

14/03/2023 15:09:44

Status update

14/03/2023 15:09:44

Last indexation

29/10/2024 11:49:20
All rights reserved by Archive ouverte UNIGE and the University of Geneva
OpenAIRE Indexed & Validated