Scientific article
English

The Choice of Textual Knowledge Base in Automated Claim Checking

Published inACM journal of data and information quality, vol. 15, no. 1, 3
Publication date2023-03-02
First online date2023-03-02
Abstract

Automated claim checking is the task of determining the veracity of a claim given evidence retrieved from a textual knowledge base of trustworthy facts. While previous work has taken the knowledge base as given and optimized the claim-checking pipeline, we take the opposite approach—taking the pipeline as given, we explore the choice of the knowledge base. Our first insight is that a claim-checking pipeline can be transferred to a new domain of claims with access to a knowledge base from the new domain. Second, we do not find a “universally best” knowledge base—higher domain overlap of a task dataset and a knowledge base tends to produce better label accuracy. Third, combining multiple knowledge bases does not tend to improve performance beyond using the closest-domain knowledge base. Finally, we show that the claim-checking pipeline’s confidence score for selecting evidence can be used to assess whether a knowledge base will perform well for a new set of claims, even in the absence of ground-truth labels.

Keywords
  • Automated claim verification
  • Textual knowledge bases
  • Evidence selection
  • Information retrieval
  • Content management
Citation (ISO format)
STAMMBACH, Dominik, ZHANG, Boya, ASH, Elliott. The Choice of Textual Knowledge Base in Automated Claim Checking. In: ACM journal of data and information quality, 2023, vol. 15, n° 1, p. 3. doi: 10.1145/3561389
Main files (1)
Article (Published version)
accessLevelRestricted
Identifiers
Additional URL for this publicationhttps://dl.acm.org/doi/10.1145/3561389
Journal ISSN1936-1963
107views
0downloads

Technical informations

Creation03/03/2023 12:36:19
First validation06/10/2023 15:22:30
Update06/10/2023 15:22:30
Status update06/10/2023 15:22:30
Last indexation01/11/2024 06:17:42
All rights reserved by Archive ouverte UNIGE and the University of GenevaunigeBlack