Scientific article

A reference data set for the evaluation of medical image retrieval systems

Published inComputerized medical imaging and graphics, vol. 28, no. 6, p. 295-305
First online date2004-07-02

Content-based image retrieval is starting to become an increasingly important factor in medical imaging research and image management systems. Several retrieval systems and methodologies exist and are used in a large variety of applications from automatic labelling of images to diagnostic aid and image classification. Still, it is very hard to compare the performance of these systems as the used databases often contain copyrighted or private images and are thus not interchangeable between research groups, also for patient privacy. Most of the currently used databases for evaluating systems are also fairly small which is partly due to the high cost in obtaining a gold standard or ground truth that is necessary for evaluation. Several large image databases, though without a gold standard, start to be available publicly, for example by the NIH (National Institutes for Health).

This article describes the creation of a large medical image database that is used in a teaching file containing more than 8,700 varied medical images. The images are anonymised and can be exchanged free of charge and copyright. Ground truth (a gold standard) has been obtained for a set of 26 images being selected as query topics for content-based query by image example. To reduce the time for the generation of ground truth, pooling methods well known from the text or information retrieval field have been used. Such a database is a good starting point for comparing the current image retrieval systems and to measure the retrieval quality, especially within the context of teaching files, image case databases and the support of teaching. For a comparison of retrieval systems for diagnostic aid, specialised image databases, including the diagnosis and a case description will need to be made available, as well, including gold standards for a proper system evaluation.

A first evaluation event for image retrieval is foreseen at the 2004 CLEF conference (Cross Language Evaluation Forum) to compare text-and content-based access mechanism to images.

  • Databases, Factual
  • Diagnostic Imaging
  • Humans
  • Information Storage and Retrieval / methods
  • Radiology Information Systems
  • Software
  • User-Computer Interface
Citation (ISO format)
MULLER, Henning et al. A reference data set for the evaluation of medical image retrieval systems. In: Computerized medical imaging and graphics, 2004, vol. 28, n° 6, p. 295–305. doi: 10.1016/j.compmedimag.2004.04.005
Main files (1)
Article (Published version)
ISSN of the journal0895-6111

Technical informations

Creation11/30/2022 12:55:56 PM
First validation05/16/2023 8:33:29 AM
Update time05/16/2023 8:33:29 AM
Status update05/16/2023 8:33:29 AM
Last indexation05/06/2024 3:52:34 PM
All rights reserved by Archive ouverte UNIGE and the University of GenevaunigeBlack