There is currently much interest in the organization and content-based querying image databases. The usual hypothesis is that image similarity can be characterized by low-level features, without further abstraction. This assumes that agreement between machine and human measures of similarity is sufficient for the database to be useful. To assess this assumption, we develop measures of the agreement between partitionings of an image set, showing that chance agreements must be considered. These measures are used to assess the agreement between human subjects and several machine clustering techniques on an image set. The results can be used to select and refine distance measures for querying and organizing image databases.