Image retrieval has long been plagued by limitations on automatic methods because they cannot reliably extract semantic data from low-level features. The result is that users must formulate awkward and inefficient queries in terms these systems can understand. Humans, on the other hand, have the ability to quickly and accurately summarise visual data. This dichotomy, named the semantic gap, is a fundamental problem in image retrieval. We aim to narrow the semantic gap in a typical retrieval scenario by motivating users to provide semantic image annotations. We propose a system of collecting image annotations based on the need for human verification on the web. Similar in principle to work by von Ahn et al. [2, 3], the idea is to exploit the requirement of users to pass tests in order to incrementally annotate images.