We consider the problem of fine-grained physical object recognition and introduce a dataset PharmaPack con- taining 1000 unique pharma packages enrolled in a controlled environment using consumer mobile phones as well as several recognition sets representing various scenarios. For perfor- mance evaluation, we extract two types of recently proposed local feature descriptors and aggregate them using popular tools. All enrolled raw and pre-processed images, extracted and aggregated descriptors are made public to promote re- producible research. To evaluate the baseline performance, we compare the methods based on aggregation of local descriptors with methods based on geometrical matching.