The increasing availability of DNA from prehistoric skeletal remains changed drastically the focus of the human population genetics field during the last decade. It is now possible to use ancient DNA to formally test evolutionary hypotheses built by inter-disciplinary studies on the basis of archaeological or anthropological information. Spatially explicit simulations offer the advantage of taking into account the spatial and temporal dynamics of populations by representing explicitly the migration of individuals within the geographic area under study. This PhD thesis is a first attempt to analyse ancient genetic material with spatially explicit simulations. Using this original approach, we analysed a large dataset of ancient, and modern, mitochondrial DNA from European populations, under various evolutionary hypotheses. We focused on the relation between past populations during the period of the Neolithic transition that started in Europe around 8,000 years ago.