One of the most fundamental open questions in modern extragalactic research is how much interstellar dust exists in distant galaxies and how much star-formation activities are obscured behind interstellar dust in the high-redshift Universe. Using the unprecedented sensitivity and spatial-resolution of Atacama Large Millimeter/submillimeter Array (ALMA), the first investigations of dust emission from “normal” star-forming galaxies in the early Universe have been enabled in the last decade. However, before the start of this thesis, early results from a small number of observations still remained controversial. Observations typically only reached shallow sensitivity using short snapshots. Deep observations of larger galaxy samples are clearly needed to obtain statistically robust results. During this thesis, I have built up increasingly larger samples and collected deeper observations to study the evolution of dust attenuation in galaxies within the first 2 Gyr of cosmic history.