The purpose of this paper is to provide an improved (up-to-date) insight into the environmental burden of textiles made of the base materials cotton, polyester (PET), nylon, acryl, and elastane. The research question is: Which base material and which life cycle stage (cradle-to-gate as well as cradle-to-grave) have the biggest impact on the environment? Methods: Life cycle inventory (LCI) data are collected from the literature, life cycle assessment (LCA) databases, and emission registration database of the Dutch government, as well as communications with both manufacturing companies of production equipment and textile companies. The output of the calculations is presented in four single indicators: Ecocosts 2012 (a prevention-based indicator), CO2 equivalent (carbon footprint), cumulative energy demand (CED), and ReCiPe (a damage-based indicator). Results and discussion: From an analysis of the data, it becomes clear that the environmental burden is not only a function of the base materials (cotton, PET, nylon, acryl, and elastane) but also of the thickness of the yarn (for this research, the range of 50–500 dtex is examined). The authors propose that the environmental burden of spinning, weaving, and knitting is a function of 1/yarn size. The cradle-to-grave analysis from raw material extraction to discarded textile demonstrates that textiles made out of acryl and PET have the least impact on the environment, followed by elastane, nylon, and cotton. The use phase has less relative impact than it is suggested in the classical literature. Conclusions: The impact of spinning and weaving is relatively high (for yarn thicknesses of less than 100 dtex), and from the environmental point of view, knitting is better than weaving. LCA on textiles can only be accurate when the yarn thickness is specified. In case the functional unit also indicates the fabric per square meter, the density must be known. LCA results of textile products over the whole value chain are case dependent, especially when dyeing and finishing processes and the use phase and end-of-life are included in the analysis. Further LCI data studies on textiles and garments are urgently needed to lower the uncertainties in contemporary LCA of textile materials and products.