Recent times have witnessed the transition towards a significantly larger scale both in the number of samples and the number of attributes characterising data collections. It is this latter aspect, the dimensionality of the data, that is at the center of the present thesis. We first analyse the evolution of the distance contrast and emphasise its dual character: absolute vs. relative. The second focus is on clustered structures, still in the context of high-dimensional data. Our purpose is to find low-dimensional embeddings with strong discriminative power. In this direction, we propose two methods, the High-Dimensional Multimodal Distribution Embedding - a distance-based embedding method that exploits distance distributions in high dimensions - and the Cluster Space - that projects points in the space of the clusters using the probabilities obtained from a Gaussian mixture model.