This dissertation explores advancements in cross-lingual word embeddings to enhance model transfer between high-resource and low-resource languages, with a focus on typologically distant pairs. First, it introduces a weakly-supervised adversarial training method that aligns words at the concept level, improving cross-lingual transfer performance. Next, it challenges the common assumption of single linear mappings across languages and proposes a multi-linear mapping approach, which better captures linguistic relationships and improves transferability across distant languages. Finally, the research extends to dynamic contextualized embeddings, proposing a cross-lingual adversarial fine-tuning method that aligns token representations in similar sentences across languages.