This thesis is devoted to the study of high-dimensional statistical models which include an infinite dimensional parameter. The manuscript contains two parts. In the first part, we propose a sieve M-estimation procedure, which combines the flexibility of semiparametric inference with the stability and reliability of local robustness. We derive the asymptotic theory of the proposed M-estimators. In the context of functional magnetic resonance imaging data analysis, we illustrate how to apply our procedure to conduct inference on a semiparametric dynamic factor model. Simulations and real data analysis exemplify the stability of our estimators, providing a comparison with the existing non robust and routinely applied sieve M-estimation. In the second part, we introduce a new parameter estimation method for locally stationary and high-dimensional time series models. To address high-dimensionality, we consider the novel concept of local sparsity, which is adapted to locally stationary models. The strength of our inference method relies on its ability to achieve both variables and domain selection for high-dimensional data, while dealing with the non-stationarity of the data. We show that the rate of convergence of the estimator matches the classical rate in standard sparse settings, and improves in locally sparse settings. We illustrate the applicability of our procedure for a locally stationary time series models: the time-varying autoregressive processes with exogenous covariates (tv-ARX). Extensive simulation studies confirm our theoretical results. We consider a real-data application to forecast monthly US inflation with many predictors.