Nowadays, the increase in data size and model complexity has led to increasingly difficult estimation problems. The numerical aspects of the estimation procedure can indeed be very challenging. To solve these estimation problems, approximate methods such as pseudo-likelihood functions or approximated estimating equations can be used as these methods are typically easier to implement numerically although they can lead to inconsistent and/or biased estimators. In this thesis, we propose a unified framework to compare four existing bias reduction estimators, two of them are based on indirect inference and two are based of bootstrap. We derive the asymptotic and finite sample properties of these bias correction methods. We demonstrate the equivalence between one version of the indirect inference and the iterative bootstrap which both correct sample biases up to the order $n^{-3}$. Therefore, our results provide different tools to correct the asymptotic as well as finite sample biases of estimators and give insight as to which method should be applied according to the problem at hand. We then apply these bias reduction techniques to robust estimation of income distributions. We used a very simple starting estimator which is known to be robust but not consistent and correct its bias with indirect inference. This is a very general way to construct robust estimators for complex models. A second illustration is provided by the estimation of Generalized Linear Latent Variable Models. We were able to compute unbiased estimates for these very complex models that have a large number of parameters without employing numerical integration techniques. As a by-product, bias reduction techniques allow to compute a goodness-of-fit test statistic for latent variable models.