Classical statistical inference relies mostly on parametric models and on optimal procedures which are mostly justified by their asymptotic properties when the data generating process corresponds to the assumed model. However, models are only ideal approximations to reality and deviations from the assumed model distribution are present on real data and can invalidate standard errors, confidence intervals, and p-values based on standard classical techniques. Moreover, the distributions needed to construct these quantities cannot typically be computed exactly and first-order asymptotic theory is used to approximate them. This can lead to a lack of accuracy, especially in the tails of the distribution, which are the regions of interest for inference. The interplay between these two issues is investigated and it is shown how to construct statistical procedures which are simultaneously robust and accurate