We discuss the effects of model misspecifications on higher-order asymptotic approximations of the distribution of estimators and test statistics. In particular we show that small deviations from the model can wipe out the nominal improvements of the accuracy obtained at the model by second-order approximations of the distribution of classical statistics. Although there is no guarantee that the first-order robustness properties of robust estimators and tests will carry over to second-order in a neighbourhood of the model, the behaviour of robust procedures in terms of second-order accuracy is generally more stable and reliable than that of their classical counterparts. Finally, we discuss some related work on robust adjustments of the profile likelihood and outline the role of computer algebra in this type of research.