We review some first-and higher-order asymptotic techniques for M-estimators and we study their stability in the presence of data contaminations. We show that the estimating function (ψ) and its derivative with respect to the parameter (∇ θ ⊤ ψ) play a central role. We discuss in detail the first-order Gaussian density approximation, saddlepoint density approximation, saddlepoint test, tail area approximation via Lugannani-Rice formula, and empirical saddlepoint density approximation (a technique related to the empirical likelihood method). For all these asymptotics, we show that a bounded (in the Euclidean norm) ψ and a bounded (e.g., in the Frobenius norm) ∇ θ ⊤ ψ yield stable inference in the presence of data contamination. We motivate and illustrate our findings by theoretical and numerical examples about the benchmark case of one-dimensional location model.