This paper proposes a Kolmogorov-type test for the shortfall order (also known in the literature as the right-spread or excess-wealth order) against parametric alternatives. In the case of the null hypothesis corresponding to the Negative Exponential distribution, this provides a test for the new better than used in expectation (NBUE) property. Such a test is particularly useful in reliability applications as well as duration and income distribution analysis. The theoretical properties of the testing procedure are established. Simulation studies reveal that the test proposed in this paper performs well, even with moderate sample sizes. Applications to real data, namely chief executive officer (CEO) compensation data and flight delay data, illustrate the empirical relevance of the techniques described in this paper.