A large number of materials and various criteria fashion material selection problems as complex multi-criteria decision-making (MCDM) problems. This paper proposes a new decision-making method called the simple ranking process (SRP) to solve complex material selection problems. The accuracy of the criteria weights has a direct impact on the outcomes of the new method. In contrast to current MCDM methods, the normalization step has been eliminated from the SRP method as a potential source of producing incorrect results. The application of the method is appropriate for situations with high levels of complexity in material selection because it only considers the ranks of alternatives in each criterion. The first scenario of vital-immaterial mediocre method (VIMM) is used as a tool to derive criteria weights based on expert assessment. The result of SRP is compared with a number of MCDM methods. In order to evaluate the findings of analytical comparison, a novel statistical measure known as compromise decision index (CDI) is proposed in this paper. CDI revealed that the MCDM methods' outputs for solving the material selection could not be theoretically proven and requires to be evaluated through practice. As a result, the dependency analysis-an additional innovative statistical measure is introduced to demonstrate the reliability of MCDM methods by assessing its dependency on criteria weights. The findings demonstrated that SRP is extremely reliant on criteria weights and its reliability rises with the number of criteria, making it a perfect tool for solving challenging MCDM problems.