To understand SARS-CoV-2 immunity after natural infection or vaccination, functional assays such as virus neutralising assays are needed. So far, assays to detect SARS-CoV-2 neutralising antibodies rely on cell-culture based infection assays either using wild type SARS-CoV-2 or pseudotyped viruses. Such assays are labour-intensive, require appropriate biosafety facilities and are difficult to standardize. Recently, a new surrogate virus neutralisation test (sVNT) was described that uses the principle of an ELISA to measure the neutralisation capacity of anti-SARS-CoV-2 antibodies directed against the receptor binding domain. Here, we performed an independent evaluation of the robustness, specificity and sensitivity on an extensive panel of sera from 269 PCR-confirmed COVID-19 cases and 259 unmatched samples collected before 2020 and compared it to cell-based neutralisation assays. We found a high specificity of 99.2 (95%CI: 96.9-99.9) and overall sensitivity of 80.3 (95%CI: 74.9-84.8) for the sVNT. Clinical sensitivity increased between early (<14 days post symptom onset or post diagnosis, dpos/dpd) and late sera (>14 dpos/dpd) from 75.0 (64.7-83.2) to 83.1 (76.5-88.1). Also, higher severity was associated with an increase in clinical sensitivity. Upon comparison with cell-based neutralisation assays we determined an analytical sensitivity of 74.3 (56.4-86.9) and 98.2 (89.4-99.9) for titres =10 to <40 and =40 to <160, respectively. Only samples with a titre =160 were always positive in the sVNT. In conclusion, the sVNT can be used as an additional assay to determine the immune status of COVID-19 infected of vaccinated individuals but its value needs to be assessed for each specific context.