Reconstructing the intrinsic Ly$α$ line flux from high-$z$ QSOs can place constraints on the neutral hydrogen content of the intergalactic medium during reionisation. There are now $\gtrsim10$ different Ly$α$ reconstruction pipelines using different methodologies to predict the Ly$α$ line flux from correlations with the spectral information redward of Ly$α$. However, there have been few attempts to directly compare the performance of these pipelines. Therefore, we devised a blind QSO challenge to compare these reconstruction pipelines on a uniform set of objects. Each author was provided de-identified, observed rest-frame QSO spectra with spectral information only redward of 1260Å rest-frame to ensure unbiased reconstruction. We constructed two samples of 30 QSOs, from X-Shooter and SDSS both spanning $3.5