Background: Rapid Advice Guidelines (RAG) provide decision makers with guidance to respond to public health emergencies by developing evidence-based recommendations in a short period of time with a scientific and standardized approach. However, the experience from the development process of a RAG has so far not been systematically summarized. Therefore, our working group will take the experience of the development of the RAG for children with COVID-19 as an example to systematically explore the methodology, advantages, and challenges in the development of the RAG. We shall propose suggestions and reflections for future research, in order to provide a more detailed reference for future development of RAGs.

Result: The development of the RAG by a group of 67 researchers from 11 countries took 50 days from the official commencement of the work (January 28, 2020) to submission (March 17, 2020). A total of 21 meetings were held with a total duration of 48 h (average 2.3 h per meeting) and an average of 16.5 participants attending. Only two of the ten recommendations were fully supported by direct evidence for COVID-19, three recommendations were supported by indirect evidence only, and the proportion of COVID-19 studies among the body of evidence in the remaining five recommendations ranged between 10 and 83%. Six of the ten recommendations used COVID-19 preprints as evidence support, and up to 50% of the studies with direct evidence on COVID-19 were preprints.

Conclusions: In order to respond to public health emergencies, the development of RAG also requires a clear and transparent formulation process, usually using a large amount of indirect and non-peer-reviewed evidence to support the formation of recommendations. Strict following of the WHO RAG handbook does not only enhance the transparency and clarity of the guideline, but also can speed up the guideline development process, thereby saving time and labor costs.