Objectives: To evaluate the analysis and reporting of comparative effectiveness research with observational data in rheumatology, informing European Alliance of Associations for Rheumatology points to consider. Methods: We performed a systematic literature review searching Ovid MEDLINE for original articles comparing drug effectiveness in longitudinal observational studies, published in key rheumatology journals between 2008 and 2019. The extracted information focused on reporting and types of analyses. We evaluated if year of publication impacted results. Results: From 9969 abstracts reviewed, 211 articles fulfilled the inclusion criteria. Ten per cent of studies did not adjust for confounding factors. Some studies did not explain how they chose covariates for adjustment (9%), used bivariate screening (21%) and/or stepwise selection procedures (18%). Only 33% studies reported the number of patients lost to follow-up and 25% acknowledged attrition (drop-out or treatment cessation). To account for attrition, studies used non-responder imputation, followed by last observation carried forward (LOCF) and complete case (CC) analyses. Most studies did not report the number of missing data on covariates (83%), and when addressed, 49% used CC and 11% LOCF. Date of publication did not influence the results. Conclusion: Most studies did not acknowledge missing data and attrition, and a tenth did not adjust for any confounding factors. When attempting to account for them, several studies used methods which potentially increase bias (LOCF, CC analysis, bivariate screening…). This study shows that there is no improvement over the last decade, highlighting the need for recommendations for the assessment and reporting of comparative drug effectiveness in observational data in rheumatology.