Drug-protein interactions have become a crucial component to study potential side effects, discover new uses for existing drugs, to name a few applications. We describe our approach based on transformer-based language models to predict relations between chemical and gene entities in DrugProt corpus. Sliding window is used to detect the relation in a passage for the individual models, and then they are combined using majority vote. Our model achieved 60% of F1-score (88% of recall and 45% of precision) in the track 1: text mining drug and chemical- protein interactions at BioCreative VII. Ensemble of transformer-based language models provides a baseline performance for drug-protein interaction extraction