A search is presented for direct pair production of the stop, the supersymmetric partner of the top quark, in a decay through an R-parity violating coupling to a charged lepton and a b-quark. The dataset corresponds to an integrated luminosity of 140 fb-1 of proton-proton collisions at a center-of-mass energy of s=13 TeV collected between 2015 and 2018 by the ATLAS detector at the LHC. The final state has two charged leptons (electrons or muons) and two b-jets. The results of the search are interpreted in the context of a Minimal Supersymmetric Standard Model with an additional B-L gauge symmetry that is spontaneously broken. No significant excess is observed over the Standard Model background, and exclusion limits on stop pair production are set at 95% confidence level. The corresponding lower limits on the stop mass for 100% branching ratios to a b-quark and an electron, muon, or tau-lepton are 1.9 TeV, 1.8 TeV and 800 GeV, respectively, extending the reach of previous LHC searches.