A search is presented for the pair production of heavy vectorlike quarks, TT¯ or BB¯, that decay into final states with jets and no reconstructed leptons. Jets in the final state are classified using a deep neural network as arising from hadronically decaying W/Z bosons, Higgs bosons, top quarks, or background. The analysis uses data from the ATLAS experiment corresponding to 36.1 fb−1 of proton-proton collisions with a center-of-mass energy of s=13 TeV delivered by the Large Hadron Collider in 2015 and 2016. No significant deviation from the Standard Model expectation is observed. Results are interpreted assuming the vectorlike quarks decay into a Standard Model boson and a third-generation-quark, T→Wb,Ht,Zt or B→Wt,Hb,Zb, for a variety of branching ratios. At 95% confidence level, the observed (expected) lower limit on the vectorlike B-quark mass for a weak-isospin doublet (B, Y) is 950 (890) GeV, and the lower limits on the masses for the pure decays B→Hb and T→Ht, where these results are strongest, are 1010 (970) GeV and 1010 (1010) GeV, respectively.