We establish a general connection between entropic uncertainty relations, Einstein-Podolsky-Rosen steering, and joint measurability. Specifically, we construct steering inequalities from any entropic uncertainty relation, given that the latter satisfies two natural properties. We obtain steering inequalities based on Rényi entropies. These turn out to be tight in many scenarios, using max- and min-entropy. Considering steering tests with two noisy measurements, our inequalities exactly recover the noise threshold for steerability. This is the case for any pair of qubit 2-outcome measurements, as well as for pairs of mutually unbiased bases in any dimension. This shows that easy-to-evaluate quantities such as entropy can optimally witness steering, despite the fact that they are coarse-grained representations of the underlying statistics.