Epistemologists have developed a diverse group of theories, known as hinge epistemology, about our epistemic practices that resort to and expand on Wittgenstein's concept of ‘hinges’ in On Certainty. Within hinge epistemology there is a debate over the epistemic status of hinges. Some hold that hinges are non-epistemic (neither known, justified, nor warranted), while others contend that they are epistemic. Philosophers on both sides of the debate have often connected this discussion to Wittgenstein's later views on mathematics. Others have directly questioned whether there are mathematical hinges, and if so, these would be axioms. Here, we give a hinge epistemology account for mathematical practices based on their contextual dynamics. We argue that 1) there are indeed mathematical hinges (and they are not axioms necessarily), and 2) a given mathematical entity can be used contextually as an epistemic hinge, a non-epistemic hinge, or a non-hinge. We sustain our arguments exegetically and empirically.