Living beings continually adapt their behaviors based on knowledge they acquire over time. The brain ability to create abstract representations and generalize concepts inferred from specific sensory inputs can fasten learning, thus is crucial for survival. In particular, shared physical properties across senses can be generalized to facilitate sensorimotor associations, a phenomenon known as cross-modal transfer generalization. Previous research has shown that not only primates but many species, including rodents, are capable of performing such cross-modal generalizations. Mice, when exploring novel environments or interacting with conspecifics, rely both on their vision and whiskers for sensory feedback. Leveraging this concurrent use of visual and haptic senses, we used mice as a valuable model for investigating visuo-tactile circuits involved in cross-modal generalization. The circuit organization and the cortical neuronal mechanisms that mediate cross-modal generalizations during adaptive decision-making remains, however, elusive. In this study, we developed behavioral paradigms in both visual and tactile modalities and demonstrated that mice can bidirectionally transfer sensorimotor task rules between these senses, by extracting abstract features of the peri-personal space. Indeed, mice could only generalize the task rules when the spatial positions of the sensory stimulations were preserved across modalities, suggesting an amodal internal representation of space. Employing state-of-the-art multi-scale calcium imaging techniques, we showed that associative areas located between the primary somatosensory and visual cortex display topographic co-alignments, results further supported by anatomical tracing of feedforward and feedback projections between these areas and primary cortices. Single-cell calcium recordings revealed two areas located within the ventral and dorsal streams with populations of multimodal cells. Importantly, we discovered that multisensory neurons in these two domains responded preferentially to part of the peri-personal space independently of the modality presented, but also generally exhibited enhancement when visual and tactile cues were presented together in the same part of the peri-personal space. Optogenetic sensory substitution and systematic silencing of associative areas revealed that a single region in the dorsal stream, the rostrolateral area RL, was necessary and sufficient for cross-modal generalization of task rules. Thus, our findings highlight a critical cortical network essential for visuo-tactile integration and cross-modal generalization, emphasizing its pivotal role in directing goal-directed behavior. This research not only deepens our comprehension of multisensory integration and cross-modal learning but also sets the stage for future investigations into the mechanisms underlying cognitive generalization.