Over the last fifteen years, research on hair simulation has made great advances in the domains of modeling, animation and rendering, and is now moving towards more innovative interaction modalities. The combination of visual and haptic interaction within a virtual hairstyling simulation framework represents an important concept evolving in this direction. Our visuo-haptic hair interaction framework consists of two layers which handle the response to the user's interaction at a local level (around the contact area), and at a global level (on the full hairstyle). Two distinct simulation models compute individual and collective hair behavior. Our multilayered approach can be used to efficiently address the specific requirements of haptics and vision. Haptic interaction with both models has been tested with virtual hairstyling tools.