This paper investigates the influence of empathizing and systemizing on cognitive and affective humor processing in two studies. Three cartoon types differing in their logical mechanisms (LMs) and cognitive requirements were presented to participants with high scores on one scale and low scores on the other (empathizers and systemizers): visual puns, semantic cartoons and Theory of Mind cartoons. Empathizers and systemizers were expected to process these cartoon types differently. While empathizers and systemizers did not differ in recognition time and comprehensibility in study one (N = 33), empathizers did portray higher funniness scores in study two (N = 55). Furthermore, empathizers more often gave emotional/motivational explanations as well as more mentalistic explanations as to why they think a cartoon is funny. In addition, Theory of Mind cartoons provoked the highest number of mentalistic explanations. This shows that stimulus characteristics (such as LMs) as well as inter-individual differences influence whether mentalizing is required and applied to processing humor, and that empathizing and systemizing influence humor appreciation.