The literature about relationships between activity engagement and cognitive performance is abundant yet inconclusive. Some studies report that higher activity engagement leads to lower cognitive decline, others report no functional links, or that higher cognitive performance leads to less decline in activity engagement. We first discuss some methodological and analytical features that may contribute to the divergent findings. We then apply a longitudinal dynamic structural equation model to five repeated measurements of the Swiss Interdisciplinary Longitudinal Study on the Oldest Old. Performance on perceptual speed and verbal fluency tasks were analyzed in relation to six different activity composite scores. Results suggest that increased media and leisure activity engagement may lessen decline in perceptual speed, but not in verbal fluency, performance, while cognitive performance does not affect change in activity engagement.