In an influential paper, Donk and van Zoest (2008; https://doi.org/d3cn5x) concluded that information on saliency dissipates within a few hundred milliseconds after onset of a stimulus array. Recently, we have shown that saliency has massive effects on the delayed recall from visual working memory (VWM) more than 1,300 ms after stimulus onset (Constant & Liesefeld, 2021; https://doi.org/gjk9jh). Thus, depending on the task design, effects of saliency or their repercussions are obviously not so short-lived after all. To understand the time course of these effects in more detail, we varied presentation duration of the VWM memory display from 14 ms to 2,000 ms and found that effects of saliency indeed decreased with time, but were still markedly present even with the longest encoding time (3,000 ms after stimulus onset). In an attempt to overrule this persistent influence of saliency we made the least salient stimulus most relevant (by probing it three times more often than the most salient stimulus). Now participants indeed managed to level the effect of saliency with 2,000 ms (but not 350 ms) presentation time and task relevance slightly gained an edge over saliency with 3,000 ms presentation time. In a final experiment, we found indication that the long-lasting effect of saliency is due to absolute saliency rather than relative saliency (i.e., with respect to other salient objects in the display). These data provide interesting constraints for mathematical models of VWM encoding and cognitive modeling will allow a closer look on how various mechanisms involved in VWM-recall tasks are affected by saliency. In contrast to what has been assumed so far, our results demonstrate that stimulus saliency has a much longer lasting (and therefore also more pervasive) effect on cognitive performance that reaches even relatively late processing stages and is difficult to overrule by volition.