In a visual search task, sensory input is matched to a representation of the search target in visual working memory (VWM). This representation is referred to as attentional template. We investigated the conditions that allow for more than a single attentional template. The attentional template of color targets was measured by means of the contingent attentional capture paradigm. We found that attentional templates did not differ between search with one and two memorized target colors, suggesting that dual target search allowed for multiple attentional templates. In the same paradigm, we asked participants to memorize target and distractor color with equal precision. Both were presented before the search task. An attentional template was set up for the target, but not for the distractor color, suggesting that keeping a color in VWM does not automatically result in the creation of multiple attentional templates. Importantly, the precision of recall of the distractor color was worse than the precision of recall of the target color, regardless of instructions, suggesting that participants always allocated fewer VWM resources to the distractor color. Thus, two attentional templates may be set up, but only when the two colors receive an equal amount of resources in VWM (i.e., in dual target search). In contrast, when one item is deprioritized because of task demands, it receives fewer resources in VWM and multiple attentional templates cannot be established. Thus, unequal roles in the search task prevented the simultaneous operation of multiple attentional templates in VWM.