We present a detailed study on variational autoencoders (VAEs) for anomalous jet tagging at the Large Hadron Collider. By taking in low-level jet constituents’ information and training with background QCD jets in an unsupervised manner, the VAE is able to encode important information for reconstructing jets, while learning an expressive posterior distribution in the latent space. When using the VAE as an anomaly detector, we present different approaches to detect anomalies: directly comparing in the input space or, instead, working in the latent space. A comprehensive series of test sets are generated to fully examine the anomalous tagging performance in different jet types. In order to facilitate general search approaches such as bump hunt, mass-decorrelated VAEs based on distance correlation regularization are also studied. We find that the naive mass-decorrelated VAEs fail at maintaining proper detection performance, by assigning higher probability to some anomalous samples. To build a performant mass-decorrelated anomalous jet tagger, we propose the outlier exposed VAE (OE-VAE), for which some outlier samples are introduced in the training process to guide the learned information. OE-VAEs are employed to achieve two goals at the same time: increasing sensitivity of outlier detection and decorrelating jet mass from the anomaly score. We succeed in reaching excellent results from both aspects.