The cullin 4-DNA-damage-binding protein 1 (CUL4-DDB1) ubiquitin ligase machinery regulates diverse cellular functions and can be subverted by pathogenic viruses. Here we report the crystal structure of DDB1 in complex with a central fragment of hepatitis B virus X protein (HBx), whose DDB1-binding activity is important for viral infection. The structure reveals that HBx binds DDB1 through an alpha-helical motif, which is also found in the unrelated paramyxovirus SV5-V protein despite their sequence divergence. Our structure-based functional analysis suggests that, like SV5-V, HBx captures DDB1 to redirect the ubiquitin ligase activity of the CUL4-DDB1 E3 ligase. We also identify the alpha-helical motif shared by these viral proteins in the cellular substrate-recruiting subunits of the E3 complex, the DDB1-CUL4-associated factors (DCAFs) that are functionally mimicked by the viral hijackers. Together, our studies reveal a common yet promiscuous structural element that is important for the assembly of cellular and virally hijacked CUL4-DDB1 E3 complexes.