Delineation of organs at risk (OARs), such as the bladder, rectum and sigmoid, plays an important role in the delivery of optimal absorbed dose to the target owing to the steep gradient in high-dose rate brachytherapy (HDR-BT). In this work, we propose a deep convolutional neural network-based approach for fast and reproducible auto-contouring of OARs in HDR-BT.