The successor representation (SR) allows for efficient and flexible value computation by representing states with their expected temporal evolution. Here, we present two extensions of the SR theory to belief state and to distribution overvalue. When states are partially observed, an optimal agent should rely on beliefstates indicating the probability of being in different states. Here, we first present an analytical expression for the SR of for fixed policies and for decision making problems with only one decision. We show that this expression also provide a good approximation to SR(b) in problems with multiple decisions such as noisy 2Dnavigation tasks. We then propose a neural network that approaches the optimal SR(b) in tasks with multiple decisions. Next, we extent the SR to distribution overvalue. In the process, we propose a new local code for distributional reinforcement learning which allows agents to recover the value distribution of a state given its SR, as well as the expected temporal evolution of the value distribution. Finally, we combine these advances into a SR model that jointly accounts for uncertainty over states and value, implemented in a biologically plausible neural network.