Dephasing in a ballistic four-terminal Aharonov-Bohm geometry due to charge and voltage fluctuations is investigated. Treating two terminals as voltage probes, we find a strong dependence of the dephasing rate on the probe configuration in agreement with a recent experiment by Kobayashi et al. (J. Phys. Soc. Jpn. 71, 2094 (2002)). Voltage fluctuations in the measurement circuit are shown to be the source of the configuration dependence.