We report constraints on nonstandard neutrino interactions (NSI) from the observation of atmospheric neutrinos with IceCube, limiting all individual coupling strengths from a single dataset. Furthermore, IceCube is the first experiment to constrain flavor-violating and nonuniversal couplings simultaneously. Hypothetical NSI are generically expected to arise due to the exchange of a new heavy mediator particle. Neutrinos propagating in matter scatter off fermions in the forward direction with negligible momentum transfer. Hence the study of the matter effect on neutrinos propagating in the Earth is sensitive to NSI independently of the energy scale of new physics. We present constraints on NSI obtained with an all-flavor event sample of atmospheric neutrinos based on three years of IceCube DeepCore data. The analysis uses neutrinos arriving from all directions, with reconstructed energies between 5.6 GeV and 100 GeV. We report constraints on the individual NSI coupling strengths considered singly, allowing for complex phases in the case of flavor-violating couplings. This demonstrates that IceCube is sensitive to the full NSI flavor structure at a level competitive with limits from the global analysis of all other experiments. In addition, we investigate a generalized matter potential, whose overall scale and flavor structure are also constrained.