In this paper we show how a recurrent neural network, of shunting type, receiving changing input can be used for pattern classification or association. An important feature of the proposed network is its ability to continuously process environmental (external) input. Such ability is very useful if one is to design a real-time reactive system in an unexpectedly changing environment. We firstly state sufficient conditions under which the network dynamics is convergent to stable punctual attractors. Therefore, we show that relaxing the condition on symmetry of connectivity and making a local quadratic approximation of the dynamics can lead the network into an oscillatory mode. The corresponding limit cycles are shown to be stable. Application of the network with punctual attractor dynamics to navigation in a changing environment is pointed out but is not in the scope of this paper.