In this paper, we present an anomaly detection and localization system for surveillance systems. A new feature descriptor is proposed. The spatio-temporal identifiers are obtained by using optical flow histogram and the structural similarity index from the videos that contain normal conditions. An artificial neural network, Selforganizing maps are used in modeling. The proposed system has been tested on the UCSD dataset.