In recent years, automatic visual surveillance  has received considerable interest in the computer vision community. This is due to the increasing numbers of crimes from robbery to terrorist attacks, as well as the inability of human operators to monitor the increasingly growing numbers of surveillance cameras deployed in security sensitive areas such as government buildings and airports, or  public places such as shopping malls and streets. According to the British Security Industry Association, the number of CCTV cameras installed in the UK was estimated to be more than 4.25 million in 2004; this figure is expected to grow rapidly particularly after the terrorist attacks that London witnessed in July 2005.  Despite the huge increase of surveillance systems,  the question whether current surveillance systems work as a deterrent to crime is still debatable. Security systems should not only be able to predict when a crime is about to happen but, more importantly, they ought to identify the individuals suspected of committing crimes, say through the use of biometrics such as gait recognition.