Robust Vehicle Segmentation

	A robust segmentation process for detecting incidents on highways is presented. This segmentation process is based on background subtraction and uses an efficient background model initialisation and update to work 24/7. A cross-correlation based shadow detection is also used for minimising ghosts.  [Demos] [Publications]

Background/foreground detection is an essential component of most video surveillance systems involving object detection and it is the core of most incident detection systems. Such systems require both robustness to lighting variation and computer feasibility. Most of the existing solutions either make strict assumptions about the scene or simply fail when handling abrupt lighting variations resulting from moving clouds or camera automated gain control. Therefore, we are presenting here a robust segmentation process for outdoor scenarios.

The system is based on background subtraction, and uses three background models for contemplating the background variations. Two distinct thresholds are used: a per-pixel threshold for a robust adaptation to the scene variations, twenty four hours a day, and a global threshold, the minimum value, to eliminate camera noise scenario features, like shaking trees and camera pole vibrations. This threshold is estimated in the system initialisation. A shadow/highlight detection algorithm based on cross-correlation is also used to discard blobs generated by lighting variation (moving clouds, artificial light changes, etc.).

Background Modelling

In order to achieve a good segmentation in an outdoor environment it is necessary to have a background model that considers all scenes' variations not classified as foreground, otherwise the false positive segmentation rate will increase. Nonetheless, the background model cannot be too comprehensive because there is an high risk of mis-detection.

Three different background images are used to model the background of the scene, namely the primary background, the secondary background and the median background. The primary background has to have, at all times, the background model that is closest to the current frame. This model shall take into consideration fast variations in the scene, like lighting changes or camera AGC adjustment. The secondary background is used to model objects classified as static but that can undergo small variations in position and/or in shape, vibration of the camera's pole, camera signal noise and digital video compression. Lastly, the median background is the pixel value median of the last n frames. This background is the cleanest one of the three backgrounds due to its median properties. The main task of the median background is to avoid corruption and deterioration of the primary background. View a video of the background modelling in a scenario with high traffic density with just 50 frames.

Shadow/Highlight Removal

One of the biggest problems to overcome at pixel level is the existence of brightness variations in parts of the image due to shadows and lighting changes. The shadows induced by vehicles themselves tend to merge the detected blobs of nearby vehicles and significantly increase the blob area. The shadows induced by the clouds can generate false positives in the segmentation process because of the fast lighting variation of the scene when the clouds are moving fast. The glares induced on the surface of the highway by vehicles' night light system and artificial lighting variation are also a problem for a robust outdoor segmentation process.

The adopted solution measures the similarity between the primary background and present frame, computing the colour normalised cross-correlation. The colour of the pixel is represented in the biconic HSL space in order to split the colour information from the brightness values. See the demo video.

The system is currently being used in several surveillance cameras of Brisa with good results in different wheather conditions and illumination changes. See the demos: video1, video2 and video3.