Proceedings Article | 8 October 2018
KEYWORDS: Sensors, Cameras, Roads, Tolerancing, Video, Surveillance, Analytical research, Human-machine interfaces, Improvised explosive devices, Defense and security, Surveillance systems, Video surveillance, Detection and tracking algorithms, Object recognition, Defense technologies
Surveillance systems are essential for the protection of military compounds and critical infrastructures. Automatic recognition of suspicious activities can augment human operators to find relevant threats. Common solutions for behavior analysis or action recognition require large amounts of training data. However, suspicious activities and threats are rare events, and the modus operandi of enemies may suddenly change, which makes it unrealistic to obtain sufficient training data in realistic situations. Therefore, we developed a demonstrator for the recognition of suspicious activity that allows users to easily define new alerts based on their expert knowledge. We developed basic modules for the computation of object detections, tracking and action recognition to generate features. The user is able to specify complex behavior with Symbols and Sentences. Symbols are low-level descriptions to analyze combinations of features. An example of a symbol is ‘approach_gate’, which consists of three conditions related to: the distance to the gate, the speed of a person and whether a person is on the compound. Sentences are high-level descriptions to analyze temporal ordering. An example of a sentence with temporal ordering is ‘entering the compound’, which consists of first approaching the gate, then entering the gate, and finally being on the compound. The demonstrator is compliant with the SAPIENT architecture, which uses multiple low-level Autonomous Sensor Modules (ASM) and a High-Level Decision Making Module (HLDMM). Features and symbols are computed in an ASM and sentences are computed in the HLDMM. Our demonstrator is tested on a multi-camera dataset to recognize suspicious behavior (e.g., digging for placement of improvised explosive devices, climbing over a fence, approaching the compound, and car standing on the road), allowing the user to interactively creates and modifies both symbols and sentences to mitigate new threats that were not implemented during design time.
