Janus: Intelligent Surveillance System
With the Janus Project we envision developing more intelligent surveillance systems suitable for a variety of applications, such as security, surveillance, public health surveillance, traffic surveillance, etc., integrating large amounts of videos, texts, and sensor data available over an extended area of interest. Janus would provide law enforcement and first responders with the intelligence needed for optimized response. The challenge to realize this vision is data overload in detecting events in large environments and over long periods of time.
The insight we used to realize Janus comes from the realization that multi-modal data integration enables more effective surveillance systems.
Our approach utilizes state-of-the-art content analysis techniques to extract incidents from live data streams and integrate the incidents in the spatiotemporal domain (rather than content domain) to detect events. Traditional one-shot content-based analysis approaches are mature in extracting incidents from data, but have difficulty in connecting the dots in large spatiotemporal scale to comprehend activities.
With Janus we have built a 2 phases system that utilizes state of the art content analysis to extract incidents from data streams and spatiotemporal queries technology to cross-reference incidents and recognize activities.
Junus is built using a 3-tier architecture suitable for live streaming data processing:
- Data Tier extracts incidents and provides data management
- Analytics Tier extracts events and provides query capability
- Presentation Tier provides interactive data visualization with search and query
PTZ Face Detection
Spatial Video Query
Line of sight query
- 15 layers Point-of-interest (POI).
- Road Network: road network from NAVTEQ.
- PTZ camera videos used for face detection and tracking.
- Mobile camera videos.
- 1000+ geotagged real-time & remote access by-request security camera videos.
- 500,000+ geotagged, near real-time, license plate reader (LPR) records from 45 cameras (paired front & back) at gates.
- 2M/mo access control data, in near real-time, 1148 Lenel records including location, ID, time, alarm, device.
- Public daily DPS reports through DPS site. Unstructured. Extracted actors, objects, activity.
- ~10/day DPS internal reports. Categorized crime information. Geotagged. Structured. Near real-time access.
- Tweets from Tweeter public API.
- Pedestrians trajectories automatically extracted from video.
- Hi-resolution face images automatically extracted from PTZ cameras videos.
- Actors, objects, activity extracted from DPS reports and Tweets.