Abstract
Traditional video surveillance systems for anomaly detection often face limitations such as high latency, frequent false alarms, and poor adaptability to diverse environments. These issues make real-time detection of unusual or criminal activities like theft, vandalism, or violence highly unreliable. Existing models also rely heavily on visual data, neglecting multimodal cues such as audio, and struggle to capture long-term temporal dependencies.
This project proposes an AI-powered real-time anomaly detection system that integrates Video Swin Transformer and Temporal Convolutional Networks (TCNs) to effectively analyze spatial and temporal features from surveillance videos. Motion dynamics are incorporated through optical flow analysis, while multimodal support enhances detection accuracy. The system is optimized for real-time deployment on both cloud and edge devices such as NVIDIA Jetson Nano, ensuring scalability in resource-limited environments. A user-friendly dashboard provides live monitoring, instant alerts, and anomaly logs for further analysis.With applications in crime prevention, traffic management, and public safety surveillance, the proposed system aims to enhance security by delivering faster, more reliable, and intelligent anomaly detection in diverse real-world scenarios.
Keywords
Anomaly Detection, Crime Activity Recognition, Video Swin Transformer, Temporal Convolutional Networks (TCNs), Optical Flow, Deep Learning, Real-Time Surveillance, Edge Deployment, UCF-Crime Dataset, Public Safety.


Objectives
The project aims to build an AI-powered anomaly detection system for real-time crime activity recognition. To achieve this, the objectives are broken down into the following detailed points:
1. Develop a real-time anomaly detection framework
o Design a surveillance system that can analyze live video streams continuously.
o Ensure that the system detects unusual or suspicious activities (e.g., theft, violence, vandalism) without manual intervention.
2. Implement advanced deep learning architectures
o Use the Video Swin Transformer to extract spatial features from video frames by applying attention mechanisms and hierarchical embeddings.
o Use Temporal Convolutional Networks (TCNs) to model temporal dependencies across frames, capturing long-term behavior patterns.
3. Incorporate motion-based features through Optical Flow
o Apply optical flow algorithms (e.g., Farneback, Horn-Schunck) to track motion between frames.
o Use these features to detect subtle anomalies like sudden running, loitering, or object abandonment.
4. Integrate multimodal inputs (Video + Audio)
o Extend the system to process audio cues (e.g., screams, glass breaking, alarms).
o Combine audio with video features for better decision-making and reduced false alarms.
5. Dataset training and validation
o Train the system using the UCF-Crime dataset, which includes labeled real-world surveillance videos of normal and abnormal activities.
o Perform data preprocessing (frame extraction, resizing, normalization, augmentation) to improve generalization.
6. Optimize real-time performance
o Ensure inference runs at ≥25 frames per second (fps) for live video feeds.
o Use model compression techniques (quantization, pruning) to achieve low-latency detection.
7. Minimize false positives and false negatives
o Improve classification accuracy with robust loss functions (Cross-Entropy + Dice Loss).
o Apply attention mechanisms to focus on critical regions and time intervals in the video.
8. Deploy the system on edge devices
o Optimize models to run on low-power edge devices like NVIDIA Jetson Nano/Xavier for cost-effective deployment.
o Ensure scalability to handle multiple surveillance cameras simultaneously.
9. Develop a user-friendly dashboard
o Create an interface displaying live camera feeds, detected anomalies, and logs with timestamps.
o Provide real-time alert notifications to security personnel for immediate action.
10. Evaluate performance and compare with baseline models
• Use standard metrics like Precision, Recall, F1-Score, AUC, IoU, and inference speed.
• Benchmark against existing models (CNN + FPN, I3D, Autoencoders) to demonstrate improvements.