Abstract
With the increasing need for security in examination centers, workplaces, and restricted environments, automated surveillance systems have become essential to reduce human dependency and improve monitoring accuracy. This project presents an intelligent real-time suspicious activity detection system integrated with automated ID card detection using deep learning techniques. The system captures live video through a webcam and performs identity verification using a custom-trained YOLOv5 object detection model to detect ID cards in real time. Simultaneously, suspicious human activities are identified using a hybrid deep learning architecture combining MobileNetV2 and Long Short-Term Memory (LSTM) networks, where spatial features are extracted from video frames and temporal behavior patterns are analyzed across frame sequences.
To enhance system security, anti-spoofing mechanisms such as screen presence detection and video replay detection are incorporated to prevent fraudulent attempts using images or prerecorded videos. When suspicious activity is detected beyond a predefined confidence threshold, the system automatically generates an email alert to notify the concerned authority. The complete system is deployed as a Flask-based web application with secure authentication and live video streaming support. The proposed solution provides an efficient, automated, and real-time surveillance framework suitable for security-sensitive environments.
Keywords
Suspicious Activity Detection, ID Card Detection, YOLOv5, MobileNetV2, LSTM, Video Surveillance, Deep Learning, Anti-Spoofing, Real-Time Monitoring, Flask Web Application












Introduction
In recent years, the demand for intelligent surveillance systems has increased significantly due to growing security concerns in environments such as examination halls, offices, institutions, and restricted-access areas. Traditional surveillance systems rely heavily on continuous human monitoring, which is time-consuming, inefficient, and prone to errors caused by fatigue or delayed response. These limitations create a need for automated systems that can monitor activities and verify identity in real time with higher accuracy and reliability.
Suspicious activities such as impersonation, unauthorized movement, and abnormal behavior pose serious threats to security-sensitive environments. In many cases, identity verification and behavior monitoring are handled separately, leading to incomplete and unreliable security solutions. An integrated system capable of performing both tasks simultaneously is essential to ensure effective surveillance.
Advancements in deep learning and computer vision have enabled automated analysis of live video streams. Object detection models such as YOLOv5 allow real-time identification of objects, including ID cards, while deep learning-based activity recognition models can analyze human behavior over time. By combining spatial feature extraction and temporal sequence analysis, modern systems can accurately distinguish between normal and suspicious activities.
This project proposes an intelligent real-time surveillance system that integrates ID card detection with suspicious activity recognition using deep learning techniques. The system also incorporates anti-spoofing mechanisms to prevent fraudulent attempts using static images or prerecorded videos. A web-based implementation using the Flask framework enables secure access, live monitoring, and automated alert generation, making the system suitable for practical deployment in real-world surveillance applications.
Literature Survey
1. Deep Learning for Video Surveillance and Human Activity Recognition – Li Deng and Dong Yu (2014)
This study highlights the limitations of traditional rule-based surveillance systems and emphasizes the effectiveness of deep learning models in automatically learning features from video data. The authors explain that temporal modeling is essential for understanding human activities, as single-frame analysis cannot capture behavioral patterns. This work provides the foundation for using deep neural networks in intelligent surveillance systems.
2. You Only Look Once (YOLO): Unified Real-Time Object Detection – Joseph Redmon et al. (2016)
This paper introduces the YOLO object detection framework, which performs object localization and classification in a single forward pass. The model achieves high detection speed with good accuracy, making it suitable for real-time applications. YOLO’s real-time performance makes it highly effective for tasks such as ID card detection in live surveillance systems.
3. YOLOv5 for Custom Object Detection Applications – Glenn Jocher et al. (2020)
YOLOv5 is an improved version of the YOLO framework that offers better accuracy, faster training, and ease of deployment. The authors demonstrate that YOLOv5 can be trained on custom datasets for domain-specific object detection. This work supports the use of YOLOv5 for detecting ID cards under different lighting and orientation conditions.
4. MobileNetV2: Inverted Residuals and Linear Bottlenecks – Mark Sandler et al. (2018)
This paper presents MobileNetV2, a lightweight convolutional neural network designed for real-time and resource-constrained environments. The model reduces computational complexity while maintaining good feature extraction capability. Due to its efficiency, MobileNetV2 is well suited for real-time surveillance and forms the spatial feature extractor in activity recognition systems.
5. Long Short-Term Memory Networks – Hochreiter and Schmidhuber (1997)
This work introduces the LSTM architecture, which is capable of learning long-term dependencies in sequential data. LSTMs overcome the limitations of traditional recurrent neural networks by handling vanishing gradient problems. This model is widely used for analyzing temporal patterns in video sequences and is essential for suspicious activity recognition.
6. Real-Time Human Activity Recognition Using CNN and LSTM – Mohamed Hussein et al. (2019)
The authors propose a hybrid CNN–LSTM architecture for recognizing human activities from video streams. CNNs extract spatial features from frames, while LSTMs analyze motion patterns across time. The study demonstrates improved accuracy compared to single-frame approaches, validating the effectiveness of hybrid models for activity detection.

Objectives
• To design and develop an intelligent real-time surveillance system for monitoring security-sensitive environments.
• To implement automated ID card detection from live video streams using a deep learning–based YOLOv5 model.
• To detect and classify suspicious human activities using a hybrid MobileNetV2 and LSTM architecture.
• To analyze both spatial and temporal features of video data for accurate behavior recognition.
• To incorporate anti-spoofing mechanisms to prevent identity fraud using images or prerecorded videos.
• To generate real-time email alerts when suspicious activity is detected.
• To deploy the system as a secure web-based application using the Flask framework.
• To reduce human dependency in surveillance and improve response time and monitoring accuracy.