Abstract
In recent years, the convergence of Artificial Intelligence (AI) and Internet of Things (IoT) technologies has enabled the creation of intelligent and secure automation systems. This project introduces a Flask-based Face Recognition and Serial Communication System that integrates computer vision, web development, and embedded hardware control. The system employs OpenCV and the face_recognition library to register users, capture facial features, and perform real-time recognition for authentication. Upon successful identification, the system establishes a serial connection with an Arduino microcontroller using PySerial, enabling automated actions such as granting access, displaying billing totals, or activating connected devices. The web interface developed in Flask allows users to register, log in, and access a built-in shopping module that maintains a session-based cart system. The recognized user’s cart details and total amount are transmitted to the Arduino for display or further control. This approach ensures contactless authentication, enhanced security, and seamless hardware integration, making the system suitable for smart canteens, automated vending systems, laboratory access control, and IoT-based retail applications. The proposed system successfully demonstrates the practical combination of machine learning, Flask-based web deployment, and microcontroller communication for real-time intelligent automation.
Keywords:
Face Recognition, Flask, OpenCV, PySerial, Arduino, Machine Learning, Internet of Things (IoT), Web Application, Serial Communication, Smart Automation.






Introduction
In the present era of digital transformation, automation and intelligent systems have become essential components of modern living. The increasing reliance on artificial intelligence (AI) and the Internet of Things (IoT) has revolutionized how humans interact with technology, bringing convenience, efficiency, and enhanced security to everyday processes. Among various emerging technologies, face recognition has gained remarkable attention due to its ability to provide secure, contactless, and highly accurate personal identification. Traditional authentication methods, such as passwords, access cards, or fingerprint scanners, often face issues related to loss, duplication, or misuse. In contrast, facial recognition uses unique physiological traits that are difficult to replicate, making it an ideal solution for identity verification and access control applications.
The concept of face recognition involves detecting a human face in an image or video stream and comparing its unique features with stored templates to verify identity. This technology has found wide applications in domains such as surveillance, banking, healthcare, education, and automated attendance systems. As computational power and machine learning algorithms continue to advance, facial recognition has become faster, more accurate, and easier to integrate into real-time systems. The combination of AI-based recognition models with IoT hardware has further expanded its potential, enabling seamless communication between intelligent software and physical devices. Such integration forms the foundation of this project, which aims to design a Flask-based Face Recognition and Serial Communication System capable of performing both secure authentication and hardware interaction through an Arduino microcontroller.
The main motivation behind developing this project is the increasing need for contactless and efficient verification systems, particularly in public spaces, offices, and institutional environments. The COVID-19 pandemic highlighted the importance of touch-free systems that minimize physical contact while maintaining high accuracy and reliability. Face recognition addresses these concerns effectively, offering a hygienic, user-friendly, and secure mode of authentication. Meanwhile, embedded systems like Arduino provide a cost-effective way to implement automation tasks such as controlling devices, displaying information, or managing sensors. By establishing serial communication between a Python-based web application and an Arduino microcontroller, this project bridges the gap between intelligent software decision-making and physical hardware actions.
The system developed in this project employs Flask, a lightweight and efficient Python web framework, to create an interactive web interface that allows users to register, train, and authenticate their faces. During registration, the system captures multiple images of the user’s face and stores them in a dedicated folder. These images are processed using the OpenCV and face_recognition libraries, which generate numerical feature encodings that uniquely represent each user’s face. Once the user’s facial data is trained and saved, they can later log in by uploading or capturing a new image. The system then compares the new image’s encodings with the stored data using similarity-based matching. If the system finds a match, the user is successfully authenticated, and further actions can be performed automatically.
What distinguishes this project from conventional recognition systems is the addition of real-time serial communication between the Flask application and the Arduino microcontroller. The communication is handled through the PySerial library, which enables bidirectional data transfer over the serial port. When a user is successfully recognized, the Flask application sends a signal or specific data (such as the total bill amount or access confirmation) to the Arduino board. The Arduino, upon receiving this data, performs a corresponding physical action — such as unlocking an access gate, activating an indicator, or displaying the billing total on an LCD screen. This direct hardware response demonstrates how web-based AI systems can interact with the physical world, achieving a higher level of automation and intelligence.
Another major feature of this project is its integrated shopping and billing interface, which enhances its real-world usability. Once authenticated, the user gains access to a virtual shop where they can browse food items, select their choices, and add them to a shopping cart. Each item includes a name and price, and the system dynamically updates the total cost based on user selections. When the user views the cart, the total bill amount is also transmitted to the Arduino via serial communication. This integration not only adds functional depth but also demonstrates how intelligent recognition systems can be expanded into commercial or service-oriented applications such as automated canteens, vending systems, or self-service kiosks.
The backend logic of the system is designed for reliability and modularity. The Flask application manages user sessions, maintains shopping cart data, and handles image uploads and encoding processes. Each user’s data is stored in a structured folder system under the static/faces directory, ensuring easy retrieval and training. The use of NumPy arrays to store encodings in .npy format ensures high-speed access during recognition. Meanwhile, the face_recognition library, built on top of dlib’s deep learning models, ensures accurate face matching even under variations in lighting and orientation. Together, these components create a robust and responsive authentication framework that can operate in real time.
From a technical standpoint, the Arduino communication module plays a critical role in bridging software intelligence with hardware control. Through the COM port interface, data can be transmitted to the Arduino at a defined baud rate, typically 9600 bps, ensuring reliable and synchronous communication. The system also incorporates mechanisms to check serial connectivity, handle exceptions, and reinitialize the serial connection in case of communication loss. This design ensures fault tolerance and stable operation even during extended runtime.
Overall, this project represents a practical and scalable implementation of AI-driven automation. It combines the strengths of machine learning, computer vision, web application development, and embedded system design into a single integrated framework. The system provides a complete cycle — from user registration and training to recognition and automated hardware response — reflecting real-world applications where security, intelligence, and automation coexist. The modular design of the system allows for easy customization and expansion; additional modules such as database integration, live video streaming, or mobile compatibility can be incorporated in the future.
In conclusion, the Flask-based Face Recognition and Serial Communication System demonstrates the powerful synergy between artificial intelligence and embedded hardware control. By leveraging Python’s rich ecosystem of AI and web frameworks along with the simplicity of Arduino hardware, the project achieves an efficient, contactless, and intelligent automation solution. Such systems can play a vital role in smart campuses, secure laboratories, automated canteens, and modern access control systems, providing both convenience and enhanced security. The success of this project highlights how software intelligence and hardware interaction can be seamlessly integrated to create a smarter, safer, and more connected world.


Objectives
The main objective of this project is to design and develop a Flask-based Face Recognition System integrated with Arduino through Serial Communication for secure authentication and automated control operations. The system focuses on bridging machine learning-based identity verification with real-time hardware interaction, ensuring efficiency, reliability, and user convenience.
The specific objectives of the project are as follows:
1. To develop a web-based face recognition application using Flask and OpenCV
o Create a lightweight, interactive Flask framework that allows users to register, capture facial images, and perform authentication seamlessly through a browser interface.
2. To implement accurate facial feature extraction and encoding
o Utilize the face_recognition library to detect facial landmarks, generate numerical encodings, and store them securely for subsequent identification and verification.
3. To establish reliable serial communication between the Flask application and Arduino
o Integrate the PySerial library to send and receive commands between the web system and the Arduino microcontroller, enabling real-time physical responses such as LED activation, display updates, or device control.
4. To design an intelligent user session and shopping interface
o Provide an authenticated user dashboard where recognized users can browse items, manage a shopping cart, and calculate total billing amounts dynamically.
5. To automate the billing display through Arduino
o After successful login and purchase confirmation, transmit the total bill amount to the Arduino for display or further action, ensuring contactless and automated transaction flow.
6. To ensure system scalability, accuracy, and low cost
o Build the project using open-source technologies and low-cost hardware components, making it adaptable for various smart automation applications such as canteens, labs, and secure access points.
7. To enhance user convenience and security through AI-driven verification
o Replace manual login and billing operations with facial authentication, minimizing identity fraud and improving overall system efficiency.

Block Diagram