Abstract
Hand gesture recognition has emerged as an effective approach for enabling natural and inclusive human–computer interaction, particularly for individuals with speech and hearing impairments. This project presents a Hand Gesture Recognition and Voice Conversion System using deep learning and computer vision techniques. The system captures real-time hand gestures through a webcam and extracts hand landmark features using MediaPipe. These features are preprocessed and classified using a trained deep neural network to accurately recognize predefined gestures. The recognized gestures are converted into corresponding text and further transformed into audible speech using a text-to-speech module.
In addition to gesture-to-voice conversion, the system supports a reverse interaction mode where voice input is captured through a microphone, converted into text using speech recognition, and mapped to corresponding gesture images. The proposed system operates in real time, does not require specialized hardware, and provides bidirectional communication using commonly available devices. Experimental evaluation demonstrates reliable performance and practical usability. The system offers a cost-effective and scalable solution for assistive communication and human–computer interaction applications.
Keywords
Hand Gesture Recognition, Deep Learning, MediaPipe, Neural Network, Text-to-Speech, Speech Recognition, Computer Vision, Human–Computer Interaction.





1. INTRODUCTION
Communication is a vital aspect of human interaction, yet individuals with speech and hearing impairments face significant challenges in expressing themselves effectively. These individuals primarily rely on hand gestures and sign language, which are not widely understood by the general population, resulting in communication barriers and social isolation. With advancements in artificial intelligence and computer vision, gesture-based communication systems have gained attention as a promising solution to bridge this gap.
Hand gesture recognition enables machines to interpret human hand movements and convert them into meaningful information. Traditional approaches often rely on wearable sensors, which are expensive and inconvenient. Vision-based approaches using cameras and deep learning techniques provide a more practical and cost-effective alternative. By combining hand gesture recognition with speech technologies, it is possible to create an intelligent system that supports inclusive and natural communication.
This project focuses on the development of a dual-mode Hand Gesture Recognition and Voice Conversion System that converts gestures into voice and voice into gesture images using deep learning and computer vision techniques.

OBJECTIVES
The objectives of the proposed system are:

• To develop a real-time hand gesture recognition system using a webcam

• To extract and preprocess hand landmark features using MediaPipe

• To train a deep neural network for accurate gesture classification

• To convert recognized gestures into meaningful text and voice output

• To support reverse communication by converting voice input into gesture images

• To provide a cost-effective and user-friendly assistive communication system