ABSTRACT:
This project proposes a comprehensive AI-powered Interview and Performance Evaluation System that integrates aptitude assessment, technical evaluation, communication analysis, and emotional understanding into a unified intelligent platform. Traditional evaluation processes are time-consuming, subjective, and dependent on human interpretation. To address these limitations, the proposed system leverages modern machine learning, speech processing, and natural language understanding to provide an automated, unbiased, and efficient assessment mechanism for candidates.
The system incorporates multiple advanced models:
• Whisper-based Speech-to-Text for high-accuracy transcription of spoken responses during interviews.
• HuBERT/Wav2Vec2 Emotion Classification for detecting emotional tone such as calm, happy, angry, or neutral from the recorded audio.
• T5 Grammar Correction Model for refining the candidate’s spoken language into grammatically correct English, enabling accurate communication skill evaluation.
• Decision Tree Classifier for analyzing aptitude test responses and predicting weak topics based on performance metrics.
All collected data—including transcripts, improved grammar versions, emotional indicators, aptitude scores, and technical test feedback—is automatically stored in an SQLite database. This allows the system to generate detailed performance reports and visualize user progress.
A user-friendly Flask web interface connects all modules, enabling users to participate in aptitude tests, record voice interviews, and receive instant AI-driven feedback. By integrating cutting-edge AI technologies with web-based interactivity, the system enhances the accuracy, reliability, and scalability of candidate assessment processes.
This project demonstrates the effectiveness of combining AI and deep learning to streamline evaluations in recruitment, training institutes, educational platforms, and career-development systems. Its modular design also allows further expansion into fully automated HR interviewing, behavioral analysis, and personalized learning recommendations.

INTRODUCTION:
In recent years, the increasing demand for efficient, unbiased, and scalable evaluation systems has driven the adoption of Artificial Intelligence (AI) across recruitment, education, and skill development sectors. Traditional assessment methods—such as written aptitude tests, manual technical interviews, and face-to-face communication evaluations—are often time-consuming, inconsistent, and influenced by human subjectivity. As organizations seek more reliable ways to evaluate candidates, AI-driven assessment systems have emerged as a powerful solution for delivering accurate, real-time, and data-driven insights.
This project introduces an AI-Driven Automated Interview and Performance Evaluation System, designed to assess multiple aspects of a candidate’s abilities through machine learning, natural language processing (NLP), and speech analysis. The system aims to replicate key components of a real interview environment by analyzing a user’s aptitude performance, technical understanding, communication clarity, and emotional tone. By integrating several AI models into a unified framework, the system enhances both the efficiency and accuracy of candidate evaluation.
A core part of the system uses the Whisper speech-to-text model to convert spoken responses into text with high accuracy, enabling automated analysis of communication skills. To evaluate emotional expression, a HuBERT/Wav2Vec2-based emotion classification model detects underlying emotional states such as calmness, happiness, or stress from the audio recording. Furthermore, a T5-based grammar correction model improves the candidate’s spoken responses into polished English, offering insights into language proficiency and clarity of expression.
In addition to communication analysis, the system incorporates an aptitude prediction component using a Decision Tree Classifier. This module analyzes user performance across specific aptitude topics and predicts weak areas, helping candidates understand where improvement is needed. The integration of SQLite as the backend database ensures efficient storage, retrieval, and management of performance data, enabling detailed report generation and user progress tracking.
The entire system is deployed through a Flask-based web application, providing an intuitive interface that allows users to take tests, submit voice responses, and instantly receive AI-generated feedback. This seamless interaction between web technologies and AI models highlights the project’s practical applicability in real-world environments.
Overall, the project demonstrates how AI can transform traditional assessment methods into intelligent, automated, and user-centric evaluation platforms. By combining speech recognition, emotion analysis, grammar enhancement, and aptitude prediction, the system serves as a robust tool for recruitment, training institutes, academic assessments, and self-evaluation. Its modularity and scalability make it a strong foundation for future expansion into more advanced AI-driven interviewing and personalized learning systems.