ABSTRACT
The exponential growth of data across industries has intensified the need for automated and scalable machine learning solutions capable of minimizing human intervention while maintaining high predictive performance. Traditional machine learning workflows require extensive expertise in data preprocessing, feature engineering, model selection, and evaluation, which can be time-consuming and resource-intensive. To address these limitations, this project presents a comprehensive Automated Machine Learning (AutoML) training and deployment pipeline designed to support both classification and regression problems within a unified architecture. The system enables users to upload structured datasets and specify the target variable, after which the pipeline autonomously performs data validation, preprocessing, and model training. Missing values are handled using statistical imputation techniques, while categorical attributes are transformed into numerical representations using label encoding methods to ensure compatibility with machine learning algorithms. Feature scaling is applied through standardization to improve numerical stability and enhance model convergence during training. The pipeline intelligently detects the nature of the prediction task by analyzing the data type and distribution of the target variable, thereby dynamically selecting appropriate evaluation metrics and algorithms. For classification tasks, models such as Logistic Regression, Random Forest Classifier, and Support Vector Machine are employed, whereas Linear Regression and Random Forest Regressor are utilized for regression problems. Each model is trained and evaluated using a stratified train–test split strategy to ensure reliable and unbiased performance assessment. Performance metrics, including accuracy for classification and R² score for regression, are computed to determine the optimal model. The integration of MLflow facilitates systematic experiment tracking, logging model parameters, metrics, and artifacts to ensure reproducibility and transparency. The best-performing model is automatically selected and persisted along with preprocessing components such as scalers and encoders, enabling consistent future predictions. Furthermore, the system incorporates a web-based interface developed using FastAPI, allowing users to interactively train models and perform real-time predictions without manual coding. Containerization through Docker ensures platform independence and simplified deployment across diverse environments.
Introduction
The emergence of data-driven technologies has fundamentally transformed modern scientific research, industrial operations, and decision-making systems. Organizations across sectors such as healthcare, finance, agriculture, manufacturing, and e-commerce increasingly rely on predictive analytics to extract actionable insights from large volumes of structured and unstructured data. Machine Learning (ML), a core subfield of artificial intelligence, enables systems to learn patterns from historical data and make predictions or decisions without being explicitly programmed for each specific scenario. Despite its transformative potential, the development of machine learning solutions traditionally involves a complex, multi-stage workflow that requires substantial domain expertise, careful experimentation, and iterative refinement. These stages typically include data acquisition, data cleaning, feature engineering, model selection, hyperparameter tuning, performance evaluation, and deployment. Each step demands technical proficiency and significant time investment, thereby creating barriers for organizations or individuals lacking specialized knowledge. In conventional ML development, data preprocessing constitutes one of the most critical and time-consuming phases. Real-world datasets often contain missing values, inconsistent formats, categorical attributes, and noisy observations. Effective preprocessing techniques such as imputation, encoding, normalization, and scaling must be applied systematically to ensure data quality and model compatibility. Furthermore, selecting an appropriate learning algorithm depends heavily on the nature of the prediction task, whether classification or regression, and the statistical properties of the dataset. Improper selection can lead to suboptimal performance, overfitting, or poor generalization. Additionally, reproducibility and experiment tracking pose significant challenges when multiple models are trained and evaluated across different parameter settings. These limitations highlight the necessity for automation frameworks capable of simplifying and standardizing the machine learning lifecycle.
Automated Machine Learning (AutoML) has emerged as a promising solution to address these challenges by reducing manual intervention and optimizing model development processes. AutoML systems aim to automate essential ML tasks, including preprocessing, feature transformation, algorithm selection, and model evaluation. By leveraging algorithmic intelligence and predefined pipelines, AutoML platforms can systematically evaluate multiple candidate models and identify the best-performing configuration based on objective performance metrics. This approach not only accelerates development time but also enhances reproducibility and scalability. Moreover, AutoML democratizes machine learning by enabling non-expert users to build predictive models without requiring deep expertise in algorithm design or statistical theory. As data continues to grow in complexity and volume, scalable AutoML solutions become increasingly critical for efficient analytics. The present project proposes a comprehensive and production-ready AutoML Training and Deployment Pipeline designed to support both classification and regression tasks within a unified framework. Unlike many traditional systems that focus exclusively on either classification or regression, the proposed pipeline incorporates an intelligent problem-type detection mechanism that dynamically determines the nature of the prediction problem based on the characteristics of the target variable. This adaptive design enhances versatility and ensures that the system can be applied across diverse domains without structural modification. The architecture is modular, scalable, and capable of handling structured datasets in CSV format, making it practical for real-world implementation.
The pipeline begins with automated dataset ingestion and validation. Upon uploading a dataset, the system verifies its structural integrity, checks for sufficient sample size, and ensures that the specified target column exists. The preprocessing module subsequently performs missing value treatment using statistical imputation strategies—mean imputation for numerical features and mode imputation for categorical features. Categorical variables are transformed into numerical form using label encoding techniques to maintain algorithm compatibility. Feature scaling is applied through standardization to normalize feature distributions and improve the stability and convergence speed of gradient-based learning algorithms. These preprocessing steps are executed automatically, eliminating manual configuration and reducing the risk of human error. Following preprocessing, the system performs model training using a predefined set of robust machine learning algorithms. For classification problems, models such as Logistic Regression, Random Forest Classifier, and Support Vector Machine are employed to capture both linear and nonlinear relationships within the data.
 
Objectives
The primary objective of this project is to design and implement a comprehensive Automated Machine Learning (AutoML) training and deployment pipeline capable of minimizing human intervention while maintaining high predictive performance. The system aims to simplify the end-to-end machine learning workflow by integrating data preprocessing, model selection, evaluation, experiment tracking, and deployment within a unified and scalable framework. By automating critical stages of the machine learning lifecycle, the project seeks to enhance efficiency, reproducibility, and accessibility for both technical and non-technical users. One of the fundamental objectives is to develop a robust data preprocessing module that can intelligently handle real-world datasets. This includes automated detection and treatment of missing values using statistical imputation methods, transformation of categorical variables into numerical representations through encoding techniques, and feature scaling using standardization methods to ensure numerical stability. The objective is to create a preprocessing pipeline that operates dynamically without requiring manual configuration, thereby reducing errors and ensuring consistent data quality across diverse datasets.
Another key objective is to implement an intelligent problem-type detection mechanism that automatically distinguishes between classification and regression tasks based on the characteristics of the target variable. This capability ensures adaptability and allows the system to operate across multiple application domains without structural modification. By automating the identification of the problem type, the system eliminates the need for manual task specification and enhances usability for users with limited machine learning expertise. The project further aims to integrate multiple machine learning algorithms within a model selection framework to ensure optimal performance. For classification problems, algorithms such as Logistic Regression, Random Forest Classifier, and Support Vector Machine are incorporated to capture both linear and nonlinear relationships in data. For regression tasks, Linear Regression and Random Forest Regressor are included to model continuous outcomes effectively. The objective is to train, evaluate, and compare these models systematically using appropriate performance metrics, such as accuracy for classification and R² score for regression, and automatically select the best-performing model based on empirical evaluation.