Abstract
The rapid advancement of intelligent systems has significantly increased the demand for accurate and reliable object detection and distance estimation techniques. Traditional vision-based object detection models, although efficient in identifying objects, often lack precise depth information and are highly sensitive to environmental conditions such as lighting variations, occlusions, and motion blur. Conversely, LiDAR-based systems provide accurate spatial and depth information but suffer from sparsity, noise, and high computational complexity. To overcome these limitations, this project proposes a robust multi-sensor fusion framework that integrates LiDAR data with vision-based object detection using the YOLOv8 model.The proposed system combines the strengths of both sensors by projecting LiDAR point cloud data onto the image plane to extract depth information corresponding to detected objects. A structured dataset is generated using features such as detection confidence, object depth, and noise characteristics derived from LiDAR data. This dataset is utilized to train a Random Forest classifier, which predicts the reliability of sensor measurements, enabling adaptive weighting during sensor fusion. This ensures that more reliable sensor data contributes more significantly to the final depth estimation.
To further enhance the stability and consistency of object tracking, an Extended Kalman Filter (EKF) is implemented. The EKF effectively reduces noise and sudden fluctuations in distance estimation by performing prediction and correction steps based on sequential observations. Additionally, the system incorporates an interactive graphical user interface developed using CustomTkinter, which allows users to perform real-time object detection, visualize depth through color-coded overlays, and manage access via a secure authentication system.The experimental results demonstrate that the proposed multi-sensor fusion approach significantly improves the accuracy, reliability, and stability of object detection and distance estimation compared to single-sensor systems.
Keywords
Multi-Sensor Fusion, LiDAR, YOLOv8, Object Detection, Distance Estimation, Random Forest, Extended Kalman Filter (EKF), Depth Estimation, Machine Learning, Computer Vision.


.Objectives
The primary objectives of the proposed system are as follows:
• To develop a multi-sensor object detection system by integrating LiDAR point cloud data with vision-based detection using YOLOv8 for improved perception accuracy.
• To design a framework for accurate distance estimation by associating LiDAR depth information with detected objects in image space.
• To generate a structured dataset using features such as detection confidence, object depth, and noise, enabling effective training of machine learning models.
• To implement a Random Forest-based machine learning model for predicting the reliability of sensor data and improving decision-making during sensor fusion.
• To perform adaptive sensor fusion by assigning dynamic weights to LiDAR and vision-based measurements based on their reliability scores.
• To incorporate an Extended Kalman Filter (EKF) for smoothing object tracking and reducing fluctuations in distance estimation over time.
• To design and implement an object tracking mechanism that maintains consistent identification of objects across multiple frames.
• To develop a user-friendly graphical interface using CustomTkinter for real-time visualization of detection and depth estimation results.
• To implement a secure user authentication system using a database for controlled access to the application.
• To provide real-time visualization with color-coded depth representation, enhancing interpretability of object distances.
• To evaluate the performance of the system in terms of accuracy, stability, and reliability under different environmental conditions.
• To create a scalable and robust system suitable for applications in autonomous vehicles, robotics, surveillance, and intelligent systems.