Intelligent Data Curation Pipelines for Training Autonomous Vehicle Models

Abstract

The development of robust autonomous vehicle (AV) models heavily depends on the quality and diversity of training data. However, raw sensor data collected from AVs is often noisy, redundant, and unstructured, posing significant challenges for efficient model training. This paper proposes an intelligent data curation pipeline designed to automate and optimize the preprocessing, filtering, annotation, and augmentation of autonomous vehicle datasets. The pipeline leverages AI techniques, including active learning, anomaly detection, and generative data augmentation, integrated within a scalable cloud-based infrastructure. Our approach facilitates the selection of the most informative and diverse samples, reduces labeling effort, and enhances data quality for supervised learning tasks. Experimental evaluations using real-world autonomous driving datasets demonstrate substantial improvements in model accuracy, training efficiency, and generalization to diverse driving scenarios. The pipeline's modular architecture allows seamless integration with existing AV development workflows and supports continuous data ingestion from multiple sensor sources such as LiDAR, cameras, and radar. The results indicate that intelligent data curation is crucial for addressing the increasing data volume challenges in autonomous driving, accelerating the training process, and improving the safety and reliability of AV systems. This study highlights the importance of automated data curation as a foundational step in building next-generation autonomous driving models.