Computer Vision Based Robust Lane Detection Via Multiple Model Adaptive Estimation Technique

Abstract

The lane-keeping system in autonomous vehicles (AV) or even as a part of the advanced driving assistant system (ADAS) is known as one of the primary options of AVs and ADAS. The developed lane-keeping systems work on either computer vision or deep learning algorithms for their lane detection section. However, even the strongest image processing units or the robust deep learning algorithms for lane detection have inaccuracies during lane detection under certain conditions. The source of these inaccuracies could be rainy or foggy weather, high contrast shades of buildings and objects on-street, or faded lines. Since the lane detection unit of these systems is responsible for controlling the steering, even a momentary loss of lane detection accuracy could result in an accident or failure. As mentioned, different lane detection algorithms have been presented based on computer vision and deep learning during the last few years, and each one has pros and cons. Each model may have a better performance in some situations and fail in others. For example, deep learning-based methods are vulnerable to new samples. In this research, multiple models of lane detection are evaluated and used together to implement a robust lane detection algorithm. The purpose of this research is to develop an estimator-based Multiple Model Adaptive Estimation (MMAE) algorithm on the lane-keeping system to improve the robustness of the lane detection system. To verify the performance of the implemented algorithm, the AirSim simulation environment was used. The test simulation vehicle was equipped with one front camera and one back camera used to implement the proposed algorithm. The front camera images are used for detecting the lane and the offset of the vehicle and center point of the lane. The rear camera, which offered better performance in lane detection, was used as an estimator for calculating the uncertainty of each model. The simulation results showed that combining two implemented models with MMAE performed robustly even in those case studies where one of the models failed. The proposed algorithm was able to detect the failures of either of the models and then switch to another good working model to improve the robustness of the lane detection system. However, the proposed algorithm had some limitations; it can be improved by replacing PID controller with an MPC controller in future studies. In addition, in the presented algorithm, two computer vision-based algorithms were used; however, adding a deep learning-based model could improve the performance of the proposed MMAE. To have a robust deep learning-based model, it is suggested to train the network based on AirSim output images. Otherwise, the network will not work accurately due to the differences in the camera's location, camera configuration, colors, and contrast.