Selected Publications

You can also find my full article list on my Google Scholar profile.

Enhancing Autonomous Vehicles’ Situational Awareness with Dynamic Maps: Cooperative Prediction on Edge, Cloud and Vehicle

Published in IEEE Transactions on Intelligent Vehicles, 2024

Abstract: Autonomous vehicles (AVs) are widely recognized as a pivotal solution for fostering efficiency and safety of future society. Meanwhile, the limitations of on-board sensors, such as physical occlusions and limited field of view (FoV), have sparked significant interest in cooperative perception (CP). However, CP introduces delays in perception and communication tasks, which may hinder AVs’ situational awareness. To propel the evolution of CP and further enhance AVs’ situational awareness, this paper introduces a novel concept: cooperative prediction (CPd). CPd extends the situational awareness of CP along the temporal axis, overcoming the line-of-sight, FoV, and range-of-time limitations. Leveraging dynamic maps, a CPd framework that seamlessly operates across edge, cloud, and vehicle is established. Edge-based, cloud-based, and vehicle-based CPd modes are proposed to validate the CPd concept. Extensive real-world experiments are carried out to evaluate the performance of our CPd system, comparing it with a conventional CP system. The experimental results show that the CPd system surpasses the CP system by providing accurate zero-lag perception data and reliable prediction data, thereby enhancing AVs’ situational awareness in both spatial and temporal domains. Besides, our findings reveal that the cloud-based CPd operates with the highest efficiency, whereas the vehicle-based CPd may encounter fatal failures. A demonstration of our CPd system can be viewed at https://youtu.be/WURn-EkjODM/ .

Recommended citation: Lu Tao, Yousuke Watanabe, Ryosuke Takeuchi, Shinichi Kusayama, Shunya Yamada, and Hiroaki Takada, "Enhancing Autonomous Vehicles Situational Awareness with Dynamic Maps: Cooperative Prediction on Edge, Cloud and Vehicle," in IEEE Transactions on Intelligent Vehicles, doi: 10.1109/TIV.2024.3462744
Download Paper

Global Navigation Satellite System/Inertial Measurement Unit/Camera/HD Map Integrated Localization for Autonomous Vehicles in Challenging Urban Tunnel Scenarios

Published in Remote Sensing, 2024

Abstract: Lane-level localization is critical for autonomous vehicles (AVs). However, complex urban scenarios, particularly tunnels, pose significant challenges to AVs’ localization systems. In this paper, we propose a fusion localization method that integrates multiple mass-production sensors, including Global Navigation Satellite Systems (GNSSs), Inertial Measurement Units (IMUs), cameras, and high-definition (HD) maps. Firstly, we use a novel electronic horizon module to assess GNSS integrity and concurrently load the HD map data surrounding the AVs. This map data are then transformed into a visual space to match the corresponding lane lines captured by the on-board camera using an improved BiSeNet. Consequently, the matched HD map data are used to correct our localization algorithm, which is driven by an extended Kalman filter that integrates multiple sources of information, encompassing a GNSS, IMU, speedometer, camera, and HD maps. Our system is designed with redundancy to handle challenging city tunnel scenarios. To evaluate the proposed system, real-world experiments were conducted on a 36-kilometer city route that includes nine consecutive tunnels, totaling near 13 km and accounting for 35% of the entire route. The experimental results reveal that 99% of lateral localization errors are less than 0.29 m, and 90% of longitudinal localization errors are less than 3.25 m, ensuring reliable lane-level localization for AVs in challenging urban tunnel scenarios.

Recommended citation: Lu Tao, Pan Zhang, Kefu Gao, and Jingnan Liu. "Global Navigation Satellite System/Inertial Measurement Unit/Camera/HD Map Integrated Localization for Autonomous Vehicles in Challenging Urban Tunnel Scenarios." Remote Sensing 16, no. 12 (2024): 2230.
Download Paper

Geo-spatial and Temporal Relation Driven Vehicle Motion Prediction: A Geographic Perspective

Published in IEEE Transactions on Intelligent Vehicles, 2024

Abstract: The constraints imposed by traffic regulations and road geometries lead to a certain degree of geo-spatial similarity in vehicular behaviors, a factor overlooked in existing studies on vehicle motion prediction. To leverage the similarity, this paper introduces three novel vehicle motion prediction algorithms driven by geo-spatial and temporal relations within a classical Kalman filter (KF) framework, integrating Tobler’s first law of geography and Newton’s second law of motion. These algorithms utilize three specialized geo-spatial and temporal search mechanisms to extract geo-spatial dependencies from a spatial trajectory database and concurrently propel the KF prediction processes. Particularly, spatial interpolation and Dempster-Shafer reasoning modules are introduced to boost the algorithms’ performance. We validate these methods using real driving data in public environments, demonstrating their state-of-the-art performance; and further examine their performance evolution in space domain and investigate their robustness in missing-measurement situations, which are unexplored in existing literature.

Recommended citation: Lu Tao, Yousuke Watanabe, and Hiroaki Takada. "Geo-spatial and temporal relation driven vehicle motion prediction: A geographic perspective." IEEE Transactions on Intelligent Vehicles (2024).
Download Paper

A Lightweight Long-Term Vehicular Motion Prediction Method Leveraging Spatial Database and Kinematic Trajectory Data

Published in ISPRS International Journal of Geo-Information, 2022

Abstract: Long-term vehicular motion prediction is a crucial function for both autonomous driving and advanced driver-assistant systems. However, due to the uncertainties of vehicle dynamics and complexities of surroundings, long-term motion prediction is never trivial work. As they combine effects of humans, vehicles and environments, kinematic trajectory data reflect several aspects of vehicles’ spatial behaviors. In this paper, we propose a novel method that leverages spatial database and kinematic trajectory data to achieve long-term vehicular motion prediction in a lightweight way. In our system, a spatial database system is initially embedded in an extended Kalman filter (EKF) framework. The spatial kinematic trajectory data are managed through the database and directly used in motion prediction; namely, weighted means are derived from the spatially retrieved kinematic data and used to update EKF predictions. The proposed method is validated in the real world. The experiments indicate that different weighting methods make a slight accuracy difference. Our method is not data-and-computation-consumed; its performance is acceptable in the limited data conditions and its prediction accuracy is improved as the size of used data sets increases; our method can predict in real time. The efficiency of an unscented Kalman filter (UKF) is compared with that of the EKF. The results show that the UKF can hardly meet real-time requirements.

Recommended citation: Lu Tao, Yousuke Watanabe, and Hiroaki Takada. "A Lightweight Long-Term Vehicular Motion Prediction Method Leveraging Spatial Database and Kinematic Trajectory Data." ISPRS International Journal of Geo-Information 11, no. 9 (2022): 463.
Download Paper

Collision Risk Assessment Service for Connected Vehicles: Leveraging Vehicular State and Motion Uncertainties

Published in IEEE Internet of Things Journal, 2021

Abstract: The Internet of Things plays an indispensable role in the development of connected vehicles, which will pave the way for road safety applications. In recent years, the concept of a cooperative collision warning system (CCWS) has been introduced and developed to enhance road safety, and it has been seen as a typical Internet-of-Vehicles application. In most CCWSs, it is vital to have a detection mechanism based on trajectory predictions where the uncertainties associated with vehicular state and motion are complex. However, most available approaches in this regard did not consider these uncertainties. Hence, this article proposes a new collision risk assessment (CRA) method where sigma trajectories that include multiple possible trajectories considering multiple aspects of vehicular motion are designed to cope with vehicular uncertainties. Our method is implemented in a novel server-based architecture, which is different from the commonly used vehicle-based controlled CCWSs. The CRA is provided as a service by a cloud server. The proposed method and architecture are validated and evaluated through extensive real-world experiments. Experimental results show that our method outperforms a referenced method in terms of CRA and achieves better robustness in tolerating communication delays and dropouts. Latencies in CRA service were analyzed, and it was found that powerful computing resources provided by cloud servers can significantly decrease computational cost, which will indirectly compensate for communication costs in the future. Based on our high-performance CRA method, the proposed architecture can be regarded as a novel option for CCWS design.

Recommended citation: Lu Tao, Yousuke Watanabe, Yixiao Li, Shunya Yamada, and Hiroaki Takada. "Collision risk assessment service for connected vehicles: Leveraging vehicular state and motion uncertainties." IEEE Internet of Things Journal 8, no. 14 (2021): 11548-11560.
Download Paper

Comparative Evaluation of Kalman Filters and Motion Models in Vehicular State Estimation and Path Prediction

Published in Journal of Navigation, 2021

Abstract: Vehicle state estimation and path prediction, which usually involve Kalman filter and motion model, are critical tasks for intelligent driving. In vehicle state estimation, the comparative performance assessment, regarding accuracy and efficiency, of the unscented Kalman filter (UKF) and the extended Kalman filter (EKF) is rarely discussed. This paper is devoted to empirically evaluating the performance of UKF and EKF incorporating different motion models and investigating the models’ properties and the affecting factors in path prediction. Extensive real world experiments have been carried out and the results show that EKF and UKF have roughly identical accuracy in state estimation; however, EKF is faster than UKF generally; the fastest filter is about 2.6 times faster than the slowest. The path prediction experiments reveal that the velocity estimate and the used motion model affect path prediction; the more realistically the model reflects the vehicle’s driving status, the more reliable its predictions.

Recommended citation: Lu Tao, Yousuke Watanabe, Shunya Yamada, and Hiroaki Takada. "Comparative evaluation of Kalman filters and motion models in vehicular state estimation and path prediction." The Journal of Navigation 74, no. 5 (2021): 1142-1160.
Download Paper

Automatically Building Linking Relations between Lane-Level Map and Commercial Navigation Map Using Topological Networks Matching

Published in Journal of Navigation, 2020

Abstract: The lane-level map, which contains the lane-level information severely lacking in widely used commercial navigation maps, has become an essential data source for autonomous driving systems. The linking relations between lane-level map and commercial navigation map can facilitate an autonomous driving system mapping information between different applications using different maps. In this paper, an approach is proposed to build the linking relations automatically. The different topology networks are first reconstructed into similar structures. Then, to build the linking relations automatically, the adaptive multi-filter algorithm and forward path exploring algorithm are proposed to detect corresponding junctions and paths, respectively. The approach is validated by two real data sets of more than 150 km of roads, mainly highway. The linking relations for nearly 94% of the total road length have been built successfully.

Recommended citation: Lu Tao, Pan Zhang, Lixin Yan, and Dunyao Zhu. "Automatically Building Linking Relations between Lane-Level Map and Commercial Navigation Map Using Topological Networks Matching." The Journal of Navigation 73, no. 5 (2020): 1159-1178.
Download Paper

The Traffic Accident Hotspot Prediction: Based on the Logistic Regression Method

Published in 2015 International conference on transportation information and safety (ICTIS), 2015

Abstract: Traffic accident has posed a threat to the safety of human life. Accident often occurs in some certain areas. In order to achieve the target of the identification and judgment of the accidents hotspots during driving, and improve the driving safety of vehicle and the traffic efficiency through early warning, a research was done based on the recent collected 400 sets of accidents data of 10 major roads in Beijing city. Through the statistics of the typical factors, and the Logistic regression analysis, the relationships between the traffic accident and the road type, the vehicle type, the driver state, the weather, the date etc., were studied. Finally, the prediction model of accident hotspot was established. The results show that the location of car in road transects, the road safety grade, the road surface condition, the visual condition, the vehicle condition and the driver state are the most significant factors which may lead to traffic accident. Meanwhile, the prediction model established in this paper was validated to be capable of predicting the occurrence of accident, and the prediction accuracy is approximate 86.67%. The study provides not only a theoretical basis for vehicle safety assistance driving, but also the guidance for collision avoidance and optimization for path planning of intelligent vehicle.

Recommended citation: Lu Tao, Dunyao Zhu, Lixin Yan, and Pan Zhang. "The traffic accident hotspot prediction: Based on the logistic regression method." In 2015 International conference on transportation information and safety (ICTIS), pp. 107-110. IEEE, 2015.
Download Paper

Dr. Lu TAO

Selected Publications