Skeleton model based behavior recognition for pedestrians and cyclists from vehicle sce ne camera

Abstract

With the significant advances in computer vision research, skeleton model based human pose recognition has become more accurate and time-efficient, although most of the applications are limited in laboratory environment or on surveillance videos. This paper proposes a pose tracking and behavior recognition method from in-vehicle scene camera. It will not only detect pedestrians on the road, but also generate their skeleton models describing head, limb, and trunk movements. Based on these more detailed movements of body parts, the proposed method is designed to track poses of pedestrians and cyclists with the potentials to enable automated pedestrian gesture reading and non-verbal interactions between autonomous vehicles and pedestrians. The proposed algorithm has been tested on different databases including TASI 110-car naturalistic driving database and Joint Attention for Autonomous Driving (JAAD) database. Results show that key frames describing different pedestrian and cyclist negotiation gestures are detected from the raw video streams using the proposed method. These results will improve our understanding of pedestrian and cyclist's intentions and can be further used for autonomous vehicle control algorithm development.