Exploiting Local Shape Information for Cross-Modal Person Re-identification

Abstract

In computer vision, person re-identification (Re-id) is an important problem, aiming to match people across multiple camera views. Most of the existing Re-id systems widely use RGB-based appearance cues, which is not suitable when lighting conditions are very poor. However, for many security reasons, sometimes continued surveillance via camera in low lighting conditions is inevitable. To overcome this problem, we take advantage of the Kinect sensor based depth camera (e.g., Microsoft Kinect), which can be installed in dark places to capture video, while RGB based cameras can be installed in good lighting conditions. Such types of heterogeneous camera networks can be advantageous due to the different sensing modalities available but face challenges to recognize people across depth and RGB cameras. In this paper, we propose a body partitioning method and novel HOG based feature extraction technique on both modalities, which extract local shape information from regions within an image. We find that combining the estimated features on both modalities can sometimes help to better reduce visual ambiguities of appearance features caused by lighting conditions and clothes. We also propose an effective metric learning approach to obtain a better re-identification accuracy across RGB and depth. Experimental results on two publicly available RGBD-ID datasets show the effectiveness of our proposed method.