A Part Fusion Model for Action Recognition in Still Images

Abstract

Recognizing actions from still images is very challenging due to the lack of human movement information and the variance of background in action categories. The existing methods in action recognition focus on extracting scene context information, modeling the human-object pair and its interactions or using human body information, of which part-based methods are one of the successful methods, which extract rich semantic information from the human body parts or pose. However, most of part-based models need to rely on expensive part annotations to improve the recognition accuracy. Different from these methods, in this paper, we propose a part fusion model which can effectively combine the discriminative parts information without extra annotations for action recognition. In our model, a guided attention module is used to further extract the more discriminative information in the image. And the part-level features are trained with the different weighted loss, which is mainly based on different object and background parts’ characteristics. In order to further enhance the model performance, part-level features are fused to form new image-level features in the global supervision learning. This method achieves state-of-the-art result on the PPMI dataset and significant competitive performance on the Stanford-40 dataset, which demonstrates the effectiveness of our method for characterizing actions.