Human Pose Estimation: Deep Learning with Small Dataset

Although human pose estimation for various computer vision (CV) applications has been studied extensively in the last few decades, yet in-bed pose estimation using camera-based vision methods has been ignored by the CV community because it is assumed to be identical to the general purpose pose estimation methods. However, in-bed pose estimation has its own specialized aspects and comes with specific challenges including the notable differences in lighting conditions throughout a day and also having different pose distribution from the common human surveillance viewpoint. In this work, we demonstrate that these challenges significantly lessen the effectiveness of existing general purpose pose estimation models. In order to address the lighting variation challenge, infrared selective (IRS) image acquisition technique is proposed to provide uniform quality data under various lighting conditions. In addition, to deal with unconventional pose perspective, a 2-end histogram of oriented gradient (HOG) rectification method is presented. Deep learning framework proves to be the most effective model in human pose estimation, however the lack of large public dataset for in-bed poses prevents us from using a large network from scratch. In this work, we explored the idea of employing a pre-trained convolutional neural network (CNN) model trained on large public datasets of general human poses and fine-tuning the model using our own shallow (limited in size and different in perspective and color) in-bed IRS dataset.

Selected Publication:

  • “In-Bed Pose Estimation: Deep Learning with Shallow Dataset,” [available at arXiv].
  • “A Vision-Based System for In-Bed Posture Tracking,” ICCV/ACVR’17. [Code]
  • “In-Bed Posture Classification Using Deep Autoencoders,” EMBC’16.


000061 000031 000001

Related dataset:

Dataset description: This in-bed pose dataset is collected via our infrared selective (IRS) system in a simulated hospital room in the College of Health Science at Northeastern University. Raw pose dataset is provided with labeling where images keep original color and resolution during collection. We also provide preprocessed version of IRS images with multiple lying directions. All images are scaled and make into 3 channel gray-scale data, which can be hooked up directly to our In-Bed-Pose-Estimation code.

In addition, we have provided a dataset of RGB version captured with a off-the-shelf webcam logitech C525. We used this dataset for comparison purpose in our paper to show the effect of color information loss during in-bed pose estimation.

Code release:

  • Here is a minimum release where we only provide the necessary part to repurpose the CPM model which includes our pretrained IRS caffe model and prototype file for different training strategies. To use this model, you need to deploy original CPM code first. [min_release]
  • We also provided a hooked up version of CPM where the interface for our dataset has been already defined. If you save time to figure out how to hook our model to original CPM, you can use this version directly. Our pretrained model is saved in min_release. If you want to run the test directly, please also download our pretrained model: In-Bed-Pose-Estimation
Blog Attachment