Open-source code/datasets developed/collected at ACLab
- Seeing Under the Cover – This is the code for our MICCAI2019 paper “Seeing Under the Cover: A Physics Guided Learning Approach for In-Bed Pose Estimation.”
- ASD-Behavior-Classification – This is the code for the MLSP2019 paper on “Recognition of Atypical Behavior in Autism Diagnosis from Video using Pose Estimation Over Time.”
- AH-CoLT: An AI-Human Co-Labeling Toolbox – This is the code for the MLSP2019 paper on “AH-CoLT: An AI-Human Co-Labeling Toolbox to Augment Efficient Groundtruth Generation”.
- DeepPBM – This is the code for the paper on “DeepPBM: Deep Probabilistic Background Model Estimation from Video Sequences”.
- Indoor-GeoNet – This is the code for the paper on “Indoor GeoNet: Weakly Supervised Hybrid Learning for Depth and Pose Estimation.”
- ScanAva Generation Toolkit – This is the code for our ECCV2018 workshop entitled, “A Semi-Supervised Data Augmentation Approach using 3D Graphical Engines.”
- ISP-GPM – This is the code for our ECCV2018 paper entitled, “Inner Space Preserving Generative Pose Machine.”
- 3D Facial Landmark Detection and Tracking -This is the code for our IJCAI2018 workshop paper entitled, Facial Expression and Peripheral Physiology Fusion to Decode Individualized Affective Experience.” To track relative movements of the facial landmarks from a video, we have developed a robust tracking approach, in which head movement is also tracked and decoupled from the facial landmark movements. We first employed an state-of-the-art 2D facial alignment algorithm to automatically localize 68 landmarks for each frame of the face video. Then, a 3D face model is used to extract the depth information from 2D frames in order to achieve 3D landmark tracking.
- Indoor Navigation via Vision-Inertial Data Fusion – This is the code for our IEEE/ION PLANS2018 paper entitled, “First-Person Indoor Navigation via Vision-Inertial Data Fusion.”
- Compressive Sensing for Plantar Pressure – This is the code for our MLHC2017 paper entitled, “Spatially-Continuous Plantar Pressure Reconstruction Using Compressive Sensing.”
- In-Bed-Pose-Estimation – In this work, we explored the idea of employing a pre-trained convolutional neural network (CNN) model trained on large public datasets of general human poses and fine-tuning the model using our own shallow (limited in size and different in perspective and color) in-bed IRS dataset.
- In-Bed-Posture-Estimation – A vision-based tracking system for pervasive yet unobtrusive long-term monitoring of in-bed postures in different environments.
- Augmented fRMC – This is the code for the paper “Moving Object Detection through Robust Matrix Completion Augmented with Objectness,” published in the IEEE Journal of Selected Topics in Signal Processing (J-STSP), 2018.
- fRMC – Our fast robust matrix completion (fRMC) models the background in the framework of matrix completion in order to detect the foreground without any prior knowledge about the moving objects.
- Kinect V2 Recorder – Microsoft Windows RGBD data recorder for Kinect V2. Has adaptive frame rate and lossy compression of RGB and lossless compression of Depth data. Data can be accessed by the included python script or the reader software below.
- Kinect V2 Reader – Microsoft Windows RGBD data playback for Kinect V2. Can playback and use Kinect RGBD data recorded by the above recorder.
- Simultaneously-collected multimodal Lying Pose (SLP) – the first-ever large scale dataset on in-bed poses called “Simultaneously-collected multimodal Lying Pose (SLP)” (is pronounced as SLEEP).
- ScanAva – This dataset is a large synthetic human pose dataset, called Scanned Avatar (ScanAva) using 3D scans of 7 individuals based on our proposed augmentation approach presented in our ECCV2018 workshop paper “A Semi-Supervised Data Augmentation Approach using 3D Graphical Engines.“
- The Emotional Voices Database – This dataset is built for the purpose of emotional speech synthesis. The transcript were based on the CMU arctic database. Our database includes recordings for four speakers (2 males and 2 females). The emotional styles are neutral, sleepiness, anger, disgust and amused. Each audio file is recorded in 16bits .wav format
- Mannequin RGB in-bed dataset (High-res) – This in-bed pose dataset is collected via regular webcam in a simulated hospital room in the College of Health Science at Northeastern University.
- Mannequin RGB in-bed dataset (Low-res) – This in-bed pose dataset is collected via regular webcam in a simulated hospital room in the College of Health Science at Northeastern University. The images have been downsampled to work with our In-Bed-Posture-Estimation code.
- Mannequin IRS in-bed dataset – This in-bed pose dataset is collected via our infrared selective (IRS) system in a simulated hospital room in the College of Health Science at Northeastern University. Raw pose dataset is provided with labeling where images keep original color and resolution during collection.
- Preprocessed multiple direction dataset – We also provide preprocessed version of IRS images with multiple lying directions. All images are scaled and make into 3 channel gray-scale data, which can be hooked up directly to our In-Bed-Pose-Estimation code.
ToolBoxes & Apps
- Biosignal-Specific Processing (Bio-SP) Tool – This Mathworks toolbox centers around the development of a biosignal-speciﬁc processing pipeline in order to analyze these physiological signals in a modular fashion based on the state-of-the-art studies reported in scientific literature. Also, our paper “A Biosignal-Specific Processing Tool for Machine Learning and Pattern Recognition” is published at the IEEE-NIH 2017 Special Topics Conference on Healthcare Innovations and Point-of-Care Technologies (HI-POCT 2017).
- ACLab Video and Motion Collector – For our vision-inertial data fusion evaluation in our IEEE ION paper “First-Person Indoor Navigation via Vision-Inertial Data Fusion“, we have developed an iPhone application that collects video and IMU data synchronously with an adjustable recording frequency.