Instructions on Creating 3D Human Models and Virtual Reality Implementation of Them Reported by: Sophia Franklin and Caleb Lee ACLab Students Mentors: Shuangjun Liu, Naveen Sehgal, Xiaofei Huang, and Isaac McGonagle YSP Summer 2019 Overvi..
Augmented Cognition is about enhancing human information-processing capabilities through the design of adaptive interfaces via physical, psychological, and cognitive state estimation.
Augmented Cognition Lab (ACLab) research centers on creating augmented cognition systems to enhance human cognition rather than replace it. The augmented cognition systems have three primary components: (1) the sensing element, (2) the analytic element, and (3) the feedback element, as shown in figure above. The sensing element gathers and fuses multi-modal data from the human and the environment including 4D color-depth videos, neurophysiological signals, and audio/speech information. Robust analytics is the cornerstone of the system. At ACLab, we use both machine learning models and biomechanically and biologically inspired structural models. When possible, a structural model is preferred because it can incorporate existing knowledge and research into the model without requiring a large training set to gain such knowledge. In addition, structural models tend to be more transparent and easier to analyze for failure modes and edge cases. Careful design of the feedback element is also critical because unless the feedback is useful, timely, and understandable, the system will be unusable regardless of the quality of the other two components.
We are actively collaborating with medical doctors, physiologists, and therapists to define problems and fine-tune augmented cognition solutions for children with Autism spectrum disorder (ASD), individuals with limited speech and physical abilities (LSPA), individual with sensory impairments, and the elderly. Even apparently simple problems in these domains have a complex web of interconnected elements with significant engineering and science implications. Therefore, at ACLab, we extensively work at the intersection of computer vision and machine learning with multidisciplinary elements from behavioral sciences. In summary, two main lines of research in ACLab are:
- Digital Prosthetics: Replacing lost sensing and data processing functionality
- Intelligence Amplification: Enhancing sensing and data processing functionality
For many of these projects, augmented reality (AR) and virtual reality (VR) tools are essential for both the assessment and enhancement portions of the project. Below, you can find some of our active research projects. Also visit our The Signal Processing, Imaging, Reasoning, and Learning (SPIRAL) Group for more information about our collaborative cluster.