An AR-EEG Hybrid System for Visual Neglect Rehabilitation

This research is funded by the NSF Award #1915065 entitled “SCH: INT: Collaborative Research: Detection, Assessment and Rehabilitation of Stroke-Induced Visual Neglect Using Augmented Reality (AR) and Electroencephalography (EEG)“. 

AR-based EEG-guided Neglect detection, assessment, and rehabilitation (AREEN) system is a neglect detection, assessment and rehabilitation tool that incorporates stimulation to draw attention to the affected side of the body and environment based on the automatic detection of neglect through neurophysiology measured through electroencephalography (EEG), thereby stimulating neural networks responsible for spatial attention and perceptual processing.

The AREEN system is composed of four interconnected units as shown in figure below. These units will work together in real time to assess the available field of vision, and individualize a program of multimodal sensory feedback designed to reduce neglect and neglect-related disability. (1) The AREEN analysis and control unit is built on PC to initialize the AR visualization and EEG monitoring units, and provide synchronization between these two units. (2) The AR visualization unit, based on Microsoft HoloLens1, will be responsible for the presentation of the “Starry Night” scheme for neglect detection and feedback for the rehabilitation program. (3) The EEG monitoring unit will record the EEG data through g.USBamp, extract salient EEG features associated with visual attention and inattention and communicate these features to the analysis and control unit. Using the EEG features, the analysis and control unit will perform an automatic neglect detection and if a target is missed, the analysis and control unit will activate the feedback unit. (4) The multimodal feedback unit will initiate haptic, auditory, and visual stimuli that will be presented to the user to direct the user’s attention to the neglected side of the visual field.

Latest News & Products:

• D. Kocanaogullari, J. Mak, J. Kersey, A. Khalaf, S. Ostadabbas, G. Wittenberg, E. Skidmore, and M. Akcakaya,” EEG-based Neglect Detection for Stroke Patients,”  IEEE EMBC 2020 (accepted for publication, presented virtually but not available online yet.)
• M.Akcakaya, D. Kocanaogullari, A. Khalaf, J. Kersey, J. Mak, X. Huang, S. Ostadabbas, G. Wittenberg, and E. Skidmore, “ An EEG-based BCI for Visual Spatial Neglect Detection and Assessment,” BCI Meeting 2020 (Abstract accepted but the conference/workshop postponed). 
• A.Khalaf, J. Kersey, S. Eldeeb, G. Alankus, E. Grattan, L. Waterstram, E. Skidmore, and M. Akcakaya, “ A Passive EEG-based Brain Computer Interface for Assessment of Visuospatial Neglect,” BCI Journal (under review)
• A Passive EEG-based Brain Computer Interface for Assessment of Visuospatial Neglect, Virtually presented at Minisymposium on Artificial Intelligence in Rehabilitation at IEEE EMBC 2020. 
• A Passive EEG-based Brain Computer Interface for Assessment of Visuospatial Neglect, was accepted to be presented at the Workshop on BCI in Stroke Rehabilitation at the BCI Meeting 2020 (the conference is postponed)
• Detection, Assessment and Rehabilitation of Stroke-Induced Visual Neglect Using Augmented Reality (AR) and Electroencephalography (EEG), Presented at the monthly meeting of Stroke Rehabilitation Research Network of Pittsburgh.  
• News @University of Pittsburgh (October 23, 2019): Bringing Attention to Visual Neglect in Stroke
• News @Northeastern University (July 31, 2019): NSF Grant to combined Augmented Reality (AR) and EEG for Stroke-Induced Visual Neglect Rehabilitation