eyeSay: Brain Visual Dynamics Decoding With Deep Learning & Edge Computing

If you need an accessible version of this item, please submit a remediation request.
Date
2022-07-25
Language
American English
Embargo Lift Date
Committee Members
Degree
Degree Year
Department
Grantor
Journal Title
Journal ISSN
Volume Title
Found At
IEEE
Abstract

Brain visual dynamics encode rich functional and biological patterns of the neural system, and if decoded, are of great promise for many applications such as intention understanding, cognitive load quantization and neural disorder measurement. We here focus on the understanding of the brain visual dynamics for the Amyotrophic lateral sclerosis (ALS) population, and propose a novel system that allows these so- called ‘lock-in’ patients to ‘speak’ with their brain visual movements. More specifically, we propose an intelligent system to decode the eye bio-potential signal, Electrooculogram (EOG), thereby understanding the patients’ intention. We first propose to leverage a deep learning framework for automatic feature learning and classification of the brain visual dynamics, aiming to translate the EOG to meaningful words. We afterwards design and develop an edge computing platform on the smart phone, which can execute the deep learning algorithm, visualize the brain visual dynamics, and demonstrate the edge inference results, all in real-time. Evaluated on 4,500 trials of brain visual movements performed by multiple users, our novel system has demonstrated a high eye-word recognition rate up to 90.47%. The system is demonstrated to be intelligent, effective and convenient for decoding brain visual dynamics for ALS patients. This research thus is expected to greatly advance the decoding and understanding of brain visual dynamics, by leveraging machine learning and edge computing innovations.

Description
item.page.description.tableofcontents
item.page.relation.haspart
Cite As
Zou, J., & Zhang, Q. (2022). eyeSay: Brain Visual Dynamics Decoding With Deep Learning & Edge Computing. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 30, 2217–2224. https://doi.org/10.1109/TNSRE.2022.3193714
ISSN
Publisher
Series/Report
Sponsorship
Major
Extent
Identifier
Relation
Journal
IEEE Transactions on Neural Systems and Rehabilitation Engineering
Source
Publisher
Alternative Title
Type
Article
Number
Volume
Conference Dates
Conference Host
Conference Location
Conference Name
Conference Panel
Conference Secretariat Location
Version
Final published version
Full Text Available at
This item is under embargo {{howLong}}