Research
Multimodal Assessment of Neonatal Pain Using Computer Vision
Abstract
Infants receiving care in the Neonatal Intensive Care Unit (NICU) experience several painful procedures during their hospitalization. Assessing neonatal pain is difficult because the current standard for assessment is subjective, inconsistent, and discontinuous. The intermittent and inconsistent assessment can induce poor treatment and, therefore, cause serious longterm outcomes. The main aim of this project is to develop a robust and comprehensive automatic system that generates a standardized pain assessment comparable to those obtained by conventional nurse-derived pain scores. The continuous monitoring of pain, using affordable, non-invasive, and easily integrable devices, provides immediate pain detection and intervention, and therefore, contribute to improved long term outcomes; i.e., reduce the outcomes of under- and over-treatment. It can also decrease caregivers鈥� bias and assessment burden. While further research is needed, the preliminary results of our research showed that the automatic assessment of neonatal pain is a viable and more efficient alternative to the manual assessment.
Team
Engineering
- Dr. Yu Sun, Professor, Bellini College of AI, Cybersecurity and Computing, USF
- Dr. Dmitry Goldgof, Professor, Bellini College of AI, Cybersecurity and Computing, USF
- Jacqueline Hausmann, PhD Student, Bellini College of AI, Cybersecurity and Computing, USF
- Md Imran Hossain, PhD Student, Bellini College of AI, Cybersecurity and Computing, USF
- Anthony McCofie, PhD Student, Bellini College of AI, Cybersecurity and Computing, USF
Medical Team
- Dr. Thao Ho D.O., Pediatrics, USF Health, USF
- Marcia Kneusel RN, MPH, Pediatrics/Neonatology, USF
- Courtney Casey RN, Clinical Research Nurse, College of Medicine Pediatrics, USF Health, USF
- PhD, MSN, NNP-BC, Director of Neonatal Research, Invoa Health Services
- MBBS, D.Phil, Pediatrics and Anesthesiology, Stanford University
- M.D., Pediatrics/Neonatology, Stanford University
- Dr. Yangxin Huang Biostatistics, College of Public Health, USF
Collaborators
- Dr. Peter Mouton, Director and Chief Scientific Officer, SRC Biosciences
- , Professor and Director of , Ben-Gurion University of the Negev, Israel
Previous Team Members
- Dr. Denise Maguire, Associate Professor, College of Nursing, USF Health, USF
- , Applied Scientist, Amazon
- Dr. Ghada Zamzmi, Research Fellow, NLM, NIH
- Jiayi Wang, PhD Student, Computer Science and Engineering, USF
- Dr. Terri Ashmeade, Professor, College of Medicine Pediatrics, USF Health, USF
- Dr. Rangachar Kasturi, Professor, CSE, USF
- Chih-Yun Pai, Master's Student, CSE, USF
Patents
- US-11631280-B2, "System and Method for Multimodal Spatiotemporal Pain Assessment", April 2023.
- US-20230309915-A1, "System and Method for Attentional Multimodal Pain Estimation", October 2023.
- US-11202604-B2, "Comprehensive and Context-sensitive Neonatal Pain Assessment System and Methods Using Multiple Modalities", December 2021.
- US-20210052215-A1, "System and Method for Multimodal Spatiotemporal Pain Assessment", Feburary 2021.
- US-20210030354-A1, "Neonatal Pain Identification From Neonatal Facial Expressions", Feburary 2021.
- US-10827973-B1, "Machine-based Infants Pain Assessment Tool", November 2020.
- US-20190320974-A1, "Comprehensive and Context-sensitive Neonatal Pain Assessment System and Methods Using Multiple Modalities", October 2020.
Published Papers
A. McCofie, A. Kandiyana, Peter R Mouton, Yu Sun, and Dmitry Goldgof
International Symposium on Computer-Based Medical Systems (CBMS), 2025- Jacqueline Hausmann, Jiayi Wang, Marcia Kneusel, Stephanie Prescott, Peter R Mouton,
Yu Sun, and Dmitry Goldgof
International Symposium on Computer-Based Medical Systems (CBMS), 2025
Md Imran Hossain, Ghada Zamzmi, Peter R Mouton, Yu Sun, and Dmitry Goldgof
IEEE Engineering in Medicine and Biology Society (EMBC24), 2024
Jacqueline Hausmann, Md Sirajus Salekin, Ghada Zamzmi, Peter R Mouton, Stephanie M.Prescott, Yu Sun,and Dmitry Goldgof
IEEE Access, 2024
Md Imran Hossain, Ghada Zamzmi, Peter R Mouton, Md Sirajus Salekin, Yu Sun, and Dmitry Goldgof
ACM Computing Surveys, 2023
Amelia Llerena, Krystal Tran, Danyal Choudhary, Jacqueline Hausmann, Dmitry Goldgof, Yu Sun, and Stephanie M.Prescott
Frontier in Pediatrics, 2023
Md Imran Hossain, Ghada Zamzmi, Peter R Mouton, Yu Sun, and Dmitry Goldgof
2023 IEEE 36th International Symposium on Computer-Based Medical Systems (CBMS)
Md Sirajus Salekin, Ghada Zamzmi, Dmitry Goldgof, Peter R. Mouton, Kanwaljeet J. S. Anand, Terri Ashmeade, Stephanie Prescott, Yangxin Huang, and Yu Sun
Medical Image Computing and Computer Assisted Intervention (MICCAI), 2022
Jacqueline Hausmann, Md Sirajus Salekin, Ghada Zamzmi, Dmitry Goldgof, and Yu Sun
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) workshops, 2021
Sidharth Srivatsav Sribhashyam, Md Sirajus Salekin, Dmitry Goldgof, Ghada Zamzmi, Mark Last, and Yu Sun
IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2021
Md Sirajus Salekin, Peter R Mouton, Ghada Zamzmi, Raj Patel, Dmitry Goldgof, Marcia Kneusel, Sammie L Elkins, Eileen Murray, Mary E Coughlin, Denise Maguire, Thao Ho, and Yu Sun
Paediatric and Neonatal Pain, 2021
Md Sirajus Salekin, Ghada Zamzmi, Jacqueline Hausmann, Dmitry Goldgof, Rangachar Kasturi, Marcia Kneusel, Terri Ashmeade, Thao Ho, Yu Sun
Data in Brief, 2021
Md Sirajus Salekin, Ghada Zamzmi, Dmitry Goldgof, Rangachar Kasturi, Thao Ho, Yu Sun
Computers in Biology and Medicine, 2021
Md Sirajus Salekin, Ghada Zamzmi, Dmitry Goldgof, Rangachar Kasturi, Thao Ho, Yu Sun
IEEE International Conference on Automatic Face and Gesture Recognition (FG), 2020
Md Sirajus Salekin, Ghada Zamzmi, Rahul Paul, Dmitry Goldgof, Rangachar Kasturi, Thao Ho, Yu Sun
IEEE Healthcare Innovations and Point of Care Technologies Conference (HI-POCT), 2019
Md Sirajus Salekin, Ghada Zamzmi, Dmitry Goldgof, Rangachar Kasturi, Thao Ho, Yu Sun
IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2019
Ghada Zamzmi, Rahul Paul, Md Sirajus Salekin, Dmitry Goldgof, Rangachar Kasturi, Thao Ho, Yu Sun
IEEE Transactions on Biometrics, Behavior, and Identity Science, 2019
Ghada Zamzmi, Rahul Paul, Dmitry Goldgof, Rangachar Kasturi, Terri Ashmeade, Yu Sun
International Joint Conference on Neural Networks (IJCNN), 2019
Ghada Zamzmi, Chih-Yun Pai, Dmitry Goldgof, Rangachar Kasturi, Terri Ashmeade, Yu Sun
IEEE Transactions on Affective Computing, 2019
Ghada Zamzmi, Dmitry Goldgof, Rangachar Kasturi, Yu Sun
UbiComp, 2018
Ruicong Zhi, Ghada Zamzmi, Dmitry Goldgof, Terri Ashmeade, Yu Sun
Journal of Clinical Medicine, 2018
Ruicong Zhi, Ghada Zamzmi, Dmitry Goldgof, Terri Ashmeade, Tingting Li, Yu Sun
IEICE Transactions on Information and Systems, 2018
Ghada Zamzmi, Rangachar Kasturi, Dmitry Goldgof, Ruicong Zhi, Terri Ashmeade, Yu Sun
IEEE Reviews in Biomedical Engineering, 2018
Ghada Zamzmi, Chih-Yun Pai, Dmitry Goldgof, Rangachar Kasturi, Yu Sun, Terri Ashmeade
Scandinavian Conference on Image Analysis (SCIA), 2017
Ghada Zamzmi, Chih-Yun Pai, Dmitry Goldgof, Rangachar Kasturi, Terri Ashmeade, Yu Sun
International Conference on Pattern Recognition (ICPR), 2016
Ghada Zamzami, Gabriel Ruiz, Dmitry Goldgof, Rangachar Kasturi, Yu Sun, Terri Ashmeade
IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), 2015
Media Coverage
- Jun 15, 2021 - USF Research and Innovation Newsroom 2021
- Jun 14, 2017 -
- Mar 24, 2017 -
- Feb 22, 2017 -
Talk / Presentation
- Aug 06, 2019 - Dr. Dmitry Goldgof, A joint seminar of the and the Department of Biomedical Engineering at Ben-Gurion University of Negev, Israel on "Healthcare in the Age of AI and Deep Learning: Automatic Assessment of Neonatal Pain"
- Jun 18, 2019 - Dr. Dmitry Goldgof, 911爆料网ekly Idea/Research Collabaration Meeting at , St. Petersburg, FL, USA, on "Automatic Neonatal Pain Detection"
- Mar 26, 2019 - Dr. Dmitry Goldgof, High Profile Talk at 911爆料网, Tampa, FL, USA, on "Healthcare in the Age of AI and Deep Learning: Automatic Assessment of Neonatal Pain"