Research

Multimodal Assessment of Neonatal Pain Using Computer Vision

Abstract

Infants receiving care in the Neonatal Intensive Care Unit (NICU) experience several painful procedures during their hospitalization. Assessing neonatal pain is difficult because the current standard for assessment is subjective, inconsistent, and discontinuous. The intermittent and inconsistent assessment can induce poor treatment and, therefore, cause serious longterm outcomes. The main aim of this project is to develop a robust and comprehensive automatic system that generates a standardized pain assessment comparable to those obtained by conventional nurse-derived pain scores. The continuous monitoring of pain, using affordable, non-invasive, and easily integrable devices, provides immediate pain detection and intervention, and therefore, contribute to improved long term outcomes; i.e., reduce the outcomes of under- and over-treatment. It can also decrease caregivers’ bias and assessment burden. While further research is needed, the preliminary results of our research showed that the automatic assessment of neonatal pain is a viable and more efficient alternative to the manual assessment.

Team

Engineering

  • Dr. Yu Sun, Professor, Bellini College of AI, Cybersecurity and Computing, USF
  • Dr. Dmitry Goldgof, Professor, Bellini College of AI, Cybersecurity and Computing, USF
  • Jacqueline Hausmann, PhD Student, Bellini College of AI, Cybersecurity and Computing, USF
  • Md Imran Hossain, PhD Student, Bellini College of AI, Cybersecurity and Computing, USF
  • Anthony McCofie, PhD Student, Bellini College of AI, Cybersecurity and Computing, USF

Medical Team

  • Dr. Thao Ho D.O., Pediatrics, USF Health, USF
  • Marcia Kneusel RN, MPH, Pediatrics/Neonatology, USF
  • Courtney Casey RN, Clinical Research Nurse, College of Medicine Pediatrics, USF Health, USF
  • Dr. Stephanie Prescott PhD, MSN, NNP-BC, Director of Neonatal Research, Invoa Health Services
  • Dr. Kanwaljeet S. Anand MBBS, D.Phil, Pediatrics and Anesthesiology, Stanford University
  • Dr. Melissa Scala M.D., Pediatrics/Neonatology, Stanford University
  • Dr. Yangxin Huang Biostatistics, College of Public Health, USF

Collaborators

Previous Team Members

  • Dr. Denise Maguire, Associate Professor, College of Nursing, USF Health, USF
  • Dr. Md Sirajus Salekin, Applied Scientist, Amazon
  • Dr. Ghada Zamzmi, Research Fellow, NLM, NIH
  • Jiayi Wang, PhD Student, Computer Science and Engineering, USF
  • Dr. Terri Ashmeade, Professor, College of Medicine Pediatrics, USF Health, USF
  • Dr. Rangachar Kasturi, Professor, CSE, USF
  • Chih-Yun Pai, Master's Student, CSE, USF

Patents

  • US-11631280-B2, "System and Method for Multimodal Spatiotemporal Pain Assessment", April 2023.
  • US-20230309915-A1, "System and Method for Attentional Multimodal Pain Estimation", October 2023.
  • US-11202604-B2, "Comprehensive and Context-sensitive Neonatal Pain Assessment System and Methods Using Multiple Modalities", December 2021.
  • US-20210052215-A1, "System and Method for Multimodal Spatiotemporal Pain Assessment", Feburary 2021.
  • US-20210030354-A1, "Neonatal Pain Identification From Neonatal Facial Expressions", Feburary 2021.
  • US-10827973-B1, "Machine-based Infants Pain Assessment Tool", November 2020.
  • US-20190320974-A1, "Comprehensive and Context-sensitive Neonatal Pain Assessment System and Methods Using Multiple Modalities", October 2020.

Published Papers

Media Coverage

Talk / Presentation

  • Aug 06, 2019 - Dr. Dmitry Goldgof, A joint seminar of the Data Science Research Center and the Department of Biomedical Engineering at Ben-Gurion University of Negev, Israel on "Healthcare in the Age of AI and Deep Learning: Automatic Assessment of Neonatal Pain"
  • Jun 18, 2019 - Dr. Dmitry Goldgof, Weekly Idea/Research Collabaration Meeting at Johns Hopkins All Children's Hospital, St. Petersburg, FL, USA, on "Automatic Neonatal Pain Detection"
  • Mar 26, 2019 - Dr. Dmitry Goldgof, High Profile Talk at University of South Florida, Tampa, FL, USA, on "Healthcare in the Age of AI and Deep Learning: Automatic Assessment of Neonatal Pain"

Our Sponsors