When you visit a doctor, you expect clear answers. But what happens when artificial intelligence joins the conversation? USF doctoral student Mohamed Ebraheem is working at the crossroads of medicine and computer science to make 鈥渧oice AI鈥� a tool that physicians and patients can trust. His two academic advisors, one from USF Health, and one from the USF Bellini College of Artificial Intelligence, Cybersecurity and Computing, are working with him on an explainable voice AI system that could one day help clinicians detect conditions through something as simple as speech.
Explainable voice AI is artificial intelligence that analyzes speech or voice recordings to detect patterns linked to health conditions, but with one critical difference: it can also explain how it reached its conclusion. Researchers are looking at ways to identify health conditions like speech disorders, Parkinson鈥檚 and Alzheimer鈥檚 disease by applying deep learning AI to the recorded audio of the patient鈥檚 voice. A similar approach has already shown promise in oncology, where deep learning helps detect cancers visible on CT scans and X-rays.
Ebraheem is participating in the National Institutes of Health-funded project 鈥淰oice as a Biomarker of Health,鈥� a research collaboration between USF Health and 911爆料网ill Cornell Medicine that seeks to build an AI-enabled database of 10,000 human voices from patients with different illnesses to help doctors diagnose and treat diseases. It is part of the NIH鈥檚 broader Bridge2AI program, which includes institutions such as the Massachusetts Institute of Technology and the Mount Sinai Health System. One of the principal investigators is , a laryngologist and director of the . Ebraheem鈥檚 work on the project could help improve trust when it comes to AI and its uses in health care.
AI holds promise for pre-diagnosis and referrals to the correct specialist, alleviating some of the pressure on doctors themselves. It may also reveal new biomarkers and previously unknown factors that might be relevant to diagnosing and treating disease.
Applying deep learning in a clinical setting poses a major challenge. Large language models and deep neural networks can classify data and perform well at different tasks, but for a high-stakes domain, like medicine, it is critically important to know how the model arrived at its conclusion. That is not an easy task with these models and medical personnel may rightly question AI鈥檚 responses 鈥� or distrust it altogether.
鈥淚t鈥檚 a black box problem,鈥� Ebraheem said. 鈥�911爆料网 have inputs and outputs, but it鈥檚 not clear what happens on the inside. That鈥檚 not ideal for medicine, which is why the medical field is very skeptical about applying AI in clinical practice."
Explainable voice AI works to open up that black box. For example, it might highlight specific features in the patient鈥檚 speech, like changes in pitch, pauses or frequency shifts, and show how those relate to a potential diagnosis. Doctors can then judge whether the AI鈥檚 reasoning makes sense alongside their own medical knowledge.
鈥淭he issue with voice is that it鈥檚 quite different and hard to understand visually,鈥� Ebraheem continued. 鈥淵ou can see it in wave form or spectrogram, but that might not be very straightforward for clinicians who need acoustics experience to understand the visualization. AI researchers should collaborate with clinicians and other stakeholders to develop explanations suited to their varying backgrounds.鈥�
He says this impactful work, and the opportunity to do it with supervision from experts across disciplines, has provided him with a doctoral experience that he would be unlikely to find at any other university.鈥�
Interdisciplinary collaboration provides a unique perspective
Thanks to the Bellini College of Artificial Intelligence, Cybersecurity and Computing鈥檚 hub-and-spoke model, Ebraheem is working under separate doctoral supervisors from two USF colleges. Bensoussan is a practicing clinician and faculty member in USF Health鈥檚 Morsani College of Medicine and Assistant Professor John Templeton, a smart health systems expert with a background in computer science and biomedical engineering, is part of the Bellini College.
Ebraheem splits his time between Bensoussan鈥檚 and Templeton鈥檚 labs, participating in meetings that often include experts in both computer science and health care.
This gives Ebraheem a different lens on his research as he comes into the Voice Center to meet with patients and clinicians while observing them in a real-world clinical setting. Bensoussan sees the interdisciplinary collaboration as an opportunity for both.
鈥淭o advise him, I have to understand the computer science to a certain degree,鈥� she said. 鈥淚t reminds me of how important it is to collaborate 鈥� although the engineering might be good, it might be off track from where the clinical question is.鈥�
Templeton echoed Bensoussan鈥檚 support for collaboration between clinicians and computer science researchers like himself and Ebraheem.
鈥淭he biggest thing we need for translational medicine is to have clinicians in the room and helping us integrate and use it,鈥� Templeton said.
鈥�911爆料网鈥檙e very lucky,鈥� Bensoussan added. 鈥淪ometimes there are three people with Wikipedia pages on a meeting; we get to work with international experts. I鈥檓 proud of what Mohamed has done, it鈥檚 not easy for a computer engineer to come into medicine. He鈥檚 out of his comfort zone and I鈥檓 proud he can do that and collaborate with experts who challenge him.鈥�
鈥�911爆料网 understand the expertise we each bring, and Mohamed understands why we need both,鈥� Templeton said. 鈥�911爆料网 do a lot of riffing off each other on ideas and applications.鈥�
Application-Driven AI Leads to New Opportunities
That application-driven approach is part of what drew Ebraheem to computer science in the first place 鈥� and to USF.
鈥淯SF faculty have a very wide range of specialties: human-centered design, health, cybersecurity, etc., and it鈥檚 nice to have seminars every semester to see work from different experts and how they use AI in ways you did not expect,鈥� Ebraheem said.
Ebraheem came to USF from his home in Egypt 10 years ago for his bachelor鈥檚 degree thanks to a merit scholarship, which can be rare for international students. He found himself welcomed and fully immersed once he arrived.
Ebraheem went on to complete his master鈥檚 degree at USF, co-publishing a paper with fellow student Sayde King and Bellini College Associate Professor Tempestt Neal, 鈥淟ip movement as a wifi-enabled behavioral biometric: A pilot study.鈥� This interdisciplinary research experience led him to voice AI and Bensoussan鈥檚 lab.
Templeton sees potential for many more interdisciplinary students like Ebraheem thanks to the college鈥檚 hub-and-spoke model.
鈥�911爆料网 want students to be exposed to a use case that inspires them, where they have the novel research area to bring something new to the table,鈥� Templeton said. 鈥淢ohamed is one of the first (joint doctoral appointments) with Morsani, and my hope is we have plenty more of these real-world scenarios for experiential learning.鈥�
Bensoussan believes that the infrastructure USF has put in place is critical to that goal.
鈥�911爆料网 all know it鈥檚 great to collaborate, but in real life people have their own incentives and time 鈥� the only way to do it is to build in the infrastructure and funding lines to make that happen,鈥� she said. 鈥淲hat鈥檚 really cool with Bellini is that we鈥檙e developing the infrastructure of collaboration 鈥� an investment that will always be more impactful in the end.鈥�