The Doctor Can Understand You Now

By Eric Mankin on September 22, 2009 1:33 PM

In medical facilities around the country, care is delayed, complicated and even jeopardized because doctors and patients do not speak the same language - a situation particularly dire in diverse megacities such as Los Angeles and New York.

Now, USC computer scientists, communication specialists and health professionals hope to create a cheap, robust and effective speech-to-speech translation system for clinics, emergency rooms and ambulances.

The initial SpeechLinks system will translate between English and Spanish. Professor Shrikanth Narayanan, who directs the Signal Analysis and Interpretation Laboratory at the USC Viterbi School of Engineering, hopes to test and deliver a working prototype within the four-year window of a recently awarded $2.2 million National Science Foundation grant.

Narayanan, who holds appointments in the USC departments of electrical engineering, computer science, linguistics and psychology, will collaborate on the project with fellow engineering faculty member Panayiotis Georgiou and professor Margaret McLaughlin of the USC Annenberg School for Communication as well as researchers and clinicians from the Keck School of Medicine of USC.

The project will include AT&T and BBN Technologies investigators who will collaborate on the research and serve as mentors to the students.

According to Narayanan, success on the project will require a system that can perceive and interpret not just words, but a wide range of human communications. “We want to let people communicate,” he said. “We need to go beyond literal translation to rich expressions in speech and non-verbal cues. We want to enhance human communication capabilities.”

The additional cues to be analyzed and incorporated into the translation mix include:

• prosodic information: Spoken language uses word prominence, emphasis and contrast, plus intonational cues. Speech also divides subjects and thoughts in ways that are not always clear in a word-by-word literal translation.
• discourse information: Capturing contextual cues of dialogue becomes especially important since the target goal is to enable interpersonal interactions, in contrast to applications in which the end result is just translated text. The group plans to model and track the cross-lingual dialogue flow to improve the information transfer between the interlocutors.

Other elements of the mix include embedding sets of analyzed cues into speech that is synthesized from inputs keyboarded into the interface as a response or a question.

The emphasis is not hardware, but rather the creation of a system that can use existing, low-cost computers and other electronic equipment effectively for the purpose.

The overall strategy involves building on existing translation techniques by expanding the reach of tools used earlier to allow machine intelligences to perceive the prosodic and contextual cues.

For example, McLaughlin noted, while machine translation of text relied on analysis of masses of parallel written texts in the two languages, the new effort will compare and analyze bodies of oral text.

A key advantage for this approach is that unlike generalized conversation, doctor-patient interactions involve a limited, controlled context, making it easier to eliminate dead-end translation possibilities and false clues.

McLaughlin also emphasized that in addition to linguistic information, the effort will incorporate cultural cues and information. “Our system will not only be bilingual, but bicultural,” she said.

Besides Narayanan, McLaughlin and Georgiou, others working on the project include Lourdes Baezconde-Garbanati and Win May of the Keck School of Medicine; Vivek Sridhar, Prem Natarajan and Rohit Prasad of BBN and Srinivas Bangalore of AT&T.

TAGS: innovation, research

Search search

Search prior to August 2008