The power of translation

For most Americans, communication is an oral endeavor. We learn to speak and read through sound, to distinguish between hard and soft  "k's," to make the hiss of a double "s" or the slight lisp of a "th."

A large chunk of the population, however, relies on their eyes to speak. These are the millions of people who use American Sign Language (ASL), a visual language built on movement, gesture and facial expression.

This difference between English and ASL--auditory versus visual--has implications for how the deaf and hearing communities interact. There's no translation app to help a deaf person navigate a doctor's visit, for example, or to aid a teacher trying to understand why a deaf student struggles to read.

That's where the Institute for Disabilities Research and Training Inc. (IDRT) comes in. With funding from the National Science Foundation (NSF), the Maryland-based small business has created translation software and assistive technologies to build bridges between English and ASL. And through a partnership between NSF and the U.S. Agency for International Development (USAID), they've adapted those technologies for use in Morocco, a country in desperate need of resources for deaf children.

"We do this to make life better for deaf people, and those who work with them," says Corinne Vinopol, president and CEO of IDRT and principal investigator on the NSF grants. "It's become clear to me that all this IT we've developed over the years can go out into the world and do some more good."

That technology includes software with an extensive translation database, which allows users to enter English words or sentences, and see images and video of how to express it in ASL. Think of it as Google Translate for sign language: Users can translate into both signs and fingerspelling, which spells English words with the ASL alphabet. The software also supports real-time ASL translation.

Much of that technology was developed with support from NSF's Small Business Innovation Research (SBIR) program, which catalyzes commercialization at startups and small businesses.

IDRT's current SBIR award includes research on gesture recognition technology through the AcceleGlove, a high-tech glove embedded with sensors. It works with 3-D camera technology to capture hand movements.

AcceleGlove has implications beyond ASL translation. It could replace a joystick to maneuver sensitive robotics--the kind that venture into dangerous environments or control heavy machinery. Or it could be adapted for artificial simulation, to help train medical technicians.

About five years ago, Vinopol was contacted by Abdelhadi Soudi, a computational linguistics professor at Morocco's Ecole National de l'Industrie Minerale. He'd found Vinopol's research and wondered if she would be interested in adapting that technology for Moroccan Arabic sign language.

"I really didn't know anything about Arabic when we started," Vinopol says. "I don't think he knew anything about sign language."

And yet their collaboration--and assistive technology developed by their team--has been so successful, Morocco's government is interested in using the technology in classrooms throughout the country.

Soudi and Vinopol received funding through Partnerships for Enhanced Engagement in Research (PEER), which links NSF-funded researchers in the U.S. with researchers in developing countries. USAID provides funding for the foreign scientist, and the ensuing collaboration benefits both countries.

Vinopol's research is the only SBIR-supported work to ever receive supplemental funding from NSF's Office of International Science and Engineering.

"The research promised international cooperation between the U.S. and Arab nations, at a time that couldn't be more important," says Glenn Larsen, a program director in NSF's Engineering Directorate, which funds the SBIR program. "We saw it as a great broader impact to handle the needs of deaf students both here and abroad."

A country in need

More than 85 percent of deaf children in Morocco lack access to education past primary school. The country has few well-trained deaf educators and has almost no sign language interpreters, which means deaf children are kept in segregated classrooms, with sparse instructional materials and little opportunity to interact with their hearing peers.

Soudi has spent the last 15 years working on machine translation between spoken languages--software to translate Arabic into French, for example. He was interested in the mechanics of translating a spoken language into a visual one.

"Translation between native spoken and sign languages involves not only analyzing linguistic differences, but also rendering translation from one cognitive processing modality (auditory) to another (visual)," he says.

It's not a word-for-word translation. For example, Vinopol's company previously helped WalMart use ASL translation for employee training, which included teaching people how to hang clothes on a rack. In English, "rack" is one word. In sign language, it depends on what the rack looks like: Is it thin or thick? High or low?

Soudi and Vinopol built technology that works as a real-time translation device and an instructional tool, converting standard Arabic into Moroccan Sign Language (MSL) and offering resources like games and quizzes to help students and parents learn MSL.

A second PEER award, received in 2013, supports the creation of an MSL thesaurus, which will allow users to describe signs (the right hand is making this shape, the left hand looks like this) and find the Arabic word equivalent.

To get this technology into the hands of schools, the team has traveled all over the country, met with over a dozen deaf associations and caught the attention of government ministries.

The robust intellectual collaboration between Vinopol and Soudi is a core criteria for international activities funded by NSF's international office, says Lara Campbell, a program director in that office.

"The unusual partnership between a small business and a foreign university brings a unique perspective to the table in terms of fundraising and structure," she says. "I think the business perspective may help the work of this project expand not just across Morocco but eventually across the region."

The most impressive results right now, however, may be how this technology affects deaf students and their families.

"Teachers, parents and students were positively astounded that software of this kind could be developed," Soudi says. "It generated hope and advocacy on the part of parents that there could be better education and higher expectations for their children."