DailyTech - Sign Language on Mobile Phones Could Help the Deaf

Submit News

Gadgets Sign Language on Mobile Phones Could Help the Deaf
Steve Kovsky (Blog) - February 16, 2007 9:48 AM

E-mail

del.icio.us

8 comment(s) - last by Justin Case.. on Feb 18 at 11:55 PM

MobileASL's skin detection algorithm in action.

Washington state researchers harness Windows Mobile Platform phones with open source video encoders to send sign language over the airwaves

While text messaging has helped give deaf people access to mobile communications, those who rely primarily on American sign language (ASL) have been left behind by cell phone technology. That could change, thanks to the MobileASL project at the University of Washington.

The MobileASL project seeks to use video compression to allow sign language communications over wireless phones. PDA phones with larger screens and built-in video capture capabilities have helped the effort toward its goal, but university researchers still face bandwidth constraints from today’s slow wireless networks. To produce the quality of video needed for intelligible ASL, they have had to invent a real time video compression scheme using a specialized H.264/AVC-based open source encoder called x264.

Officials with the project say that they have been able to almost double the compression ratios of MPEG-2, allowing them to transmit video that allows users to understand semantics of ASL, regardless of the bandwidth issues posed by existing wireless networks. The National Science Foundation-sponsored project relies on cell phones running the Windows Mobile Platform.

The MobileASL stretches the bandwidth even further by using motion and skin detection algorithms to focus in on the most important areas in the video – the hands and face. By concentrating on the portions of the image that contain skin pixels, the researchers found they could then encode those regions at higher rates than the rest of the image.

The MobileASL group is inviting a few of the more than one million deaf or hard of hearing Americans who are fluent in sign language to take part in an eye-tracking study to determine visual patterns in ASL conversations.

Comments

Threshold

Username
Password
remember me

This article is over a month old, voting and posting comments is disabled

Text?

By shoeish on 2/16/2007 1:03:26 PM , Rating: 4

What is wrong with text messaging?

RE: Text?

By Dactyl on 2/16/2007 1:17:15 PM , Rating: 2

Deaf people already do a great deal of texting.

The "Sidekick" phones are considered to be the cool ones.

This sign language by video sounds like a fun gimmick, but would use wasteful amounts of bandwidth and wouldn't be any more effective at sending a message. It would more of a human touch, that's all.

Parent

RE: Text?

By FijiJohn on 2/16/2007 1:21:46 PM , Rating: 2

Try not speaking all day and simply texting ALL of your vocalizations (for ALL conversations). You will quickly see why fluent speech is so much more efficient. ASL is a beautiful language (not simply English converted to signs) and to make people use English text instead of ASL is very limiting.

Parent

RE: Text?

By Sinbad on 2/16/07, Rating: 0

RE: Text?

By glenn8 on 2/16/2007 2:09:45 PM , Rating: 2

I guess signing is more universal than typing in English. And probably the people that are fluent in it can sign faster than they can type on the phone.
However, how would this work in practice? Would you have to put your phone down and make sure the camera is properly aiming at you? Seems rather impractical for a mobile device.

Parent

RE: Text?

By Justin Case on 2/18/2007 11:55:06 PM , Rating: 2

Sign language is different for each country so no, it's not more universal (if anything, it's less - some countries with the same spoken language use different sign languages).

Really, this is all very nice but it's not going to work, for several reasons. First, to let the phone's camera frame your upper body, you need to put the phone somewhere (on a tripod?), then stand back and sign. This is obviously not practical.

Second, to be even remotely expressive, it would have to use a ton of bandwidth which is expensive (and in most places simply isn't available). Lower the bandwidth and I'll end up looking like a slideshow. I'm sure it's "intelligible", as the article says, but it'll probably be less expressive than text.

People might play with it, but after a while I bet 99% will just go back to using text.

Even people that can hear and speak perfectly are using text messages more and more, these days.

Parent