Print 45 comment(s) - last by The Raven.. on Dec 14 at 10:31 AM

For using off-the-shelf components, MIT researcher Garratt Gallagher's hand-driven interface is quite impressive.  (Source: MIT-CSAIL via YouTube)

The system uses the Xbox 360's shiny new Kinect accessory and open-source libraries.  (Source: Engadget)
Now why didn't Microsoft include THIS with its Kinect bundle

It's frustrating to always watch films like The Minority Report and Iron Man II where characters zip through their ultra-productive days, leveraging exotic interfaces that perform the toughest tasks and ease.  And they even look cool using these visualizations to boot.  But when the movie light fades, you're off to your car with its boring old dash display and probably will greet work the next day in front of the same boring old UI.

Fortunately, the brains at Massachusetts Institute of Technology's Computer Science and Artificial Intelligence Lab (MIT-CSAIL, for short) have created [video] a real life exotic interface that uses a gaming console that you might find in your family den or living room.

Using ROS ("an open-source, meta-operating system for your robot") and libfreenet ("a lower-level API for sending and receiving messages over a Freenet connection, and for reading and writing raw data streams"), and an off-the-shelf Xbox 360 with Microsoft Kinect, MIT researcher Garratt Gallagher has created a system similar to 
The Minority Report's hand driven GUI.

The Microsoft Kinect can "see" the position of your fingers and palms, allowing you to control your floating GUI with ease.  You can resize images, move them around, spin them, delete them, scroll down a list of images, or relegate images to a special tray in the demonstration.

There's probably a few questions on your mind now.  First, why didn't Microsoft think this up in the first place?  Well, give the folks at Microsoft some credit for designing as impressive a piece of hardware as the Kinect -- that made life for Mr. Gallagher a whole lot easier.  But hopefully Microsoft does take note and decides to scoop up this idea and add it to the default Kinect bundle.

Your second question might be, how did this MIT researcher come up with such a intriguing GUI?  Well his creation is perhaps less than surprising, given that the design team for 
The Minority Report and other typical futuristic flicks are full of MIT alumnis.  For example The Minority Report leveraged the talents of MIT Media Lab alumnus John Underkoffler and former MIT architecture dean William J. Mitchell.  

Comments     Threshold

This article is over a month old, voting and posting comments is disabled

alumni is plural already
By Sifl on 12/9/2010 12:07:11 PM , Rating: 3
full of MIT alumnis.

should say "full of MIT alumni" or possibly "alumnae" if they are only women.

quick Latin:
alumnus = male, singular
alumna = female, singular
alumni = plural (either all male or mixed)
alumnae = plural, only female, no males allowed

RE: alumni is plural already
By ebakke on 12/9/10, Rating: 0
RE: alumni is plural already
By Sifl on 12/9/2010 12:42:14 PM , Rating: 5
you are correct, "no," most people do not need to know or understand Latin. However, if you write papers for school, or use it (or any other foreign language) in print or online, it would help to know some of the basics.

it is kinda cool though to go to all the really old churches in Europe where all the inscriptions are in Latin and be able to read them yourself. But I digress...

RE: alumni is plural already
By foolsgambit11 on 12/9/2010 3:32:15 PM , Rating: 2
You're absolutely right. Tempora mutantur, nos et mutamur in illis....

RE: alumni is plural already
By Sifl on 12/10/2010 12:30:00 PM , Rating: 2
ooh, that's intense. can't get the meaning of the 2nd half. 1st bit is something like "if time changes" or something like that?

RE: alumni is plural already
By freaqie on 12/11/2010 11:56:21 AM , Rating: 2
Times are changing, and we are changing with them.

freely translated, by my old long forgotten latin...
so I might be wrong,

anyway, nice piece of research,
there is a video on youtube on it, and it does still seem a bit slow and chunky, then again who can expect it to be really fluid so short after the Tech has been lanched and "jailbroken"

RE: alumni is plural already
By The Raven on 12/14/2010 10:31:51 AM , Rating: 2
I think he was trying to make an ironic comment. Like saying Latin is nearly dead and so that's why people don't get why it is good to know.

The "times change" part speaks to the fact that Latin is not a living language. The "we are changing with them" part is speaking to the fact that the first respondant to the OP thinks it is near useless to know Latin.

I don't know... it's all Greek to me!

RE: alumni is plural already
By ClownPuncher on 12/9/2010 1:09:21 PM , Rating: 2
A good chunk of English is based on Latin, so I would hope so.

RE: alumni is plural already
By FITCamaro on 12/10/2010 7:23:23 AM , Rating: 2
Wat r u talkin bout?

RE: alumni is plural already
By Alexvrb on 12/12/2010 2:04:09 PM , Rating: 2

RE: alumni is plural already
By EricMartello on 12/10/2010 9:31:33 PM , Rating: 2
Just like a movie "based on true events" does not mean the content of said movie is true, a language "based on Latin" does not mean that knowing Latin would improve your understanding of English. Spanish is also based on Latin but it sounds quite a bit different from English doesn't it? Languages evolve over time and English is no different...and quite frankly it's better to know where the language is headed rather than where it came from, at least then you won't be asking "What is mean 'she binged me'"?

RE: alumni is plural already
By Murloc on 12/10/2010 7:48:30 AM , Rating: 2
I agree, but if you use latin words you're supposed to write them correctly.

RE: alumni is plural already
By Fritzr on 12/10/2010 7:28:23 PM , Rating: 2
Does that mean we should also spell English words like colour and labour or French words like centre correctly also?

Just wondering...

RE: alumni is plural already
By Xaussie on 12/9/2010 1:47:01 PM , Rating: 3
There's probably a few questions on your mind now.

Expecting correct Latin is hard when people don't even write correct English. This should have been "There are probably a few questions on your mind now." Questions is plural so takes the plural form of the verb (are) not the singular (is).

RE: alumni is plural already
By hyvonen on 12/10/2010 11:52:35 AM , Rating: 3
Whoever says DailyTech has an actual Editor is a liar.

RE: alumni is plural already
By Belard on 12/9/2010 9:10:25 PM , Rating: 2
I don't know latin at all, okay a few here and there... but even I thought that the article looked wrong, "alumni" is far more well known.

RE: alumni is plural already
By FITCamaro on 12/10/2010 7:22:46 AM , Rating: 2
I noticed this.

leveraging exotic interfaces that perform the toughest tasks and ease.

Uniformed Journalists Annoy Me
By zephyrxero on 12/9/2010 12:09:30 PM , Rating: 5
Well, give the folks at Microsoft some credit for designing as impressive a piece of hardware as the Kinect

Microsoft didn't create Kinect, they bought a company that had already come up with the technology!

By chagrinnin on 12/9/2010 1:22:14 PM , Rating: 2
Uniforms are okay for day to day jobs 'cause you don't have to decide what you're going to wear everyday. Einstein wore...the same....oh, wait...never mind. :P

RE: Uniformed Journalists Annoy Me
By Smilin on 12/9/2010 3:50:27 PM , Rating: 2
The primesense sensor is only part of Kinect and Microsoft did not buy that company.

RE: Uniformed Journalists Annoy Me
By The Raven on 12/9/2010 4:14:41 PM , Rating: 2
Umm... impressive on your heartbeat (which wouldn't be a bad thing for most people). It is impressive as a piece of video game exercise equipment, but this Minority Report mumbo jumbo will never be mainstream as a replacement for a mouse since it is easier to use a mouse clicking buttons than it is having to wave your arms around to check your mail.

It looks cool and what not, but it is impressive like the Segway. Or this...

Near useless technology for most of us in my opinion. And yes, that is just my opinion. The thing seems to be selling well so other's must disagree.

RE: Uniformed Journalists Annoy Me
By FITCamaro on 12/10/2010 7:28:24 AM , Rating: 2
Oh come on. I'd map closing IM windows at work to flipping it off.

But yeah there really isn't a simpler, cheaper way to implement the basic functionality of a mouse. Sure you could do touch screens, but then you have fingerprints all over your display. Until machine-brain interfaces are perfected to where you can think to move a mouse and type just as fast as you can move/type, then the keyboard and mouse are here to stay. And even then, how do you filter out the million random thoughts going through a brain when typing and what you actually want to type?

Even in Star Trek they still use touch displays instead of direct neural control. I think as humans, we are far more comfortable interacting with something through touch to generate a result than just sitting there thinking it.

RE: Uniformed Journalists Annoy Me
By The Raven on 12/14/2010 10:24:48 AM , Rating: 2

Touchscreens are still less friendly than mice on the desktop because instead holding your arm up for hours to navigate a 17" touchscreen area, you can use like 8" of space with a mouse and you don't have to hold your arm (/s. That's right. I'm looking at you multitouch displays...) up all day.

In general, touch is only good for small handheld screens.

RE: Neural control, I always think of the chaos that could result with a stray thought. Take this bit from Kyle Dunnigan as an example...

RE: Uniformed Journalists Annoy Me
By Fritzr on 12/10/2010 7:33:17 PM , Rating: 2
There is a major drawback to the motion based display.

You're working on that important report and just as you are adding the final changes your Italian coworker stops to say a few words...the gestures he uses to accent his speech, closes and erases your report, then activates a secure drive erase ... oopsy

Mice don't do that kind of thing...yet :P

RE: Uniformed Journalists Annoy Me
By ekv on 12/11/2010 3:03:35 AM , Rating: 2
Gestures vs. speech.

Gestures could require face recognition before being allowed to execute.

Speech could require speaker identification. Of course, that's an extremely difficult problem. There is the microphone array ... if it has sufficient spatial resolution then we can determine that the speech commands were coming from a non-recognized/non-authorized source.

Lastly, even if you use a mouse to erase your report you can "undo" it. If you try to close your report (w/o saving) via mouse interface then there is still a "Are you sure?" dialog. It'd be reasonable to expect something similar for speech. And why the heck didn't you speak up and say "NOOOOOO!" [Your name isn't Adam is it? 8]

Games, Maybe. Work, Never.
By DaveLessnau on 12/9/2010 11:33:18 AM , Rating: 2
Using this kind of UI (manipulating objects by holding your arms out in front of you and moving them about) might be workable for games. But, unless you want to cripple yourself within a couple of hours, it's a ridiculous thing to suggest for any production environment. People have carpal tunnel problems now. Imagine the stress on your arms, shoulders, and back this kind of interface would cause.

RE: Games, Maybe. Work, Never.
By zmatt on 12/9/2010 3:46:17 PM , Rating: 2
I disagree, before the computer age we sued our arms much more in daily activities. it may at first be tiring to people used to typing all day, but many people use their arms all day in their job and have no problem. If anything, it is healthier because you are more active.

RE: Games, Maybe. Work, Never.
By Akrovah on 12/9/2010 4:21:46 PM , Rating: 2
Hehe..."we sued our arms"


RE: Games, Maybe. Work, Never.
By ekv on 12/9/2010 10:35:09 PM , Rating: 2
Good ol' gorilla arm. From the days of yore and light-pen interfaces. I believe the article, as well as the video, mentions something about recognizing hands and fingertips.

Don't forget the microphone "array".

By lainofthewired on 12/9/2010 11:56:30 AM , Rating: 3
And this is why Kinect > Move.

RE: Competition
By BansheeX on 12/12/2010 5:02:27 PM , Rating: 2
100% motion sensing is going to be frustrating for traditional fare. Nobody wants to be forced to use their body for movement all the time, nor can the turning be as precise as a controller.

probably in conference on it right now!
By atlmann10 on 12/9/2010 11:28:41 AM , Rating: 2
As to Microsoft not including this in the initial package, see my subject :)

By Smilin on 12/9/2010 3:56:42 PM , Rating: 2
I'm pretty sure that conference ended a long time ago. Project Natal did not turn into the Kinect. Kinect is just a fork of NUI

(Natal User Interface...which is kind of a crappy acronym since it conflicts with Natural User Interface...which of course is what NUI..meaning

Ok that probably was confusing. Point: Microsoft had the "minority report" discussion probably a couple years back. Kinect got everyone thinking about it, but Microsoft had Kinect years before everyone else.

where are we headed?
By AntiM on 12/9/10, Rating: 0
RE: where are we headed?
By zephyrxero on 12/9/2010 12:24:49 PM , Rating: 2
Don't worry....if we have a Matrix/Dalek future in store, it won't be for hundreds if not thousands of years...evolution is a slow beast ;)

RE: where are we headed?
By bigboxes on 12/9/2010 2:38:09 PM , Rating: 2
Great... now I won't get anything accomplished. Nothing but nekkid wimmen on my screen!

By GuinnessKMF on 12/9/2010 12:19:42 PM , Rating: 2
Their usage of the system is actually a little underwhelming, the NUI group has been doing things like this put to much cooler applications. As for why didn't Microsoft think of it, they've been working on practical Natural User Interfaces for some time, what has come out of the Surface team is simply amazing.

Not to crap on this too much I'm sure they will put this to good use (something better than just image browsing), and we'll see some really cool stuff in the future.

RE: Usage
By Fritzr on 12/13/2010 10:38:01 PM , Rating: 2
What makes this of great interest to the nerdy DIY community is that it was done using hardware sold at your local department store. There is no longer a need to be an electronics hobbyist to play with these controllers.

the lag time due to deciphering the gesture is something that people could get used to. Mice were unnatural and hard to get used to in the beginning also...nowadays people who don't remember the premouse days often believe that you cannot use a computer if the mouse doesn't work ... that may actually be true for some programs, but that was a conscious design decision :P

no lag?
By superPC on 12/9/2010 1:02:35 PM , Rating: 2
i looked at the video and notice no lag at all. the interface is buttery smooth. kinect worked flawlessly. why then the game shows noticeable lag? and it can track fingers with great precision too.

this kind of interface would be great for working on 3D model. rotating view, zoom, pan, and stuff.

RE: no lag?
By BansheeX on 12/12/2010 4:34:36 PM , Rating: 2
All motion sensing devices have major delay, about 250ms for Kinect. A wireless mouse with that kind of cursor delay would make you gouge your eyeballs out.

By wolrah on 12/9/2010 2:51:19 PM , Rating: 4
Using ROS ("an open-source, meta-operating system for your robot") and libfreenet ("a lower-level API for sending and receiving messages over a Freenet connection, and for reading and writing raw data streams"), and an off-the-shelf Xbox 360 with Microsoft Kinect, MIT researcher Garratt Gallagher has created a system similar to The Minority Report's hand driven GUI.

They used libfreenect, a library for interacting with the Kinect available from, not libfreenet, a library for communicating on the Freenet anonymous network.

It's even in literally the first five seconds of the video.

Had it been a simple typo it would be easily excusable, but obviously you went to the libfreenet site, read enough of the description to copy/paste, and still didn't have any mental red flags.

By Jason H on 12/9/2010 5:59:01 PM , Rating: 3
The name of the movie is "Minority Report".

3D virtual keyboard
By Shadowmaster625 on 12/10/2010 10:06:23 AM , Rating: 2
Do they have a 3D virtual keyboard, or maybe we can dream up something a little more modern?

cool concept - thx for info
By Sifl on 12/10/2010 12:34:46 PM , Rating: 2
forgot to mention this is trey cool. thx for sharing Jason !

kinda cool watching super smart creative people take 1 core element and create something totally different. even if it's not immediately useful, it still shows more things can be done with what is sold as just a "gaming platform"

By nstott on 12/9/2010 11:37:34 AM , Rating: 1
I love/hate/miss that place/paradise...

"Well, we didn't have anyone in line that got shot waiting for our system." -- Nintendo of America Vice President Perrin Kaplan
Related Articles

Latest Headlines
Inspiron Laptops & 2-in-1 PCs
September 25, 2016, 9:00 AM
The Samsung Galaxy S7
September 14, 2016, 6:00 AM
Apple Watch 2 – Coming September 7th
September 3, 2016, 6:30 AM
Apple says “See you on the 7th.”
September 1, 2016, 6:30 AM

Most Popular ArticlesSmartphone Screen Protectors – What To Look For
September 21, 2016, 9:33 AM
UN Meeting to Tackle Antimicrobial Resistance
September 21, 2016, 9:52 AM
Walmart may get "Robot Shopping Carts?"
September 17, 2016, 6:01 AM
5 Cases for iPhone 7 and 7 iPhone Plus
September 18, 2016, 10:08 AM
Update: Problem-Free Galaxy Note7s CPSC Approved
September 22, 2016, 5:30 AM

Copyright 2016 DailyTech LLC. - RSS Feed | Advertise | About Us | Ethics | FAQ | Terms, Conditions & Privacy Information | Kristopher Kubicki