backtop


Print E-mail del.icio.us 29 comment(s) - last by Screwballl.. on Sep 3 at 1:32 PM

Professors Shai Avidan and Ariel Shamir recently demonstrated a new image resizing technique titled "Content Aware Image Resizing".

Today a simple html tag can declare a text entry to be a percentage of the window's width or height, allowing the text to be elegantly wrapped on window resize.  Fonts are also easily adjustable in size via the control key and the scroll of a mouse button. 

While this has made browsing a much more pleasant experience, images unfortunately have been unable to scale successfully in modern browsers, due to the complexity of shrinking or enlarging an image, without losing data or creating artifacts.

Some image resizing techniques do exist, such as bilinear or bicubic filtering, but they tend to sharpen or smooth an image, and are computationally intensive.

Professor Shai Avidan, funded by Mitsubishi Electric Research Lab, and Ariel Shamir, working for and funded by the The Interdisciplinary Center & MERL, have created a promising new technique for image resizing.  They demonstrated their technique at San Francisco’s SIGGRAPH 2007 conference, which they have demonstrated in a video on YouTube.

The video demonstrates them employing their technique to easily resize images.  The technique involves using gradients to calculate a pixel path of least importance.  This pixel path selects one pixel from every row of pixels to remove.  By using gradients, the impact on the image's content is minimized, effectively.

Further, they demonstrate the shortcomings of their application, and how they have developed a resizing program utility to overcome them.  One such limitation is faces, which compress poorly and are prone to artifacting.  Their utility compensates for this by providing a protection brush that allows you to paint with a transparency brush over portions of the image you want to protect.  These areas are automatically rejected by the pixel path routes, so are not removed or resized.

Further, a side application of the technology demonstrated was the ability to use a different color transparency brush to designate certain areas to always remove.  Whether the user is a jilted lover, looking to remove their significant other from certain pictures, or the US or Chinese governments looking to remove images of their weapons, discretely, from the public eye, this technology may have many uses. 

While solutions like this have existed before, they typically look much more noticeable than the subtle, almost unnoticeable, removal that the "Content Aware Image Resizing" technique accomplishes.

For the paper about this technique titled "Seam-Carving for Content-Aware Image Resizing", or more of Dr. Shamir's diverse science and technology related research, please view the professor's website.

While the technique is likely computationally intensive, it may provide something to put tommorrow's quad core chip powerhouses to work, in creating more dynamic media content.

Hopefully, this technique will be coming soon to a browser or imaging suite near you.


Comments     Threshold


This article is over a month old, voting and posting comments is disabled

Could it be used in video encoding?
By Cogman on 8/31/2007 8:53:08 PM , Rating: 3
I could see something like this being VERY useful in a video encoding environment. Do you think it could possibly be used when the video is going to be stretched or skewed?




RE: Could it be used in video encoding?
By i4mt3hwin on 8/31/2007 8:59:48 PM , Rating: 3
I was reading a bit about the technology over at cgtalk.com, apparently it's extremely processor intensive. I would suspect that it's possible with video, but would require a lot of time and a lot of processing power.


RE: Could it be used in video encoding?
By PlasmaBomb on 9/1/2007 6:15:55 AM , Rating: 2
On video the algorithms would have to be applied to each frame to resize it, perhaps in the future it can be offloaded to the GPU?


RE: Could it be used in video encoding?
By i4mt3hwin on 9/1/2007 7:38:59 AM , Rating: 2
Another thing is, unless the algorithm can be applied to only frames or frame sections that remain the same visually in a shot it would artifact the image.


By eion on 9/1/2007 9:57:17 PM , Rating: 2
Certain aspects of video encoding, such as motion compensation, typically involve energy functions as I recall (although it has been a long time since I studied that stuff). I suspect that if you get the energy function right then artefacts of the kind you're talking about (although I'm not sure they're technically 'artefacts') won't be an issue; rather, resizing will simply reflect the changing composition of the scene, so that the technique will 'work' in the way it does with still images.


By LogicallyGenius on 9/2/2007 9:40:19 AM , Rating: 2
I am sure we can use the latest 64 core processor for the purpose ;-)


partial release
By Screwballl on 9/1/2007 12:53:55 PM , Rating: 2
Looks like someone already released a tool that can at least do this with vertical carving... whats to say you couldn't rotate it in your photo app program and then do it again... then rotate back to proper perspective...

http://www.thegedanken.com/retarget/default.asp?ln...




RE: partial release
By Rogie on 9/1/2007 8:41:11 PM , Rating: 2
That's pretty awesome stuff.
His bicubic filter could be a bit better though.


RE: partial release
By RaminS on 9/2/2007 4:41:18 PM , Rating: 2
Just finished implementing the height-carving. With the version 03 released soon (Win and Linux), it is possible to resize in both directions.

Also there is a new feature 'Animated Retarget' looks really nice :-)

Keep in mind not to open too big pictures for the beginning.

Regards
Ramin


RE: partial release
By RaminS on 9/2/2007 6:24:18 PM , Rating: 3
Version 03 released (win & Linux) http://www.thegedanken.com/retarget
Now Width and Height changes possible !

Have fun !


RE: partial release
By Screwballl on 9/3/2007 1:32:52 PM , Rating: 2
now we just need an official (or unofficial) 3rd party photoshop plugin or even GIMP plugin do do this.... which is why I assume the story states he was hired by Adobe...


Smells like fresh tech!
By therealnickdanger on 8/31/2007 2:51:40 PM , Rating: 2
Great article, Jason! His website is a cool read, although I think the traffic from DT is slowing it to a crawl. It took forever to grab that PDF. Innovations like this are bound to make web apps and general interactivity much more interesting.




RE: Smells like fresh tech!
By JasonMick (blog) on 8/31/2007 5:45:37 PM , Rating: 3
Thanks Nick!
Well, I hope I don't kill the poor guy's website =P I did notice it took me forever to download, too, which is why I linked to his homepage, instead.

He and the other researcher definitely deserve recognition for their work.

Being from a multidisciplinary computer science/chemistry/biology background, I really appreciate the variety to Professor Shamir's work. His image processing method alone is very exciting, but his website has everything from 3d modeling and imaging techniques, to molecular modeling and simulation. Really good stuff.

Wish I had time to read all of them!


RE: Smells like fresh tech!
By leidegre on 8/31/2007 8:03:33 PM , Rating: 2
I wonder if this has any practical usage in realtime rendering. I tend to hover around siggraph like a hawk looking for new ways to make realtime graphics more intresting, this is certainly some very useful and practical tech, but how would you use it outside web browsers? (those removal brushes sure was awsome)


RE: Smells like fresh tech!
By colonelclaw on 9/3/2007 8:18:57 AM , Rating: 3
does this mean i can easily remove the mother-in-law from the family photograph on my tft picture frame? best of all if she pops around unannounced one afternoon i can quickly restore her with the click of a button (or drag of the mouse)


Very cool, but...
By TP715 on 9/1/2007 1:19:12 PM , Rating: 2
the content dependent resizing appears to seriously change the content of the image. The woman holding the baby ends up having a freakishly large head. All of the images change content, as well as size, and no longer represent reality. I hope this wouldn't be used for news photos.

I think if web images I was viewing resized themselves this way, it would be a tad disturbing; I wouldn't know which zoom factor gave the "real" image. Basically the algorithm removes "dull stuff" from the image (i.e. areas with low derivatives), but that changes the spatial relationships between the "non dull stuff".




RE: Very cool, but...
By eion on 9/1/2007 10:02:31 PM , Rating: 3
If you watch the video, they solve the distortion problem by allowing certain areas to be 'prioritised'.

Conversely, they demonstrate how to remove specific parts of a picture (akin to airbrushing stuff out) by deprioritising them. It's pretty clever technology, to put it mildly.


Automation
By Tom Milhauz on 8/31/2007 12:52:44 PM , Rating: 3
It would be interesting if something like that could be implemented in cameras (or some simple software) to make automatic selections of content to hide first or what to protect, so it would be part of the header (or EXIF) of an image for simple use...




Co-inventor joins Adobe
By vijay333 on 8/31/2007 7:39:36 PM , Rating: 3
By Webreviews on 8/31/2007 11:35:50 PM , Rating: 2
This looks like some really exciting stuff.

Well worth watching the SIGGRAPH video.




holy molly
By Xajel on 9/1/2007 12:28:21 AM , Rating: 2
I was thinking about this method several years ago !!




fonts
By Lord Evermore on 9/2/2007 6:40:00 AM , Rating: 2
quote:
Fonts are also easily adjustable in size via the control key and the scroll of a mouse button.


Except when it doesn't work. Blame it on IE6 but it's still a PITA that something as simple as TEXT can be broken by adding complexity.




redraw
By Lord Evermore on 9/2/2007 6:55:57 AM , Rating: 2
The end result of this basically seems to be "redrawing" the image, as if you had a photo and drew it yourself on another sheet of paper at a different size. You'd focus on the important bits and make sure everything remains continuous and positioned the same but bits that can't be seen in the smaller size just aren't shown, and to make it bigger you fill in what's needed to maintain the flow.

I don't think I want that happening automatically, or at least not without a quick way to undo it when it comes to web images. Image hosting sites like to auto-shrink things to fit into a window, but then you just click the image to see the original.

Turning off the auto-resize in Internet Explorer is one of the first things I do on a new machine. Annoys the piss out of me.

If someone is editing an image and uses this, then that's fine, they're going to make it look proper, the way they want it to be seen before sending it off to the web or whatever. But if it just gets blanket applied to everything like on an image hosting site, then there will be chaos, inevitably a lot of images will come out looking like crap. Ultimately it seems to be better as a tool for a content creator.

And dear lord videos like that are dull.




eh...
By kevinkreiser on 9/3/2007 11:05:49 AM , Rating: 2
i was at SIGGRAPH for this paper's presentation and it seemed like an ok idea. it's a neat trick but other than that... all i can say is eh...




Revolution
By Egglick on 9/1/07, Rating: 0
Opera?
By DeepThought86 on 8/31/07, Rating: -1
RE: Opera?
By fehu on 8/31/2007 8:34:38 PM , Rating: 5
I'm not sure that you read more than the title of this article :P


RE: Opera?
By HaZaRd2K6 on 8/31/2007 11:11:42 PM , Rating: 2
"DeepThought86" seems inappropriate to me, as well.


"Intel is investing heavily (think gazillions of dollars and bazillions of engineering man hours) in resources to create an Intel host controllers spec in order to speed time to market of the USB 3.0 technology." -- Intel blogger Nick Knupffer














botimage
Copyright 2009 DailyTech LLC. - RSS Feed | Advertise | About Us | Ethics | FAQ | Terms, Conditions & Privacy Information | Kristopher Kubicki