backtop


Print 34 comment(s) - last by ElFenix.. on Sep 13 at 5:29 PM


Photosynth composing a scene from many photographs

Complex algorithms analyze photos and stitch them together with amazing accuracy

Don't know what something is? Take a photo of it and have Photosynth find information for you
Turning photographs into 3D spaces and more

Imaging technology, in both physical media such as print and in digital have come a long way since the pencil and paper. Consequently, photos and online websites dedicated to hosting and displaying photos and images have also become extremely popular and in some cases made some revolutionary steps forward. Sites such as Flickr, DeviantArt and others are all contributing to moving digital imaging as well as digital photography forward. Interestingly, Microsoft also has something that's sure to stir up the hornet's nest in online and offline visual media.

Microsoft is currently hard at work on a project it calls Photosynth. Currently, the project is under the Microsoft Live development team, and no release date has been planned. However, what is known about Photosynth is that it will attempt to revolutionize the way people view their photo collections as well as change the way people take photos. Incorporating exciting new technology, Photosynth can take a collection of photos, analyze them, and model them into a 3D virtual scene, placing each photo in their perspective-correct position -- users can see where one photo was taken in relation to another photo!

The technology is indeed a big step forward over the traditional slide show. Say for example a photography enthusiast visited Las Vegas. Over the course of a few days, the user has taken several hundred photographs of the Las Vegas strip. With Photosynth, the user can catalog all the photos and build a virtual world out of them, modeling Las Vegas as he saw it. Photosynth allows users to move around in the 3D world, viewing scenes from any angle -- regardless of how the photos were taken. Photosynth's powerful algorithms do all the background work. According to Microsoft:

Each photo is processed by computer vision algorithms to extract hundreds of distinctive features, like the corner of a window frame or a doorhandle. Then, photos that share features are linked together in a web. When a feature’s found in multiple images, its 3D position can be calculated. It’s similar to depth perception—what your brain does to perceive the 3D positions of things in your field of view based on their images in both of your eyes.

Of course, Photosynth can perform traditional photo collection management duties too. Using its analysis engine, Photosynth also allows users to search their collection for "similar" photos, perhaps of a particular person or scene. The possibilities right now sound very exciting.

In February of this year, Microsoft acquired a company called Seadragon Software. Since then, Microsoft has been working Seadragon's imaging technologies into its own projects and Photosynth incorporates a good portion of them. According to Microsoft, Photosynth will allow users to zoom in, pan and fly through any photo or 3D scene without hiccups or pauses -- no matter if a photo is 1 megapixel or 1 gigapixel. Microsoft claims "scaling is near perfect and rapid for screens of any resolution."

Another very interesting and potentially powerful feature of Photosynth is its ability to "DNA" and recognize photos featuring distinct similarities. Microsoft claims that Photosynth will allow users to take a picture of an object, have Photosynth analyze it, and automatically find information about it online. For example, if a user was on a trip and took a picture of the Petronas Towers in Malaysia, they could import it and have Photosynth correctly identify the image as the Petronas Towers and automatically do an Internet search for information on the towers.

With Photosynth you can:

  • Construct a virtual 3D scene composed of several photos
  • Walk or fly through a scene to see photos from any angle
  • Seamlessly zoom in or out of a photograph whether it’s megapixels or gigapixels in size
  • See where pictures were taken in relation to one another
  • Find similar photos to the one you’re currently viewing
  • Find information on an object in a photo
  • Explore a custom tour
  • Send a collection to a friend

Clearly, Microsoft has some very large plans for Photosynth, and it's also clear that Photosynth incorporates some very interesting and powerful technologies. From the looks of it, it also appears that Microsoft is also planning to allow users to construct virtual scenes online, for other viewers to browse "fly" through and experience.

As of right now, Microsoft says that Photosynth will be arriving soon, and we hope so because we can't wait to try it.



Comments     Threshold


This article is over a month old, voting and posting comments is disabled

RE: Using for games/movies!-
By lemonadesoda on 9/5/2006 4:22:39 AM , Rating: 2
Odd thing for you to say if you are a pro 3D modeller:
quote:
they would simply have too many polys for use in a game
1./ Technology is moving toward ever more integration of APU/GPU units
2./ GPU horsepower breaks Moore's law, hence we only need to add another 2-3 years to get 10x the power, or another 5-6 years to get 100x the power
3./ Just as in any simple vector processing package, there are smoothing, or reducing, algorithms that reduce the overall vector count. Similar algorithms can be used for game manufacture.

The OP was correct in this obvious use. A game "map" can be created very very quickly, e.g. The White House, A village, A mountain range, then reduced, then edited for additional detail by hand.

It seems that the techology works well for solid masses viewed from their exterior. Any interiors (needed for 3d gaming environments) are another challenge...


RE: Using for games/movies!-
By pixelslave on 9/5/2006 10:10:05 AM , Rating: 2
We may have a GPU capable of doing that, but if we use "scanned" 3D objects in game, it will be poor usage of the GPU's power. We didn't even perfect 2D bitmap-to-vector conversion! Just try it on a medium level complex bitmap image and send it to a printer. A manually drawn vector almost always print faster. 3D object is similar -- just way more complicated.


RE: Using for games/movies!-
By JeffDM on 9/6/2006 12:50:59 AM , Rating: 2
I can see it working, but I can also see the value of manual intervention to do the final clean-up and tuning because it is a very performance-demanding market segment.

I've toyed with the idea of making software that simplifies a vector object by removing superflous objects and joining objects that are below a certain threshold. For example, the program could find two or more small line segments in a series that don't change much in angle and replace it with one larger line segment, simplifying the object. I've done this to some success with CNC engraving, but have not made a general form program that has an acceptable user interface and such.

The same concept can be applied to 3D surfaces too.


RE: Using for games/movies!-
By ElFenix on 9/13/2006 5:29:28 PM , Rating: 3
GPU power does not break moore's law. moore's law only applies to the maximum potential, not to designs that are behind the cutting edge. when nvidia was doubling and tripling power every 6 months they were not breaking moore's law because they were coming from a position of nothing. now that they've caught up to the cutting edge you've seen GPU releases slow.


"Game reviewers fought each other to write the most glowing coverage possible for the powerhouse Sony, MS systems. Reviewers flipped coins to see who would review the Nintendo Wii. The losers got stuck with the job." -- Andy Marken

Related Articles













botimage
Copyright 2014 DailyTech LLC. - RSS Feed | Advertise | About Us | Ethics | FAQ | Terms, Conditions & Privacy Information | Kristopher Kubicki