Software Automatically Analyzes and Tags Images Based on Content

http://www.cbc.ca/technology/story/2006/11/02/tech-imagetag-061101.html
from WinCustomize Forums

Researchers at Penn State University in the College of Information Sciences & Technology have developed software that can automatically recognize image content and properly "tag" it in plain English.  This would present a significant improvement to image search engines that rely on manually entered tags to describe image content.  The article linked below uses an example of a photo of a polo match being tagged by the system with words such as "sport", "people", "horse", and "polo."

The system is called the Automatic Linguistic Indexing of Pictures Real-Time (ALIPR) and it currently has a vocabulary of 332 words to describe what it detects in an image.  The project is headed by associate professors James Wang and Jia Li.

The system is "trained" by inputting vast numbers of pictures, and then telling the program which automatically selected tags are correct.  The idea is that as more images are uploaded and more tags identified, the more accurate it becomes.  Currently, more than half the time, the first tag the computer selects out of 15 is correct.  The analysis takes about 1.4 seconds per image and has a 98% success rate in identifying at least one correct tag.

If you're interested in trying the system out for yourself, they've opened up the project on Dr. Wang's website so you can help improve the system by uploading your own images and selecting the correct tags.  http://www.alipr.com/

For further information on Dr. Wang's image analysis research, check here: http://wang.ist.psu.edu/IMAGE/

This technology is very promising for companies like Google who are attempting to improve their image processing and search capabilities.

3,849 views 4 replies
Reply #1 from WinCustomize Forums Top
That's an interesting idea. I went and uploaded a few images and then it refused to add more. Not sure if they limit your contribution or if thier servers are just getting swamped
Reply #2 from WinCustomize Forums Top
It's a great idea, but I think they're going to need a few more words.
Siege of Centauri
Reply #3 from WinCustomize Forums Top
It's a great idea, but I think they're going to need a few more words.


I was thinking that also. Another thing that comes to mind is, different individuals could upload the exact same photo with different descriptors. You could add a photo of a waterfall, tag it man-made, horizontal, orange, how will the system know any different?

Reply #4 from WinCustomize Forums Top
The idea is that given enough images and enough tags identified, the system will be able to accurately identify images. People trying to fake-out the system with bogus tags would be an anomalous blip on the radar and wouldn't impact the tagging. And I'm sure more words will be added to the program's vocabulary, this is still in "proof of concept" mode. I'm sure it'll be much more robust in a year's time. Remember that it's still a research project. As a matter of disclosure, I'm a graduate from PSU, from the College of Info Sciences & Tech I didn't have any classes with Dr. Wang and didn't participate in any way with this project... wish I had though, it's kinda neat.