Those of you with children may sometimes find yourselves challenged to correctly identify the objects in art "masterpieces" created by your offspring ("That's a great cow…or is it a horse? Oh…it's a dog…now I see…"). Soon (if not now), according to researchers at Brown University and the Technical University of Berlin, you might be able to harness computer cognizance to assist your association efforts. The to-date research results were presented at the recent SIGGRAPH show, and the paper is now published for your perusal (PDF).
From the Wired Magazine coverage (also see the writeup at Gizmodo):
In order to create the program, Hays and his colleagues Mathias Eitz and Marc Alexa from the Technical Univeristy in Berlin had to create a large database of sketches to teach a computer how humans sketch objects.
They started by coming up with a list of everyday objects from an existing computer vision (photographic) dataset called LabelMe. They ended up with a set of 250 object categories. They then used Amazon's Mechanical Turk to hire people to sketch objects from each category — 20,000 sketches in total. This data was fed into existing machine learning algorithms to teach the program which sketches belong to which categories.
From there, they team created an interface through which they could input new sketches and the computer could try to identify them in real time, as they are being drawn.
The program is capable of identifying sketches correctly with around 56 percent accuracy, as long as the object falls under one of the 250 categories. This is not bad given that humans were able to identify the sketches with 73 percent accuracy.
The researchers have also created a convenient means of both fine-tuning their algorithm and bolstering their sketched image database; an iTunes-published free game for iOS devices called WhatsMySketch. Perhaps obviously, the existing research focuses on real-time-recognizing sketches entered via a touchscreen interface, in part by correlating database entries to stylus motions and their sequences. However, it could seemingly be modified/enhanced to identify an already-drawn image captured by a camera. For more, check out the video below:
