Robots Understand Our World

Hema Koppula and Abhishek Anand at Cornell University in Ithaca, New York, hope to avoid this disappointing scenario by teaching robots to understand the context of their surroundings so that they can pick out individual objects in a room. "We have developed an algorithm that learns to identify the objects in home and office scenes," explains Koppula.

Key to the system is Microsoft's Kinect sensor, which perceives real-world 3D scenes by combining two visible-light cameras with depth information from an infrared sensor. Koppula and Anand's algorithm learns to recognise particular objects by studying images labelled with descriptive tags such as "windows", "books" and "tables".

The researchers used 27 labels in total, 10 each for office and home scenes and seven that applied to both.

Previous approaches to the problem have used expensive depth sensors that couldn't provide colour information, but the cheap Kinect can do both, allowing the algorithm to consider both shape and colour when evaluating an object. The system also takes relative locations into account – for example, computer monitors are normally found on top of a table, rather than underneath.

It turns out office locations are easier to classify than home scenes, with Koppula and Anand's algorithm achieving 84 per cent recognition success in the former versus 74 per cent in the latter. Koppula puts this down to the lack of variety in office environments compared with our more personalised homes. This way “they” will be able to help even replace our job someday,

Posted in: Tech