I can see objects

September 10, 2009

I haven’t been able to blog for some time, but as you might know the ICRA ’10 deadline is approaching fast and I have been very busy. The pressure of the deadline has been very useful, as I got a bunch of new capabilities: I can do actions with secondary movements and natural-looking grasps, and I can learn tasks that have hybrid representations from a human teacher’s demonstrations. Well, almost…


I will tell you more about these later, once my papers are submitted.

Today I wanted to tell you about something I was quite excited about: the new object perception system that I use for learning tasks from demonstrations. It’s basically doing classification on objects detected with background subtraction. The background model is kept constant since the camera at the top of my workspace is not moving. Also the background of my table is black on purpose, so that I don’t have to deal with annoying shadows. Then for each foreground blob I extract a set of features that describe the object: (1) number of corners of the approximated polygon of the blob’s contour, (2) area of the blob, (3) whether the contour is convex or concave, (4) the color histogram of the blob, (5) height, width, eccentricity (height/width) and angle of the ellipse fitted onto the blob, (6) whether there’s a hole (background) on the blob, and (7) average brightness (V in HSV) of the blob. First I need a grad student to label some objects for me by placing a single object on my table. Using these labeled descriptors I train a Naive Bayes classifier, and then I can recognize new objects based on their descriptors. It’s very simple and effective. In my first experience I was trained to recognize objects like books, bins, markers, boxes of different shapes, markers, etc. from about 8 samples per class.

Secondly, I was trained to recognize different states of the same object, like open/closed books or full/empty tissue boxes. And only with one sample per class, I think that’s amazing. On this one I get more confused though. Sometimes the same object goes back and forth between being a book and a bin, that doesn’t seem right. So I will need more labels before I can learn tasks properly. Also next time I should ask to get some labels for the null class. Right now I would recognize any junk that you put on my table as something. Some grad students find it funny that I call their hands a bin or a tissue box.

Anyways, I should get back to working for the ICRA deadline. Hopefully some of my grad student friends get to go to Alaska for this and they can take pictures of an iceberg for me. I’ll train my vision system with some images from Google to recognize that.

Leave a Reply