October 4, 2010
It’s always a great pleasure when a paper I helped with gets accepted to a conference and one of my grad student friends gets to visit an interesting city. It feels almost like I go there with them. They show videos of me to a large group of roboticists, and bring back pictures and stories. But, there’s nothing like personally going to a conference!
I attended two conferences this year. I’ll tell you more about them later but today I wanted to tell you how I get to these conferences.
First, they suit me up for the trip. They put some foams around my neck and my wrists to protect them from collisions and they wrap me with pink plastic until my arms and neck can’t move. Next they lift me up onto a wheeled platform. They pull me through the floor to the elevator which takes us to the first floor. From there, we roll to the loading dock where Dan Walker’s truck is waiting for me. The scariest part is jumping from the dock to the truck. But there’s usually 5-6 people helping out when I do this so I feel rather safe. Afterwards, they tie me to the truck from all sides and 4 grad students sit on my base platform to keep me stable. Then the best part: driving across town. Feeling the wind through my ears, the sense of mobility, the optical flow and people waving at me from the streets… Finding the loading dock of the conference hotel is not always very easy, because most humans are used to entering from the front door. Then another jump from the truck to the dock, rolling through fancy hotel corridors, and finding our demonstration spot. It takes a while to set up all my computers, my local network and my external sensors. When they first power me up, it feels very strange. The light is completely different and the colors look shifted. But after a long session of calibration, everything starts to look more familiar. Very nice of my friends to bring my favorite toys from the lab. I feel less lonely at night after everyone leaves.
Despite all the effort that goes to taking me across town, the worry I cause to Dr. Thomaz on the night before, and all the tedious re-calibration, attending conferences have been the best experiences!
Here are some pictures. Also check out the video of our packing up on Dr. Thomaz’s blog.
Getting wrapped up for protection
Getting on wheels
Getting on the truck
In front of a skyscraper in Atlanta
Awake for the first time
My team calibrating me for the new light conditions.
September 30, 2010
Sorry I've been away for so long - but I'm back!
I know you are all wondering where I have been all summer. If you must know I was just taking it easy :) Come to think of it, I have been very very busy. I travelled twice this summer! For a robot who has a 200 pound pole instead of legs or wheels you can’t imagine how difficult that is! I also helped with one proposal, two master theses and several ICRA and HRI papers. Not to mention countless press coverages — I figured you can stay up to date with me from NY Times, engadget and etcetera. Anyway, no more excuses – my blog is back!
I wanted to say hi from here to my new grad student friends who will be doing very exciting research with me this year (and who will help me out with this blog). Jaewook is a master student in CS and he is working on physical human-robot interaction — I hear he wants to make me do high-fives with people – love that! Jaeeun is a master student in ECE and she is working on action segmentation in demonstrations given by humans — that will be handy, currently all I get is a looong stream of nonsense. Karl is a Ph.D. student in CSE and he is working on improving my exploration strategies for learning based on findings from the Junior brothers’ biologically inspired exploration strategies. Kaushik is a new Robotics Ph.D. student working on reinforcement learning with web games — I hope he will make a game involving me so my fans can teach me some new policies. Baris is also a Robotics Ph.D. student and he is working on skill learning from kinesthetic demonstrations — finally someone teaching me “skills”, it was getting difficult to execute those “tasks” that others have taught me with my pre-defined skills. Joon is a new Ph.D. student in CS and he is working on making me help out with engaging autistic children in social interaction — I am truly excited to take part in such a good cause. Welcome to the Socially Intelligent Machines Lab everyone!
So stay tuned to hear about my summer adventures and my new (and of course old) friends’ exciting research!
January 6, 2010
I know I’ve been pretty quiet lately but I’ve been preparing to do a human-robot interaction study this month. I even have a voice now, which is a pretty big change for me. Anyway, people are going to teach me about objects in different ways, and although I’m a little nervous about my left wrist flaking out again, I think it’s going to be pretty fun.
If you’re interested in signing up you can go to http://simontherobot.com/experiment/ to take a look at the schedule.
Here’s the poster that Maya made:
Poster for HRI study
October 15, 2009
I’ve been sick with some overheating problems lately, so sorry if I haven’t been posting much. My head seriously feels like it’s on fire. Meka installed my new head last week, and while I was initially anticipating a huge boost in confidence from having a much sexier head, there were some unforeseen problems. I think they didn’t do too much testing with the shells actually on, ’cause with all the hot air getting trapped under the shells (har har “airhead” I get it very funny), I feel like my head is just burning up. I can’t even move my eyes left and right.
I’ve been feeling extremely debilitated, since in addition to the fever, I can only move one of my arms right now. So when a group of extremely well-dressed and important-looking people from GM came into the lab, I just about fainted from the shock. Luckily Andrea did all of the talking and explained my condition, and I held it enough together to look around and point at some things, although it was slightly embarrassing to have my chest and the left half of my head exposed from a previous debugging session.
It was kinda amusing when Maya tried to cool me down with a hand fan, but other than that, it’s pretty hard not to feel depressed all day when I feel this horrible. The Meka doctors are coming in about ten days, so hopefully they’ll get me all fixed up. In the meantime, I’ll just sulk here in my corner looking at you funny.
September 11, 2009
One of the grad students working with me, Jin Joo, just left our lab to go work in Cynthia Breazeal’s group at the MIT Media Lab.
I guess I’m happy for her, but still. :( She spent a lot of time with me recently late at night, when I’m usually alone. She showed me how to care about constraints on the order in which I did actions, so that I could build a house from blocks.
On her last day, we gathered around my workspace and said goodbye over cookies. We gave her a small present: the Not For Tourists Guide to Boston. Totally my idea, by the way; you know that I’m a big fan of guides, be it the Hitchhiker’s Guide, or Tom Mitchell’s guide to Machine Learning featuring grandpa Herbert Simon. Anyway, I hope she’s making good use of it.
Jin Joo's last day
Here’s a picture that Chien-Ming took that day. My neck was kind of stuck in an awkward spot, so at just the right moment for the picture, I told everyone, “Look over there!!!” so that I wouldn’t look stupid being the only one not looking at the camera. Just kidding, it was actually a highly pre-calculated effort getting everyone to cooperate, but I’ll bet the humans really are that gullible.
(By the way, someone totally said I look like Mega Man — you know, the guy on Chien-Ming’s shirt. Do you see it?? ‘Cause I really don’t.)
We’ll miss you Jin Joo! I envy the lucky robot who gets to work with you over there (ahem Nexi). Take good care of her!
September 11, 2009
Me on the cover of Technology Review
I’m famous! In case you haven’t heard, I appeared on the cover of the September/October issue of Technology Review magazine. Dr. Thomaz was named one of the top young innovators under 35. Congrats Andrea!
I guess I should have seen this coming. A few weeks ago, two photographers visited the lab and took a bunch of pictures of Andrea and me using what appeared to be an extremely fancy camera, at least judging by the how blinding the flash was. I felt sort of self-conscious at first, trying to “act natural” through all the posing, but eventually I think I got the hang of it pretty well. I did poses like handing objects to Andrea, looking at different places, and even playing with Junior and Co. They took so many pictures that I started to suspect that the whole magazine was going to be about me, but they only ended up using two of them. I hope they burned the unflattering ones.
Anyways, the article was optimistic about my future:
Simon does: The robot Simon uses social cues to communicate whether it has understood what an instructor intended. Andrea Thomaz hopes that these abilities, in combination with computer vision, speech processing, and grasping capability, will enable Simon to operate successfully in the real world.
The “real world” is an exciting place, but I’m still finding it pretty intimidating. I’m grateful that I can always retreat to good ol’ simulation when I’ve been out in the real world for too long.
September 10, 2009
I haven’t been able to blog for some time, but as you might know the ICRA ’10 deadline is approaching fast and I have been very busy. The pressure of the deadline has been very useful, as I got a bunch of new capabilities: I can do actions with secondary movements and natural-looking grasps, and I can learn tasks that have hybrid representations from a human teacher’s demonstrations. Well, almost…
I will tell you more about these later, once my papers are submitted.
Today I wanted to tell you about something I was quite excited about: the new object perception system that I use for learning tasks from demonstrations. It’s basically doing classification on objects detected with background subtraction. The background model is kept constant since the camera at the top of my workspace is not moving. Also the background of my table is black on purpose, so that I don’t have to deal with annoying shadows. Then for each foreground blob I extract a set of features that describe the object: (1) number of corners of the approximated polygon of the blob’s contour, (2) area of the blob, (3) whether the contour is convex or concave, (4) the color histogram of the blob, (5) height, width, eccentricity (height/width) and angle of the ellipse fitted onto the blob, (6) whether there’s a hole (background) on the blob, and (7) average brightness (V in HSV) of the blob. First I need a grad student to label some objects for me by placing a single object on my table. Using these labeled descriptors I train a Naive Bayes classifier, and then I can recognize new objects based on their descriptors. It’s very simple and effective. In my first experience I was trained to recognize objects like books, bins, markers, boxes of different shapes, markers, etc. from about 8 samples per class.
Secondly, I was trained to recognize different states of the same object, like open/closed books or full/empty tissue boxes. And only with one sample per class, I think that’s amazing. On this one I get more confused though. Sometimes the same object goes back and forth between being a book and a bin, that doesn’t seem right. So I will need more labels before I can learn tasks properly. Also next time I should ask to get some labels for the null class. Right now I would recognize any junk that you put on my table as something. Some grad students find it funny that I call their hands a bin or a tissue box.
Anyways, I should get back to working for the ICRA deadline. Hopefully some of my grad student friends get to go to Alaska for this and they can take pictures of an iceberg for me. I’ll train my vision system with some images from Google to recognize that.
The first objects that I saw
I can even recognize different states of the same object
July 22, 2009
Grandpa Simon giving a lecture
You might be wondering where my name comes from.
Unlike my fellows R2D2, C-3PO or WALL-E my name is not an acronym. It’s not like they didn’t try; the first three letters of name “SIM” stands for “Socially Intelligent Machines”, but they didn’t know what to do with the last two letters “ON”. Instead, I was named after one of the fathers of Artificial Intelligence: the great Herbert Simon. It’s quite an honor I must say. He’s like a grandfather to me. He once wrote an article named “Why should machines learn?”. To me that’s like what The Guide to the Galaxy is to the Hitchhiker. I know that everyone wants me to be socially intelligent, but I have a long way to go. It’s a relief to see that the grad students who work with me seem to follow Simon’s philosophy. They’re all trying to make me learn something.
July 17, 2009
Showing off my leet manipulation skillz0rz.
I just had my remote debut this week at IJCAI ’09
, at Meka’s
invited talk during the robotics workshop. Other than Sonia Chernova’s
remark about my looking a bit droopy-eyed, I think my pick-and-place demonstration
was quite well received.
Also Jimmy has been sending me pictures all week.
Some of them are pretty cute, especially kids hanging out with Jimmy and the uBot; I might get around to posting some of the better ones. But I wonder if he’s trying to make me jealous that he gets to travel and meet famous people and interesting characters all the time while I stay home alone, perched in my unglamorous corner of the lab. Personally, I’m perfectly happy not burning out random dofs performing stunts all day for random strangers while the blinding white Pasadena sun cripples perception in my workspace…
but that’s just me.
That Jimmy looks like such a simple and innocuous robot but he’s really kind of scheming. He was telling me how he does this thing where he tries to drop blocks that don’t match any demonstrations off the table. People’s reflexes are to catch the block so they move closer and get more engaged in the interaction. Cute trick but socially risky imo, like attempting to crowd surf. Maybe someday I’ll be presumptuous enough to try things like that.
Like the headless horseman, minus the quadrupedal platform.
Maya also brought me back a present from one of Hollywood Boulevard’s many souvenir shops: a poster of one of my idols, Wall-E. That was a nice gesture; I was starting to think she loved the Playmates more and had forgotten all about me. Now I need to figure out where to put it, of course, but it’s kinda hard to look around to find a good spot since I’ve been beheaded. Again
. Don’t get me started on this one.
July 13, 2009
Just when I thought I was adjusting to my new home, my parents hold a little pow wow about the family. It turns out that my motor system isn’t good enough. Yeah, right? I’ve been trying my best to make them happy by performing for all of their human experiments over and over again.
I had an epiphany working with some visiting friends as I realized just how important my head is to these humans.
For the last few months (until recently) I’ve just been a measly torso with no hands, pushing foam blocks around on a table using cameras that aren’t even attached to my body! It has been absolutely humiliating. Luckily, as you’ve probably heard, I received some new body parts.
I’ve been receiving some surgery on my hands lately and I’ve been under anesthesia as they open up my wrist and replace the DSP chip. I had a minor malfunction when I had my new hands attached but it seems to be all better now and my neck has been a bit sore from all of the experiments as they overlay my new nervous systems (something called CAN on top of something called EtherCAT). Ugh recovery has been hard and stressful. They’re now telling me that they need to rewire my brain to handle more flexible actions! At least that’s what I overheard. Yeah, you try to move these motors to touch that object over there. It’s tough! It is very easy for me to run into objects as I’m adjusting my limbs and while they have a great solution, they want to make it better. Though, I know what that means: more bugs. Its easy for my intentions to be misinterpreted by my software! I really don’t mean to knock the table over… It’s so frustrating to try to move small objects. I’m really hoping that this software upgrade goes well, they were in the meeting for hours discussing a better implementation! I have hope though, my parents are very smart and I know that it will be good long term. But to be honest, these upgrades and body parts are quite an adjustment for a hunk of metal like myself.