Simon, the robot, can read human reactions

The invention
Andrea Thomaz's lab at the Georgia Institute of Technology is the birthplace of Simon. The scientists are working and experimenting with him. The robot is designed to explore the possibilities of communications between humans and robots.

Researcher Aaron Bobick, a roboticist, specializing in robot vision at the Georgia Institute of Technology, said, "We would like to bring robots into the human world. That means they have to engage with human beings, and human beings have an expectation of being engaged in a way similar to the way other human beings would engage with them."

That’s the motto behind Simon’s design. The researchers are programming robots to understand when they have gained a person's attention. They are exploring the future in human-robot relationship and hoping that sooner or later it will help robots interact with humans the way Homo sapiens communicate with each other.

Who is Simon?
Bobick said, "Simon is a humanoid-torso robot, pretty close to the size of a smaller adult, except with no lower body and a disproportionately bigger head than the body might entail.”

The mission
The mission of the roboticists is to observe if Simon could tell when the he had successfully caught the eye of a person who is busy in his work.

"People were given a couple of tasks, such as talking on a cell phone, playing with blocks, or working on a Rubik's Cube," Bobick elucidated.

To gain people’s attention, Simon would wave his hands or indicate them to come closer. "The computer vision task was to try to determine whether or not you had captured the attention of the human being," Bobick said.

The experiment
During the experiment Simon monitored a person for several seconds before waving his hands and three seconds afterwards to find out any changes in that person’s behavior.

"Maybe people would wave back, or change the direction they were looking," Bobick said. "Simon would look at what their motions across their bodies were like, and if there were any deviations in the patterns of behavior they showed before and after the signal that Simon gave."

The outcome of the experiment is really impressive. Simon successfully identified whether someone was paying attention to him or overlooking him. His success rate is almost 80 percent. The main gadget that helped Simon to recognize people’s reaction is his guide cameras.

Bobick explained the team’s vision and the purpose of Simon in future. "Whether its purpose is to assist the elderly or to help a human with manufacturing and assembly, natural interactions between robots and human beings are necessary for the collaboration to be fluid. This ability to understand human reactions is part of our vision."

These dedicated scientists are also making some improvements. To make Simon more intelligent they are integrating other functionalities such as observation of human eyeball direction. "Fundamentally, human behavior is quite complex and often subtle," Bobick added, "This makes the perception challenging."

Shortcomings
However, there is one major shortcoming in Simon. His motions are not really human like. Bobick said, "For example, Simon moves more slowly than one would expect a human to move. This sometimes causes humans to also move slowly or in a more stilted fashion, which of course sometimes reduces our ability to detect the behavioral change. As Simon moves more naturally, we can expect people to react more naturally, which will likely make the perception easier."

On March 8, at the Human-Robot Interaction conference in Lausanne, Switzerland, Bobick and his associates shared their discoveries.