A horse, a zebra and artificial intelligence helped researchers teach a robot to recognize water and pour it into a glass.
Water is a tough challenge for robots because it is clear. Robots have learned how to pour water before, but previous techniques, such as heating the water and using a thermal camera or placing the glass in front of a checkerboard background, don’t fit well into everyday life.
A simpler solution would enable robot servers to refill water glasses, robot pharmacists to measure and mix medicines, or robot gardeners to water plants.
Now researchers have used AI and image translation to solve the problem.
Image translation algorithms use collections of images to train artificial intelligence to convert images from one style to another, such as transforming a photograph into a Monet-style painting or making an image of a horse resemble a zebra. For this study, the team used a method called contrastive learning for unpaired picture-to-picture translation, or CUT for short.
“You need a way to tell the algorithm what the right and wrong answers are during the training phase of learning,” said David Held, an assistant professor at the Robotics Institute at Carnegie Mellon University. “However, labeling data can be a time-consuming process, especially teaching a robot to pour water, which may require the human to label individual water droplets in an image.”
Enter the horse and zebra.
“Just as we can train a model to translate an image of a horse to look like a zebra, in the same way we can train a model to convert an image of colored liquid into an image of transparent liquid,” says Held. “We used this model to enable the robot to understand transparent liquids.”
A transparent liquid such as water is difficult for a robot to see because the way it reflects, refracts and absorbs light varies depending on the background. To teach the computer to see different backgrounds through a glass of water, the team played YouTube videos behind a transparent glass full of water. By training the system in this way, the robot can pour water against different backgrounds in the real world, no matter where the robot is.
“Even for humans, it’s sometimes difficult to precisely identify the boundary between water and air,” said Gautham Narasimhan, who received his master’s degree from the Robotics Institute in 2020 and worked with a team in the Institute’s Robots Perceiving and Doing Lab on the new work.
Their method allowed the robot to pour the water into a glass until it reached a certain height. They then repeated the experiment with glasses of different shapes and sizes.
Narasimhan says there is room for future research to extend this method: adding different lighting conditions, challenging the robot to pour water from one container to another, or estimating not only the height of the water, but also its volume.
The researchers presented their work at the IEEE International Conference on Robotics and Automation in May 2022.
“People in robotics really appreciate it when research works in the real world and not just in simulation,” said Narasimhan, who now works as a computer vision engineer at Path Robotics in Columbus, Ohio. “We wanted to do something that is quite simple but effective.”
Funding for the work came from LG Electronics and a grant from the National Science Foundation.
Source: Carnegie Mellon University