How this humanoid robot learned to make coffee by watching videos

3K

What if I told you that a robot can make you a cup of coffee just by watching a video of a human doing it? It’s almost hard to believe, but it actually happened. Thanks to a breakthrough in artificial intelligence, humanoid robots can now watch and learn from humans, just like we do.

The company behind the coffee-making robot

The company behind this innovation is Figure, which has developed a full-body humanoid robot called Figure-01. This AI robot can walk, talk, and interact with its surroundings, and it can also learn new skills by watching videos.

MORE: THE NEXT GENERATION OF TESLA’S HUMANOID ROBOT MAKES IT DEBUT  

 

How the robot was able to make a cup of coffee

Figure recently demonstrated how Figure-01 can make coffee using a Keurig machine. The demonstration took place at Figure’s headquarters in San Francisco, where a human asked Figure-01 to make him a coffee. The robot responded with a cheerful “Sure, I can do that!” and proceeded to the Keurig machine.

The robot had previously watched a video of a human making coffee using the same machine, and it had learned how to mimic the human’s actions. It used its vision system to recognize the machine and its parts, and its motion planning system to coordinate its movements.

It opened the hatch, inserted the pod, closed the hatch, and pressed the button, all by itself. It then waited for the coffee to be ready, and let the human grab the full cup out of the machine himself.

MORE: 10 COFFEE MAKERS TO MAKE YOUR MORNINGS BETTER

 

The whole process took less than a minute, and the robot did not need any human assistance or guidance. It was able to perform the task autonomously, using only the video as a reference. This is an example of end-to-end AI, where the neural networks take video in and trajectories out.

MORE: MEET THE WORLD’S FIRST AI MASSAGE ROBOT  

 

What this means for AI and robotics

The demonstration was not only impressive but also significant and impactful. It showed that Figure-01 has added a new autonomous action to its library, which can be transferred to any other Figure robot running on the same system via swarm learning.

This means that if one robot learns something new, all the other robots can learn it, too, without having to watch the video themselves. This makes the learning process faster and more efficient and enables the robots to share their knowledge and skills with each other.

The demonstration also showed that this learning process could be used across a broad range of different tasks and that Figure-01 can learn to do anything from peeling bananas to using power tools to making art, all by watching videos. The robot can also learn not only what to do but also why and how to do it and how to adapt to different situations and contexts.

MORE: HOW THIS ROBOT HELPS YOU PROTECT AND CONNECT YOUR HOME 

 

What’s next for Figure-01 and video learning

The demonstration was just the beginning. Figure says that 01 can learn to do anything by watching videos and that it is constantly improving its capabilities and performance. The company has big plans for the future, and it wants to make its robot more versatile, creative, and collaborative.

 

Kurt’s key takeaways

Imagine if Figure-01 could soon go find a cup in the kitchen, check that the Keurig’s plugged in and has plenty of water in it, make the press-button coffee, and bring it to your desk without spilling it, all by watching videos and using its walking and language capabilities.  These are some of the possibilities and applications of video learning for robots, and they are not far from reality.

Does this sound like progress or simply terrifying where this may lead? What would you like to teach a robot by showing it a video? Let us know in the comments below.

FOR MORE OF MY SECURITY ALERTS, SUBSCRIBE TO MY FREE CYBERGUY REPORT NEWSLETTER HERE