Key Takeaways

• Generalist AI’s GEN-1.5 robot can learn physical tasks from a single demonstration, showcasing one-shot learning capabilities.
• The robot achieved an average success rate of 59% with one-shot prompting and 83% with few-shot learning, indicating room for improvement.
• Physical prompts provide a flexible learning mechanism, allowing the robot to generalize tasks across different simulations and real-world contexts.

One-shot Learning in Robotics

Generalist AI has introduced the GEN-1.5 robot, notable for its ability to learn new physical tasks from just one demonstration. This innovation stands out as it allows the robot to adapt without extensive, task-specific programming, challenging the traditional methods that often take weeks to implement.

The GEN-1.5 model is a large multimodal system that integrates video, sensor, language, and proprioceptive data. It processes actions at 100 Hz and can remember up to 30 seconds of context. By using a single demonstration lasting between three and twelve seconds, the robot can replicate the task immediately, without any prior training. Generalist AI did not modify the architecture for this capability, positing that its success could stem from parallels with language learning and the repetitive nature of physical tasks.

Despite its advancements, the success rates for these tasks remain modest. In evaluations across ten tasks—ranging from twisting off jar lids to folding paper—the one-shot learning approach yielded an average success rate of 59%, while a few-shot method, where the model used about 50 demonstrations, raised the success rate to 83%. The tasks included simple actions like stacking cups and retrieving objects.

Progressing Through Training

The development of GEN-1.5 builds on its predecessor, GEN-0, which demonstrated predictable behavior during its pretraining phase. The current model has undergone over eight months of continuous training and has shown a decrease in the gradient steps needed for task adaptation—distilling from hundreds to a single step.

Generalist AI went further by testing if GEN-1.5 could learn a task entirely from context without any gradient updates, a feat not previously observed in robotic models. This was successfully accomplished when demonstrations from simulations informed real-world actions, allowing the robot to adapt to different objects and contexts.

Chaining Prompts for Complex Tasks

The GEN-1.5 model also exhibits the capability to chain prompts—from different demonstrations—into a cohesive task. For example, it combined two independent prompts involving unzipping a pencil pouch and retrieving money into a single continuous action sequence. This demonstrates the model’s ability to create complex tasks from simple demonstrations.

Furthermore, GEN-1.5 can generalize actions to new environments, as evidenced by successful adaptations in various real-world scenarios. The robot demonstrated the ability to creatively substitute tools, like using a banana as a broom, highlighting its innovative problem-solving capabilities.

The Future of Robot Learning

Generalist AI emphasizes the potential of one-shot learning to transform industrial robotics, where previous methods necessitated extensive programming by experts. The company believes that teaching a machine through demonstration can significantly reduce the time and effort associated with adapting robots to new tasks once they reach sufficient pretraining levels.

However, it is crucial to recognize the limitations of skills learned solely through contextual prompting, which can be less reliable than those perfected through fine-tuning. As the technology progresses, it holds promise for more intuitive and adaptable robotic systems capable of functioning in diverse real-world applications.

Learn more about advancements in physical AI at the upcoming Physical AI Expo across various locations, including Amsterdam and North America.

The content above is a summary. For more details, see the source article.