Google’s “Robotics Transformer 1” to usher in the era of large robot models

[ad_1]

Summary

Google demonstrates a new AI model with a large data set for controlling robots in real time.

Recent successes in the development of artificial intelligence systems that process images or natural language are based on a common approach: large and diverse datasets processed by powerful and efficient models.

Generative AI models for text or images, such as GPT-3 and DALL-E, get their data from the Internet and rely only on specific datasets for tuning. OpenAI, for example, uses human feedback datasets to better tailor a large AI model to human needs.

Robotics lack gigantic datasets like those that exist for text and images. Masses of robot data would have to be collected by autonomous or human-teleoperated operations, making it expensive and difficult to create. Furthermore, there is not yet an AI model that can learn from that data and generalize in real time.

A.D

THE DECODER Newsletter

Top AI news straight to your inbox.

✓ Weekly

✓ Free

✓ Cancel at any time

Some researchers instead rely on training AI robots in simulations. Others are trying to make AI learn from internet video.

Google’s Robotics Transformer 1 learns from several modalities

Google now introduces Robotics Transformer 1 (RT-1), an AI model for controlling robots. The model is accompanied by a large real-world data set for robot training.

The model uses text and image instructions as input, which are tokenized by a FiLM EfficientNet model and compressed with an additional method (TokenLearner). The inputs are then passed to the Transformer, which sends the commands to the robot. According to Google, this makes the model fast enough to control bots in real time.

RT-1 can efficiently process images and text instructions, according to Google. | Image: Google

To share

Recommend our article

Share Google’s RT-1 learns from over 100,000 examples

To train RT-1, Google used a large dataset of 130,000 examples of more than 700 robotic tasks like pick, deposit, open, and more that the company collected in 17 months with 13 robots from Everyday Robots, a robotics under Alphabet. The data includes robot joint movements, robot bases, camera footage, and textual descriptions of tasks.

After training, the Google team compared RT-1 to other methods in various visible and invisible tasks, as well as how robustly the compared models coped with different environments.

Recommendation

RT-1 clearly outperformed the other methods, including Deepmind’s Gato, in all scenarios. Google also experimented with other data sources from another robot model. The results of these experiments would suggest that RT-1 can learn new skills with training data from other robots, writes Google.

Google’s RT-1 improves SayCan

The team also tested whether the performance of Google’s SayCan could be improved with RT-1. In fact, the combined system performed nearly 20 percent better and was able to maintain that success rate even in a more complicated kitchen environment.

RT-1 Robotics Transformer is a simple and scalable action generation model for real-world robotics tasks. Tokenize all inputs and outputs and use a pre-trained EfficientNet model with initial language fusion and a token learner for compression. RT-1 shows strong performance in hundreds of tasks and broad generalization capabilities and robustness in real-world settings.

Google

The team hopes to increase the number of robot skills learned faster in the future. To do so, it plans to bring in people with no experience in robotic teleoperation to contribute to the training dataset. It also aims to further improve reaction times and the ability to retain context over time.

The code for RT-1 is available on GitHub.

[ad_2]