In a significant advancement in robotics, Google DeepMind has introduced a new AI agent named RoboCat. This agent is designed to learn a variety of tasks across different robotic arms, showcasing the ability to self-generate new training data to improve its techniques, marking a crucial step towards the creation of general-purpose robots.
RoboCat, a Transformer model with a VQ-GAN encoder, was released in June 2023. It is primarily intended for research into learning to accomplish a wide variety of tasks from expert demonstrations or multiple real robot embodiments for manipulation.
The primary intended users are Google DeepMind researchers, and it's not intended for commercial or production use.
RoboCat's standout feature is its learning speed. It can master a new task with as few as 100 demonstrations, leveraging a large and diverse dataset. This capability reduces the need for human-supervised training, potentially accelerating the pace of robotics research.
RoboCat's training involves a comprehensive five-step self-improvement process. It starts with collecting 100-1000 demonstrations of a new task or robot, using a robotic arm controlled by a human. This new task or arm data is used to fine-tune RoboCat, creating a specialized spin-off agent. This agent practices the new task or arm an average of 10,000 times, generating more training data.
The demonstration data and self-generated data are then incorporated into RoboCats existing training dataset, and a new version of RoboCat is trained on the updated dataset.
This process enables RoboCat to learn from a wide range of tasks and diverse training data types. Having been trained on millions of trajectories from both real and simulated robotic arms, RoboCat handles a variety of tasks involving different objects and variations, sourced from Reinforcement Learning (RL), Teleoperation (Teleop), and RoboCat itself.
These tasks include stacking RGB objects, tower and pyramid building with RGB objects, and lifting NIST-i gears, among others. The training involved four different types of robots and many robotic arms to collect vision-based data representing the tasks RoboCat would be trained to perform.
RoboCat demonstrates impressive adaptability by quickly learning to operate different robotic arms. For example, after observing 1000 demonstrations controlled by humans, RoboCat could successfully direct a new arm with a three-fingered gripper and twice as many controllable inputs, achieving an 86% success rate in picking up gears.
Moreover, the more new tasks RoboCat learns, the better it gets at learning additional new tasks. The initial version of RoboCat achieved a 36% success rate on previously unseen tasks after learning from 500 demonstrations per task. However, the latest version, trained on a more diverse set of tasks, more than doubled this success rate on the same tasks.
RoboCat's performance was evaluated through various tasks, such as inserting and removing objects from a bowl and lifting large gears. These evaluations were conducted in both simulated and real-world environments and compared to the performance of human teleoperators.
During the training process, RoboCat uses different observations to understand the robot's position and grip. These observations include joint angles, TCP position, gripper joint angle, and gripper grasp status. The specific observations depend on the robot and objects being used.
In the development of RoboCat, an interesting comparison was made between the VQ-GAN tokenizer and the patch ResNet used in Gato. The patch ResNet tokenizer performed better during training tasks but performed worse on tasks that were not included during training.
It's important to note that RoboCat is currently an early research model and has not been evaluated for deployment and safety outside of research environments. As RoboCat's capabilities expand, potential ethical and safety risks need to be carefully addressed. Therefore, caution should be exercised when considering the use of RoboCat outside of research settings. Nonetheless, the development of RoboCat represents a significant milestone in the field of robotics and AI, bringing us closer to a future where robots are an integral part of our everyday lives.
Read more:
RoboCat: Google DeepMind's innovative leap into AI-powered ... - The Jerusalem Post
- Nickelytics and Starship release robots at UCLA and Utah universities - Robotics and Automation News - May 1st, 2024 [May 1st, 2024]
- Next-Level Robotics: Enhancing Accuracy with Advanced Multisensory Interfaces - AZoRobotics - May 1st, 2024 [May 1st, 2024]
- Death of Lake Orion teen on robotics trip determined to be suicide, Houston police say - Detroit News - May 1st, 2024 [May 1st, 2024]
- Niqo Robotics Raises $9 Million In Funding Round Led By Brida Innovation Ventures: Report - Outlook India - May 1st, 2024 [May 1st, 2024]
- Mission to 'Holy Grail of Shipwrecks' Will Employ Robotics - USNI News - May 1st, 2024 [May 1st, 2024]
- Kiwibot acquires Auto Mobility Solutions Robotics & Automation News - Robotics and Automation News - May 1st, 2024 [May 1st, 2024]
- Untethered soft actuators for soft standalone robotics - Nature.com - May 1st, 2024 [May 1st, 2024]
- Robotics in the restaurant industry are here to stay: Carlos Gazitua - Fox Business - May 1st, 2024 [May 1st, 2024]
- It's time for UK business to embrace robotics - Design Products & Applications - May 1st, 2024 [May 1st, 2024]
- Meet the `One Percenters' - The Top Ranked Robotics Group from Franklin - Franklin Observer - May 1st, 2024 [May 1st, 2024]
- Atlas, forefather of humanoid robots, gives way to next generation - Marketplace - May 1st, 2024 [May 1st, 2024]
- BlueBotics releases new version of its server fleet management software - Robotics and Automation News - May 1st, 2024 [May 1st, 2024]
- U.S. manufacturers invested heavily in robotics in 2023, finds IFR - Robot Report - May 1st, 2024 [May 1st, 2024]
- This is a unique time: ARK Invests chief futurist tackles tech innovation from AI to robotics - CNBC - May 1st, 2024 [May 1st, 2024]
- U.S. Companies Invest Heavily in Robots - IFR Preliminary Results EMSNow - EMSNow - May 1st, 2024 [May 1st, 2024]
- MiR launches new autonomous mobile forklift-type robot - Robotics and Automation News - May 1st, 2024 [May 1st, 2024]
- Rutland Area Robotics' IBOTS compete in FIRST World Championships finals for first time - Rutland Herald - May 1st, 2024 [May 1st, 2024]
- Micropsi Industries introduces new AI-vision software Robotics & Automation News - Robotics and Automation News - May 1st, 2024 [May 1st, 2024]
- Robots on a Roll: The Future of Farming Is Here Now - Growing Produce - May 1st, 2024 [May 1st, 2024]
- Grand Force, Grand Forks' only FIRST Robotics team, returns from world championship with good memories - Grand Forks Herald - May 1st, 2024 [May 1st, 2024]
- Reston Robotics Team Wins National Competition At Tech Conference - Patch - May 1st, 2024 [May 1st, 2024]
- CapSen Robotics Introduces CapSen PiC 2.0 Bin-Picking Software - Supply and Demand Chain Executive - May 1st, 2024 [May 1st, 2024]
- Gecko Robotics and Al Masaood Energy Partner to Help ADNOC Gas Revolutionize Asset Operations and ... - Business Wire - May 1st, 2024 [May 1st, 2024]
- Vention Launches New Cloud-Robotic and AI Capabilities to Accelerate the Design and Deployment of Robot Cells - InvestorsObserver - May 1st, 2024 [May 1st, 2024]
- Inside Wilsonville High School's robotics team - KGW.com - May 1st, 2024 [May 1st, 2024]
- Talking AUTOMATE 2024 with TM Robotics | RoboticsTomorrow - Robotics Tomorrow - May 1st, 2024 [May 1st, 2024]
- Advancing obstetric and gynecologic surgery through robotic innovation - Contemporary Obgyn - May 1st, 2024 [May 1st, 2024]
- On the Horizon: A Robot to Assist ALS Patients | RoboticsTomorrow - Robotics Tomorrow - May 1st, 2024 [May 1st, 2024]
- Robots can use air to move objects - Earth.com - May 1st, 2024 [May 1st, 2024]
- Plymouth Robotics Team 586E takes 6th place VEX Robotics World Championship in Dallas, TX - WTCA - May 1st, 2024 [May 1st, 2024]
- These flying origami-inspired robots change shape in mid-air - Engadget - September 29th, 2023 [September 29th, 2023]
- Industrial Robotics Market Size To Reach USD 94.8 Billion By 2032 ... - GlobeNewswire - September 29th, 2023 [September 29th, 2023]
- ForwardX Robotics and DHL Collaborate To support Expansion - PR Newswire - September 29th, 2023 [September 29th, 2023]
- Dexterity AI and FedEx Unveil First-of-its-Kind Robotics Trailer ... - Robotics Tomorrow - September 29th, 2023 [September 29th, 2023]
- Newton High Schools Team Aperture attends super bowl of robotics - New Jersey Education Association - September 29th, 2023 [September 29th, 2023]
- Exoskeleton Report eMagazine: The Latest in Wearable Robotics ... - Exoskeleton Report - September 29th, 2023 [September 29th, 2023]
- 12.5m for robotics and automation to boost sustainable farming - GOV.UK - September 29th, 2023 [September 29th, 2023]
- Robotics As A Service (RaaS) Market size to increase by USD 1.50 ... - PR Newswire - September 29th, 2023 [September 29th, 2023]
- Learn how to create socially intelligent robots at RoboBusiness - Robot Report - September 29th, 2023 [September 29th, 2023]
- Monument Health Welcomes Two New Robotics-Trained General ... - Newscenter1.tv - September 29th, 2023 [September 29th, 2023]
- 3D-Printed Robotic Gripper Works Without Electronics - Design News - September 29th, 2023 [September 29th, 2023]
- WVU Today | WVU researcher to help send swarm of marine robots ... - WVU Today - September 29th, 2023 [September 29th, 2023]
- Using robotics & AI to reduce water - Times of India - September 29th, 2023 [September 29th, 2023]
- We need a new social contract for the coming golden age of robotics - Big Think - July 2nd, 2023 [July 2nd, 2023]
- The Intersection of Autonomous Robotics and Solar Panel Cleaning ... - EnergyPortal.eu - July 2nd, 2023 [July 2nd, 2023]
- Seen at HIMSS: the latest innovations in artificial intelligence, mixed ... - Kevin MD - July 2nd, 2023 [July 2nd, 2023]
- KEENON Robotics Empowers Future of Hotel Experiences as ... - PR Newswire - July 2nd, 2023 [July 2nd, 2023]
- Brief Overview of Investing in Robotics in 2023 - Analytics Insight - July 2nd, 2023 [July 2nd, 2023]
- Greek post office makes further commitment to Robotics parcel ... - Post and Parcel - July 2nd, 2023 [July 2nd, 2023]
- Robotic glove that 'feels' lends a 'hand' to relearn playing piano after ... - Science Daily - July 2nd, 2023 [July 2nd, 2023]
- 'Bionic professor' aims to transform the field of wearable robotics - University of Toronto - July 2nd, 2023 [July 2nd, 2023]
- The Advantages of Adopting Robotics for Energy Infrastructure ... - EnergyPortal.eu - July 2nd, 2023 [July 2nd, 2023]
- Upskill with course in robotics, IS handler told terrorists: NIA - Times of India - July 2nd, 2023 [July 2nd, 2023]
- Saving the World's Coral Reefs with AI and Robotics - TriplePundit - July 2nd, 2023 [July 2nd, 2023]
- How To Learn Robotics and Coding - StartupGuys.net - July 2nd, 2023 [July 2nd, 2023]
- Transformative Impact of Robotics in the Healthcare Industry - Analytics Insight - June 4th, 2023 [June 4th, 2023]
- Implementing robotics could improve workplace safety, report says - HR Dive - June 4th, 2023 [June 4th, 2023]
- Rise of the machines: What will drive adoption of robotics and ... - AgFunderNews - June 4th, 2023 [June 4th, 2023]
- Robotics investments top $1.63B in April 2023 - Robot Report - June 4th, 2023 [June 4th, 2023]
- We are pleased to announce our 3rd Reddit Robotics Showcase! - Robohub - June 4th, 2023 [June 4th, 2023]
- Jersey City's Oishii partners with robotics company to bring ... - ROI-NJ.com - June 4th, 2023 [June 4th, 2023]
- New Dog, New Tricks: Reflections on Construction, Robotics, and ... - Archinect - June 4th, 2023 [June 4th, 2023]
- Underwater robots for operations in challenging and dangerous ... - Inceptive Mind - June 4th, 2023 [June 4th, 2023]
- NVIDIA Brings Advanced Autonomy to Mobile Robots With Isaac AMR - Nvidia - June 4th, 2023 [June 4th, 2023]
- Robotic assisted surgery now available at Northeast Regional ... - Kirksville Daily Express and Daily News - June 4th, 2023 [June 4th, 2023]
- Robotic hand offers innovative nuclear solution : Waste & Recycling - World Nuclear News - June 2nd, 2023 [June 2nd, 2023]
- Hy-Tek Intralogistics and Hai Robotics Announce Partnership ... - Robotics Tomorrow - June 2nd, 2023 [June 2nd, 2023]
- UNA dubbed new home to BEST Robotics Competition - WHNT News 19 - June 2nd, 2023 [June 2nd, 2023]
- Serve Robotics to deploy up to 2000 delivery bots on Uber Eats - Supply Chain Dive - June 2nd, 2023 [June 2nd, 2023]
- Amazon executive joins Medtronic to spearhead development in ... - Vascular News - June 2nd, 2023 [June 2nd, 2023]
- Armach Robotics' To Showcase Their Novel Hull Cleaning Solution ... - Marine Insight - June 2nd, 2023 [June 2nd, 2023]
- AMP Robotics breaks in Colorado headquarters as it eyes future ... - Waste Dive - June 2nd, 2023 [June 2nd, 2023]
- West Linn robotics team 2B Determined learns to lead and change ... - West Linn Tidings - June 2nd, 2023 [June 2nd, 2023]
- No Longer a 'Titan' in Surgical Robotics - Medical Device and Diagnostics Industry - June 2nd, 2023 [June 2nd, 2023]
- PERRONE ROBOTICS UNVEILS GROUNDBREAKING FUSION OF ... - PR Newswire - June 2nd, 2023 [June 2nd, 2023]
- uWare robotics map what lies beneath the ocean with its ... - Tech.eu - June 2nd, 2023 [June 2nd, 2023]
- Secretary of State's opening speech at Robotics and Automation ... - GOV.UK - June 2nd, 2023 [June 2nd, 2023]
- Mukilteo Robotics team Modulo reflects on their World ... - Lynnwood Times - June 2nd, 2023 [June 2nd, 2023]
- ROBO A Buy Amid Robotics Optimism And Ongoing Re-Shoring ... - Seeking Alpha - June 2nd, 2023 [June 2nd, 2023]
- Prosthetic hand adapted for robotic use in nuclear environments - Professional Engineering - June 2nd, 2023 [June 2nd, 2023]