Join us in returning to NYC on June 5th to collaborate with executive leaders in exploring comprehensive methods for auditing AI models regarding bias, performance, and ethical compliance across diverse organizations. Find out how you can attend here.
Large language models (LLMs) can accelerate the training of robotics systems in super-human ways, according to a new study by scientists at Nvidia, the University of Pennsylvania and the University of Texas, Austin.
The study introduces DrEureka, a technique that can automatically create reward functions and randomization distributions for robotics systems. DrEureka stands for Domain Randomization Eureka. DrEureka only requires a high-level description of the target task and is faster and more efficient than human-designed rewards in transferring learned policies from simulated environments to the real world.
The implications can be great for the fast-moving world of robotics, which has recently gotten a renewed boost from the advances in language and vision models.
When designing robotics models for new tasks, a policy is usually trained in a simulated environment and deployed to the real world. The difference between simulation and real-world environments, referred to as the sim-to-real gap, is one of the big challenges of any robotics system. Configuring and fine-tuning the policy for optimal performance usually requires a bit of back and forth between simulation and real-world environments.
The AI Impact Tour: The AI Audit
Join us as we return to NYC on June 5th to engage with top executive leaders, delving into strategies for auditing AI models to ensure fairness, optimal performance, and ethical compliance across diverse organizations. Secure your attendance for this exclusive invite-only event.
Recent works have shown that LLMs can combine their vast world knowledge and reasoning capabilities with the physics engines of virtual simulators to learn complex low-level skills. For example, LLMs can be used to design reward functions, the components that steer the robotics reinforcement learning (RL) system to find the correct sequences of actions for the desired task.
However, once a policy is learned in simulation, transferring it to the real world requires a lot of manual tweaking of the reward functions and simulation parameters.
The goal of DrEureka is to use LLMs to automate the intensive human efforts required in the sim-to-real transfer process.
DrEureka builds on Eureka, a technique that was introduced in October 2023. Eureka takes a robotic task description and uses an LLM to generate software implementations for a reward function that measures success in that task. These reward functions are then run in simulation and the results are returned to the LLM, which reflects on the outcome and modifies it to the reward function. The advantage of this technique is that it can be run in parallel with hundreds of reward functions, all generated by the LLM. It can then pick the best functions and continue to improve them.
While the reward functions of Eureka are great for training RL policies in simulation, it does not account for the messiness of the real world and therefore requires manual sim-to-real transfer. DrEureka addresses this shortcoming by automatically configuring domain randomization (DR) parameters.
DR techniques randomize the physical parameters of the simulation environment so that the RL policy can generalize to the unpredictable perturbances it meets in the real world. One of the important challenges of DR is choosing the right parameters and range of perturbations. Adjusting parameters requires commonsense physical reasoning and knowledge of the target robot.
These characteristics of designing DR parameters make it an ideal problem for LLMs to tackle because of their strong grasp of physical knowledge and effectiveness in generating hypotheses, providing good initializations to complex search and black-box optimization problems in a zero-shot manner, the researchers wrote.
DrEureka uses a multi-step process to break down the complexity of optimizing reward functions and domain randomization parameters at the same time. First, an LLM generates reward functions based on a task description and safety instructions about the robot and the environment. DrEureka uses these instructions to create an initial reward function and learn a policy as in the original Eureka. The model then runs tests with the policy and reward function to determine the suitable range of physics parameters, such as friction and gravity.
The LLM then uses this information to select the optimal domain randomization configurations. Finally, the policy is retrained with the DR configurations to become robust against the noisiness of the real world.
The researchers described DrEureka as a language-model driven pipeline for sim-to-real transfer with minimal human intervention.
The researchers evaluated DrEureka on quadruped and dexterous manipulator platforms, although the method is general and applicable to diverse robots and tasks. Their findings show that in quadruped locomotion, policies trained with DrEureka outperform the classic human-designed systems by 34% in forward velocity and 20% in distance traveled across various real-world evaluation terrains. They also tested DrEureka on dexterous manipulation with robotic hands. Given a fixed amount of time, the best policy trained by DrEureka performed 300% more cube rotations than human-developed policies.
But the most interesting finding was the application of DrEureka on the novel task of having a robo-dog balancing and walking on a yoga ball. The LLM was able to design a reward function and DR configurations that allowed the trained policy to be transferred to the real world with no extra configurations and perform well enough on diverse indoor and outdoor terrains with minimal safety support.
Interestingly the study found that the safety instruction included in the task description plays an important role in ensuring that the LLM generates logical instructions that transfer to the real world.
We believe that DrEureka demonstrates the potential of accelerating robot learning research by using foundation models to automate the difficult design aspects of low-level skill learning, the researchers wrote.
VB Daily
Stay in the know! Get the latest news in your inbox daily
By subscribing, you agree to VentureBeat's Terms of Service.
Thanks for subscribing. Check out more VB newsletters here.
An error occured.
Read the rest here:
Nvidia's DrEureka outperforms humans in training robotics systems - VentureBeat
- Area legislators pay tribute to Southern Tier robotics team - The New York State Senate - June 15th, 2024 [June 15th, 2024]
- Local Students Head to Robotics World Championship - TPG Online Daily - June 15th, 2024 [June 15th, 2024]
- 3 Robotics Stocks to Turn $1000 into $1 Million: June 2024 - InvestorPlace - June 15th, 2024 [June 15th, 2024]
- Senators honor Southern Tier robotics team for international success - FingerLakes1.com - June 15th, 2024 [June 15th, 2024]
- Assessing the Broad Robotics Landscape - ETF Trends - June 15th, 2024 [June 15th, 2024]
- Why Is Nauticus Robotics (KITT) Stock Up 59% Today? - InvestorPlace - June 15th, 2024 [June 15th, 2024]
- Exploring the Latest Trends in Robotics Software for Supply Chain Optimization - BBN Times - June 15th, 2024 [June 15th, 2024]
- How Is The Market Feeling About Nauticus Robotics? - Nauticus Robotics (NASDAQ:KITT) - Benzinga - June 15th, 2024 [June 15th, 2024]
- Mentor High Robotics Team finishes in top 7 at national competition - News-Herald - June 15th, 2024 [June 15th, 2024]
- Collaborative Robotics expands with new Seattle office and AI team - Robot Report - June 15th, 2024 [June 15th, 2024]
- WVU Robotics Team Wins - WDTV - June 15th, 2024 [June 15th, 2024]
- The 7 Best Robotics Stocks to Buy in June 2024 - InvestorPlace - June 15th, 2024 [June 15th, 2024]
- Local robotics team honored by New York State legislators - WENY-TV - June 15th, 2024 [June 15th, 2024]
- Robots as a Data Source: Obtaining Insights From Robotic Autonomy - AI Business - June 15th, 2024 [June 15th, 2024]
- How west county is growing: Palms West Hospital to build 5-story tower for robotics surgery - Palm Beach Post - June 15th, 2024 [June 15th, 2024]
- Garrett robotics team wins Appalachian Lab Inspiration Award - Cumberland Times-News - June 15th, 2024 [June 15th, 2024]
- A Robotics business Co-founded by Synapse's CEO is raising cash with Ubreasonable Claims Regarding GM ties ... - CIO Look - June 15th, 2024 [June 15th, 2024]
- Apple's next nebulous idea: smart home robots - The Verge - June 15th, 2024 [June 15th, 2024]
- FIRST robotics victory is a win and hope for Israeli education - The Jerusalem Post - June 15th, 2024 [June 15th, 2024]
- Bay Area robotics startup to open Seattle office after $100M round - The Business Journals - June 15th, 2024 [June 15th, 2024]
- Field robotics with Stratom Inc and inside standards at ASTM - Robot Report - June 15th, 2024 [June 15th, 2024]
- Helping Robots Grasp the Unpredictable - The Good Men Project - June 15th, 2024 [June 15th, 2024]
- Tesla restarts hiring beyond AI and robotics in a big way - Electrek.co - June 15th, 2024 [June 15th, 2024]
- Iowa manufacturers outline advancements, challenges from integrating AI, robotics, vision systems to production - Business Record - June 15th, 2024 [June 15th, 2024]
- Generative AI takes robots a step closer to general purpose - TechCrunch - June 15th, 2024 [June 15th, 2024]
- Tesla jumps on talent of Koch company making direct drive for robots - Electrek.co - June 15th, 2024 [June 15th, 2024]
- Robotics Manufacturing Hub to help small and midsize U.S. manufacturers compete - Robot Report - June 15th, 2024 [June 15th, 2024]
- Collaborative Robotics Opens Seattle Office After $100MM Raise - The Registry Seattle - June 15th, 2024 [June 15th, 2024]
- DeepMind experimenting with 'Shadow Hand' that can withstand a severe beating in the name of AI research - Livescience.com - June 15th, 2024 [June 15th, 2024]
- Top 6 examples of humanoid robots - TechTarget - June 15th, 2024 [June 15th, 2024]
- Brief: NVIDIA's venture arm invests in autonomous weeding startup Carbon Robotics - AgFunderNews - May 11th, 2024 [May 11th, 2024]
- 7 Robotics Stocks to Bet On for Reliable AI-Powered Returns - InvestorPlace - May 11th, 2024 [May 11th, 2024]
- NSF and USDA join forces to boost innovation in agricultural robotics - Future Farming - May 11th, 2024 [May 11th, 2024]
- Meet The Swiss Start-Up Taking On The Tech Giants In Robotics And AI - Forbes - May 11th, 2024 [May 11th, 2024]
- LimX Dynamics' W1 robot can stand, walk and roll on its hind 'legs' - Interesting Engineering - May 11th, 2024 [May 11th, 2024]
- Lear Corporation's acquisition showcases the industry's use of robotics and AI for efficiency and innovation - CBT Automotive News - May 11th, 2024 [May 11th, 2024]
- OMRON and NEURA Robotics Partner to Unveil New Cognitive Robot at Automate 2024 - Automation.com - May 11th, 2024 [May 11th, 2024]
- Community briefs: Ravenswood robotics at world championship, Bike to the Library Day - The Almanac Online - May 11th, 2024 [May 11th, 2024]
- JCHS Robotics shimes at VEX World Competition | Community News - The Tomahawk - May 11th, 2024 [May 11th, 2024]
- Carbon Robotics to Help Revolutionize Farming with AI, Using Investment from NVentures - Business Wire - May 11th, 2024 [May 11th, 2024]
- Humanoid Robots: The Next-Generation Robotic Workforce - Tech Briefs - May 11th, 2024 [May 11th, 2024]
- PHOTOS Motiva helping fund Port Arthur ISD robotics and engineering - Port Arthur News - The Port Arthur News - May 11th, 2024 [May 11th, 2024]
- Kiwanis Club of Miami helps fund McMillan MS robotics team trip - Miami's Community Newspapers - May 11th, 2024 [May 11th, 2024]
- The Interplay Between Robotics and Artificial Intelligence in Manufacturing | RoboticsTomorrow - Robotics Tomorrow - May 11th, 2024 [May 11th, 2024]
- Doosan Robotics releases its longest reach cobots with PRIME SERIES - Robot Report - May 11th, 2024 [May 11th, 2024]
- KION North America and Fox Robotics Announce Non-Exclusive Strategic Partnership - Automation.com - May 11th, 2024 [May 11th, 2024]
- Ag tech startup Carbon Robotics lands Nvidia investment - The Business Journals - May 11th, 2024 [May 11th, 2024]
- Mars Robotics offers chance to drive Mars rover on Saturday - Butler Eagle - May 11th, 2024 [May 11th, 2024]
- Comparative analysis of robot-assisted language learning systems and human tutors in English conversation lessons - EurekAlert - May 11th, 2024 [May 11th, 2024]
- Valencia College to more than double students training in semiconductors, robotics and optics - The Business Journals - May 11th, 2024 [May 11th, 2024]
- DeepMind is experimenting with a nearly indestructible robot hand - New Scientist - May 11th, 2024 [May 11th, 2024]
- Five Things to Consider Before Adopting Robotics for Warehouse Automation - Automation.com - May 11th, 2024 [May 11th, 2024]
- Nvidia and Alphabet's Intrinsic aim to revolutionize next-gen robotics - VentureBeat - May 11th, 2024 [May 11th, 2024]
- NVIDIA and Alphabet's Intrinsic Put Next-Gen Robotics Within Grasp - Automation.com - May 11th, 2024 [May 11th, 2024]
- STEM robotics team celebrates 10 years and remembers hero who saved lives - Denver 7 Colorado News - May 11th, 2024 [May 11th, 2024]
- Zero-shot learning helps Intrinsic pave the future for robotics - FierceElectronics - May 11th, 2024 [May 11th, 2024]
- EN robotics team demonstrates real-world learning | News Sun | kpcnews.com - KPCnews.com - May 11th, 2024 [May 11th, 2024]
- Future Innovators Challenged to Design Nuclear Fission and Fusion Decommissioning Robots - Automation.com - May 11th, 2024 [May 11th, 2024]
- Meet Pepper and Bernard: The robots shaping AI research at SDSU - Daily Aztec - May 11th, 2024 [May 11th, 2024]
- Gene Haas Foundation Partner Hosts FIRST Robotics District Competitions - Kettering University - May 5th, 2024 [May 5th, 2024]
- NASA Showcases Innovations at the 2024 FIRST Robotics World Championships - NASA - May 5th, 2024 [May 5th, 2024]
- Robots are invading the San Carlos Branch Library - San Diego Community Newspaper Group - May 5th, 2024 [May 5th, 2024]
- Wyoming-Based Company Makes High-Tech Robots To Go Where Humans Don't Want To - Cowboy State Daily - May 5th, 2024 [May 5th, 2024]
- Anzu Robotics launches U.S.-based clone of DJI Mavic 3 - Robot Report - May 5th, 2024 [May 5th, 2024]
- Carnegie Mellon University's Robotics Institute begins developing drones to fight wildfires - CBS Pittsburgh - May 5th, 2024 [May 5th, 2024]
- 3 Robotics Stocks That Could Be Multibaggers in the Making: Spring Edition - MSN - May 5th, 2024 [May 5th, 2024]
- Hopkinton teams advance to Robotics World Championships - Hopkinton Independent - May 5th, 2024 [May 5th, 2024]
- "Big Cat Robotics" team from Wellsville returns from VRC World Championship - THE WELLSVILLE SUN - THE WELLSVILLE SUN - May 5th, 2024 [May 5th, 2024]
- PropStream-Sponsored Robotics Team Places in Competition - RisMedia.com - May 5th, 2024 [May 5th, 2024]
- Bellefontaine Robotics Team Travels to World Championship Peak of Ohio - peakofohio.com - May 5th, 2024 [May 5th, 2024]
- ASL Aviation Holdings Inks Deal with Reliable Robotics for 30 Aircraft Autonomy Systems - Business Wire - May 5th, 2024 [May 5th, 2024]
- Global Nano Robotics Market Projected to Surge to US$ 19.86 Billion by 2032: Healthcare Trends Catalyzing Nano ... - GlobeNewswire - May 5th, 2024 [May 5th, 2024]
- 'Afghan Dreamers' | Film about Afghan girls robotics team to be screened on May 15 - Piedmont Exedra - May 5th, 2024 [May 5th, 2024]
- 3 Robotics Stocks That Could Be Multibaggers in the Making: Spring Edition - InvestorPlace - May 5th, 2024 [May 5th, 2024]
- The Rise of the Navy Robots - Signal Magazine - May 5th, 2024 [May 5th, 2024]
- Elon Musk Keeps Talking About the Future of Robotics With Optimus, but This Phenomenal Growth Stock Has Already ... - The Motley Fool - May 5th, 2024 [May 5th, 2024]
- Robosen Robotics Teams Up with Hasbro to Debut the Worlds First Auto-Converting Decepticon Megatron! - aNb Media - May 5th, 2024 [May 5th, 2024]
- John P. Stevens Robotics Team Wins Excellence in Technology at Innovators Awards - TAPinto.net - May 5th, 2024 [May 5th, 2024]
- Nala Robotics and Re-Up partner to develop AI-powered robotic chef - Robotics and Automation News - May 5th, 2024 [May 5th, 2024]
- LCSD1 to Host Final Robotics Competition of the School Year Tomorrow - The Cheyenne Post - May 5th, 2024 [May 5th, 2024]