At its recent GTC 2024 event, Nvidia announced a new foundational model to build intelligent humanoid robots. Dubbed GR00T, short for Generalist Robot 00 Technology, the model will understand natural language and be able to observe human actions and emulate human movements.
According to Nvidia CEO Jensen Huang, creating intelligent humanoid robots is the most exciting AI problem today. GR00T robots will learn coordination and other skills by observing humans to be able to navigate, adapt and interact with the real world. At the conference keynote, Huang showed several demos of what GR00T is capable of at the moment, including some robots performing a number of tasks.
The GR00T model takes multimodal instructions and past interactions as input and produces the actions for the robot to execute.
To power GR00T, Nvidia has created a new family of systems-on-modules, called Jetson Thor, using the latest Blackwell graphics architecture from the company and able to provide 800 teraflops (TFLOPS) of eight-bit floating-point compute.
At the foundation of GR00T lies Nvidia Isaac Sim, an extensible, Omniverse-based platform for robotics simulation aimed to improve the way AI-based robots are designed and tested, according to the company.
To train GR00T at scale, Nvidia has also built a new compute orchestration platform, Nvidia Osmo, aimed at coordinating training and inference across several Nvidia systems, including DGX systems for training, OVX systems for simulation, and IGX and AGX systems for hardware-in-the-loop validation.
Embodied AI models require massive amounts of real and synthetic data. The new Isaac Lab is a GPU-accelerated, lightweight, performance-optimized application built on Isaac Sim specifically for running thousands of parallel simulations for robot learning.
While GR00T is still very much a work in progress, Nvidia has announced two of the building blocks that will compose it, as part of the Isaac platform: a foundational model for robotic-arm manipulators, called Isaac Manipulator, and a collection of hardware-accelerated packages for visual AI and perception, the Isaac Perceptor.
According to Nvidia, Isaac Manipulator
provides up to an 80x speedup in path planning and zero-shot perception increases efficiency and throughput, enabling developers to automate a greater number of new robotic tasks.
On the other hand, Isaac Perceptor aims to improve efficiency and safety in environments where autonomous mobile robots are used, such as in manufacturing and fulfillment operations.
Both the Manipulator and the Perceptor should become available in the next quarter, says Huang.
On a related note, Nvidia has joined the Open Source Robotics Alliance, which aims to provide financial and industry support to the Robot Operating System (ROS). The company has not detailed if they plan to use ROS for GR00T robots, though.
Link:
Nvidia Announces Robotics-Oriented AI Foundational Model - InfoQ.com
- Robotics Academy - December 14th, 2016 [December 14th, 2016]
- KUKA Robotics | KUKA AG - December 14th, 2016 [December 14th, 2016]
- Robotics - The Great Courses - December 22nd, 2016 [December 22nd, 2016]
- Robotics - reddit - January 11th, 2017 [January 11th, 2017]
- Chatham Robotics Goes Undefeated and Moves On to State Championship - TAPinto.net - February 7th, 2017 [February 7th, 2017]
- Meridian robotics team gears up for Idaho FRC Regional - KBOI-TV - February 7th, 2017 [February 7th, 2017]
- Local robotics students headed to World Championship tournament - KOLO - February 7th, 2017 [February 7th, 2017]
- Report: Nearshoring favors robotics, will require heavy capital investment - Supply Chain Dive - February 7th, 2017 [February 7th, 2017]
- Belding hosts largest robotics competition in history of event - Greenville Daily News - February 7th, 2017 [February 7th, 2017]
- First Tech Challenge: High schools battle in robotics competition - The Journal News | LoHud.com - February 7th, 2017 [February 7th, 2017]
- Robotics Team Takes Home Trophy at Lego Competition - Patch.com - February 7th, 2017 [February 7th, 2017]
- OHS robotics club dazzles McKinley students - Southernminn.com - February 7th, 2017 [February 7th, 2017]
- Toyota Industries Acquires Warehouse Robotics Developer - Wall Street Journal - February 7th, 2017 [February 7th, 2017]
- South High wins Volvo's robotics tournament - Herald-Mail Media - February 7th, 2017 [February 7th, 2017]
- Belding hosts robotics competition - Sentinel-Standard - February 7th, 2017 [February 7th, 2017]
- CMS robotics program soars - SW News Media - February 7th, 2017 [February 7th, 2017]
- Update On The Megatrend of Robotics - CTOvision (blog) - February 7th, 2017 [February 7th, 2017]
- 33C3: Edible Soft Robotics - Hackaday - February 7th, 2017 [February 7th, 2017]
- Shopping robots on the march in Ocado - BBC News - February 7th, 2017 [February 7th, 2017]
- Keller students Starstruck on robotics - Fort Worth Star Telegram - February 7th, 2017 [February 7th, 2017]
- Our obsession with robots keeps getting creepier - New York Post - February 7th, 2017 [February 7th, 2017]
- Rethink Robotics makes its workplace bots easier to train and redeploy - TechCrunch - February 7th, 2017 [February 7th, 2017]
- Donation will cover costs for Sequim robotic competition - Peninsula Daily News - February 8th, 2017 [February 8th, 2017]
- Derby Middle School robotics club competes at high school competition - The Derby Informer - February 8th, 2017 [February 8th, 2017]
- Raspberry Pi-powered arm: This kit aims to make robotics simple enough for kids - TechRepublic - February 8th, 2017 [February 8th, 2017]
- WPI Robotics Team Has a Mission with Mars | News | WPI - WPI News - February 8th, 2017 [February 8th, 2017]
- Rehab robotics field promises to return control, mobility to aging population - ScienceBlog.com (blog) - February 8th, 2017 [February 8th, 2017]
- Rethink's Robots Get Massive Software Upgrade, Rodney Brooks So Excited - IEEE Spectrum - February 8th, 2017 [February 8th, 2017]
- Robotics: A robot that flies like a bat : Nature : Nature Research - Nature.com - February 9th, 2017 [February 9th, 2017]
- Robotics teams at CCS stay busy at tournaments - Imperial Republican - February 9th, 2017 [February 9th, 2017]
- Rethink Robotics rethinks its software | ZDNet - ZDNet - February 9th, 2017 [February 9th, 2017]
- Hartland robotics teams take on the FIRST Tech challenge at state competition - LC Sussex Sun - February 9th, 2017 [February 9th, 2017]
- Home-schooled students studying robotics - Valencia County News Bulletin - February 9th, 2017 [February 9th, 2017]
- Agility Robotics Introduces Cassie, a Dynamic and Talented Robot Delivery Ostrich - IEEE Spectrum - February 9th, 2017 [February 9th, 2017]
- Memphis robotics team headed to Super-Regionals - New Baltimore Voice Newspapers - February 10th, 2017 [February 10th, 2017]
- Patriot Robotics Alliance upsets Brentwood Academy Alliances - Clarksville Now - February 10th, 2017 [February 10th, 2017]
- Raptor legs & human hips: Giant leap for walking robots - RT - February 10th, 2017 [February 10th, 2017]
- How drones and robotics may shape the future of conflict under President Trump - PRI - February 10th, 2017 [February 10th, 2017]
- Robotics teams wins at state meet - The Citizen.com - February 10th, 2017 [February 10th, 2017]
- Ford Bets $1B on Startup Founded by Waymo, Uber Vets - ABC News - February 10th, 2017 [February 10th, 2017]
- South Beach robotics squad advances to super regionals - The Daily World - February 11th, 2017 [February 11th, 2017]
- Personal robotics the next technological revolution: Dr Vivian Balakrishnan - Channel NewsAsia - February 11th, 2017 [February 11th, 2017]
- This bipedal robot could deliver your packages one day - The Verge - The Verge - February 11th, 2017 [February 11th, 2017]
- Robotics event offers lesson in teamwork - Lewiston Morning Tribune (subscription) - February 12th, 2017 [February 12th, 2017]
- Essex County Freeholders Honor Livingston Robotics Club Teams - TAPinto.net - February 12th, 2017 [February 12th, 2017]
- College Town: WPI Robotics team aims for $1M NASA prize - Worcester Telegram - February 12th, 2017 [February 12th, 2017]
- Alice Shaw robotics team building invention to save wolves | Local ... - Lompoc Record - February 12th, 2017 [February 12th, 2017]
- Ford invests $1B in robotics startup in driverless car quest - CBS News - February 12th, 2017 [February 12th, 2017]
- Freshman Caldwell team takes first place in Regional Robotics Tournament - Idaho Press-Tribune - February 12th, 2017 [February 12th, 2017]
- What went seriously wrong with Lily Robotics - VentureBeat - February 12th, 2017 [February 12th, 2017]
- Face off: Marion students compete in robotics competition - The Exponent Telegram (press release) (registration) - February 12th, 2017 [February 12th, 2017]
- Robotics Competition slated for Monday - San Angelo Standard Times - February 13th, 2017 [February 13th, 2017]
- Cornerstone Prep robotics team brings home trophy in first year - MDJOnline.com - February 13th, 2017 [February 13th, 2017]
- Center Grove robotics team headed to state competition - Daily Journal - February 13th, 2017 [February 13th, 2017]
- Hundreds piled into a Fort Mill gym. They cheered. They competed. They drove robots. - The Herald - February 13th, 2017 [February 13th, 2017]
- My Turn: Arts education paved the way for girls' robotics win - AZCentral.com - February 13th, 2017 [February 13th, 2017]
- Young Alberta engineers face off in robotics showdown - CBC.ca - February 13th, 2017 [February 13th, 2017]
- Cougar robotics team advances to Super-Regionals - Times Bulletin - Times Bulletin - February 13th, 2017 [February 13th, 2017]
- South Carroll Robotics teams compete at Maryland State Robotics Competition - Carroll County Times - February 13th, 2017 [February 13th, 2017]
- Cave Spring High School's Robotics Club wins in the First Tech ... - Roanoke Times - February 13th, 2017 [February 13th, 2017]
- FIRST Robotics Competition - Wikipedia - February 13th, 2017 [February 13th, 2017]
- Designing robots to look more like ostriches can solve a key problem for our future biped overlords - Quartz - February 14th, 2017 [February 14th, 2017]
- More than 1000 high school students expected to Battlefield HS robotics tournament - PotomacLocal.com - February 14th, 2017 [February 14th, 2017]
- Event to look at Scotland's pivotal role in robotics - The Scotsman - February 14th, 2017 [February 14th, 2017]
- Kennett Coders to compete in NH State Robotics Championships on Saturday - Conway Daily Sun - February 14th, 2017 [February 14th, 2017]
- Are there enough robots? - Robotics Tomorrow (press release) - February 15th, 2017 [February 15th, 2017]
- Girl Scouts of Southern Illinois robotics team makes state tournament - Alton Telegraph - February 15th, 2017 [February 15th, 2017]
- Intuition Robotics raised $6 million for its ElliQ elder care assistant robot - TechCrunch - February 15th, 2017 [February 15th, 2017]
- Richmond robotics team gears up for districts - New Baltimore Voice Newspapers - February 15th, 2017 [February 15th, 2017]
- Microsoft lets you crash drones and robots in its new real world simulator - The Verge - February 15th, 2017 [February 15th, 2017]
- Makeblock's Lego-like 'Neuron' teaches kids robotics and code - Engadget - February 15th, 2017 [February 15th, 2017]
- West Ridge robotics team maneuvers into national championship - Austin American-Statesman - February 15th, 2017 [February 15th, 2017]
- Immokalee High team set for robotics state championships Friday - Naples Daily News - February 16th, 2017 [February 16th, 2017]
- What has Robotics got in store for our future? - RTE.ie - February 16th, 2017 [February 16th, 2017]
- Uber Is Becoming a Robotics Company, CEO Says - Yahoo News - February 16th, 2017 [February 16th, 2017]
- 3Doodler's new kits help kids craft their own robots - Engadget - Engadget - February 16th, 2017 [February 16th, 2017]
- Magnetic Control Could Help Robots Navigate Inside Your Body - IEEE Spectrum - February 16th, 2017 [February 16th, 2017]
- Boston's new hub, MassRobotics, is like a WeWork for robotics startups - TechCrunch - February 16th, 2017 [February 16th, 2017]
- 3Doodler announces a robotics kit, Star Trek and Powerpuff Girls ... - TechCrunch - February 17th, 2017 [February 17th, 2017]
- Chart: Are Robots Replacing High-Cost Workers? - Seeking Alpha - February 17th, 2017 [February 17th, 2017]