This article is part of our reviews of AI research papers, a series of posts that explore the latest findings in artificial intelligence.
Hundreds of millions of years of evolution have blessed our planet with a wide variety of lifeforms, each intelligent in its own fashion. Each species has evolved to develop innate skills, learning capacities, and a physical form that ensure its survival in its environment.
But despite being inspired by nature and evolution, the field of artificial intelligence has largely focused on creating the elements of intelligence separately and fusing them together after development. While this approach has yielded great results, it has also limited the flexibility of AI agents in some of the basic skills found in even the simplest lifeforms.
In a new paper published in the scientific journal Nature, AI researchers at Stanford University present a new technique that can help take steps toward overcoming some of these limits. Titled Deep Evolutionary Reinforcement Learning, the new technique uses a complex virtual environment and reinforcement learning to create virtual agents that can evolve both in their physical structure and learning capacities. The findings can have important implications for the future of AI and robotics research.
Credit: Ben Dickson / TechTalks
In nature, body and brain evolve together. Across many generations, every animal species has gone through countless cycles of mutation to grow limbs, organs, and a nervous system to support the functions it needs in its environment. Mosquitos have thermal vision to spot body heat. Bats have wings to fly and an echolocation apparatus to navigate dark places. Sea turtles have flippers to swim and a magnetic field detector system to travel very long distances. Humans have an upright posture that frees their arms and lets them see the far horizon, hands and nimble fingers that can manipulate objects, and a brain that makes them the best social creatures and problem solvers on the planet.
Interestingly, all these species descended from the first lifeform that appeared on Earth several billion years ago. Based on the selection pressures caused by the environment, the descendants of those first living beings evolved in many different directions.
Studying the evolution of life and intelligence is interesting. But replicating it is extremely difficult. An AI system that would want to recreate intelligent life in the same way that evolution did would have to search a very large space of possible morphologies, which is extremely expensive computationally. It would need a lot of parallel and sequential trial-and-error cycles.
AI researchers use several shortcuts and predesigned features to overcome some of these challenges. For example, they fix the architecture or physical design of an AI or robotic system and focus on optimizing the learnable parameters. Another shortcut is the use of Lamarckian rather than Darwinian evolution, in which AI agents pass on their learned parameters to their descendants. Yet another approach is to train different AI subsystems separately (vision, locomotion, language, etc.) and then tack them on together in a final AI or robotic system. While these approaches speed up the process and reduce the costs of training and evolving AI agents, they also limit the flexibility and variety of results that can be achieved.
Credit: Ben Dickson / TechTalks
In their new work, the researchers at Stanford aim to bring AI research a step closer to the real evolutionary process while keeping the costs as low as possible. Our goal is to elucidate some principles governing relations between environmental complexity, evolved morphology, and the learnability of intelligent control, they write in their paper.
Their framework is called Deep Evolutionary Reinforcement Learning. In DERL each agent uses deep reinforcement learning to acquire the skills required to maximize its goals during its lifetime. DERL uses Darwinian evolution to search the morphological space for optimal solutions, which means that when a new generation of AI agents are spawned, they only inherit the physical and architectural traits of their parents (along with slight mutations). None of the learned parameters are passed on across generations.
DERL opens the door to performing large-scale in silico experiments to yield scientific insights into how learning and evolution cooperatively create sophisticated relationships between environmental complexity, morphological intelligence, and the learnability of control tasks, the researchers write.
For their framework, the researchers used MuJoCo, a virtual environment that provides highly accurate rigid-body physics simulation. Their design space is called UNIversal aniMAL (UNIMAL), in which the goal is to create morphologies that learn locomotion and object-manipulation tasks in a variety of terrains.
Each agent in the environment is composed of a genotype that defines its limbs and joints. The direct descendant of each agent inherits the parents genotype and goes through mutations that can create new limbs, remove existing limbs, or make small modifications to characteristics such as the degrees of freedom or the size of limbs.
Each agent is trained with reinforcement learning to maximize rewards in various environments. The most basic task is locomotion, in which the agent is rewarded for the distance it travels during an episode. Agents whose physical structure are better suited for traversing terrain learn faster to use their limbs for moving around.
To test the systems results, the researchers generated agents in three types of terrains: flat (FT), variable (VT), and variable terrains with modifiable objects (MVT). The flat terrain puts the least selection pressure on the agents morphology. The variable terrains, on the other hand, force the agents to develop a more versatile physical structure that can climb slopes and move around obstacles. The MVT variant has the added challenge of requiring the agents to manipulate objects to achieve their goals.
One of the interesting findings of the DERL is the diversity of the results. Other approaches to evolutionary AI tend to converge on one solution because new agents directly inherit the physique and learnings of their parents. But in DERL, only morphological data is passed on to descendants, the system ends up creating a diverse set of successful morphologies, including bipeds, tripeds, and quadrupeds with and without arms.
At the same time, the system shows traits of the Baldwin effect, which suggests that agents that learn faster are more likely to reproduce and pass on their genes to the next generation. DERL shows that evolution selects for faster learners without any direct selection pressure for doing so, according to the Stanford paper.
Intriguingly, the existence of this morphological Baldwin effect could be exploited in future studies to create embodied agents with lower sample complexity and higher generalization capacity, the researchers write.
Finally, the DERL framework also validates the hypothesis that more complex environments will give rise to more intelligent agents. The researchers tested the evolved agents across eight different tasks, including patrolling, escaping, manipulating objects, and exploration. Their findings show that in general, agents that have evolved in variable terrains learn faster and perform better than AI agents that have only experienced flat terrain.
Their findings seem to be in line with another hypothesis by DeepMind researchers that a complex environment, a suitable reward structure, and reinforcement learning can eventually lead to the emergence of all kinds of intelligent behaviors.
The DERL environment only has a fraction of the complexities of the real world. Although DERL enables us to take a significant step forward in scaling the complexity of evolutionary environments, an important line of future work will involve designing more open-ended, physically realistic, and multi-agent evolutionary environments, the researchers write.
In the future, the researchers will expand the range of evaluation tasks to better assess how the agents can enhance their ability to learn human-relevant behaviors.
The work can have important implications for the future of AI and robotics and push researchers to use exploration methods that are much more similar to natural evolution.
We hope our work encourages further large-scale explorations of learning and evolution in other contexts to yield new scientific insights into the emergence of rapidly learnable intelligent behaviors, as well as new engineering advances in our ability to instantiate them in machines, the researchers write.
This article was originally published by Ben Dickson onTechTalks, a publication that examines trends in technology, how they affect the way we live and do business, and the problems they solve. But we also discuss the evil side of technology, the darker implications of new tech, and what we need to look out for. You can read the original articlehere.
Read the original:
AI is now learning to evolve like earthly lifeforms - The Next Web
- Days of our Lives' Suzanne Rogers on the Evolution of Maggie: "She Knows Who She Is Now, and She's Not Relying ... - Michael Fairman TV - March 14th, 2024 [March 14th, 2024]
- Kylie Jenner Talks About Her Style Evolution - The Cut - March 14th, 2024 [March 14th, 2024]
- Equator Coffees Unveils New Packaging Design, Reflecting Brand Evolution & Vision For The Future - Sprudge - March 14th, 2024 [March 14th, 2024]
- Rosewood Hotel Group Accelerates Growth And Evolution Across Its Four Distinctive Brands - Hospitality Net - March 14th, 2024 [March 14th, 2024]
- Thomson Reuters Unveils New Brand Evolution - Adweek - March 14th, 2024 [March 14th, 2024]
- Is It Becoming Acceptable to Speak of Design? - Discovery Institute - March 14th, 2024 [March 14th, 2024]
- Did Charles Darwin Convert to Christianity and Discredit Evolution on His Deathbed? - Snopes.com - March 14th, 2024 [March 14th, 2024]
- Milk, it's not just for mammals: An amphibian makes it too - NPR - March 14th, 2024 [March 14th, 2024]
- Discover Puerto Rico Debuts Evolution of Its Successful 'Live Boricua' Brand Campaign Aimed at Engaging Visitors ... - Yahoo Finance - March 14th, 2024 [March 14th, 2024]
- A Journey Through Time: The Evolution of Ras Al Khaimah Art - Business Wire - March 14th, 2024 [March 14th, 2024]
- Empowering Women: The Evolution and Innovation of coto Social Platform - CXOToday.com - March 14th, 2024 [March 14th, 2024]
- The Evolution of Da'Vine Joy Randolph - The Root - March 14th, 2024 [March 14th, 2024]
- Study on mating behaviors offers clues into the evolution of attraction - Phys.org - March 14th, 2024 [March 14th, 2024]
- Dragonball Evolutions live-action Goku says goodbye to Toriyama: Sorry we messed up - AS USA - March 14th, 2024 [March 14th, 2024]
- Investec, evolution of SMEs in the materials handling sector - Leasing Life - March 14th, 2024 [March 14th, 2024]
- Pride & Prejudice and the evolution of the female gaze on screen - Yahoo News UK - March 6th, 2024 [March 6th, 2024]
- Joe Wong's Musical Evolution - Shepherd Express - March 6th, 2024 [March 6th, 2024]
- A global survey of prokaryotic genomes reveals the eco-evolutionary pressures driving horizontal gene transfer - Nature.com - March 6th, 2024 [March 6th, 2024]
- Redefining Intelligence: Chimpanzees Break Through the Cultural Evolution Barrier - Medriva - March 6th, 2024 [March 6th, 2024]
- Mollusk Eyes Reveal How Future Evolution Depends on the Past - Quanta Magazine - March 6th, 2024 [March 6th, 2024]
- Levy Delves Into the Evolution of ADCs in NSCLC - OncLive - March 6th, 2024 [March 6th, 2024]
- The Snake Is The Spearhead of Reptile Evolution, But Why? - ScienceAlert - March 6th, 2024 [March 6th, 2024]
- 'A very special day: Birds linked to Darwins theory of evolution reintroduced to Galapagos Islands - Euronews - March 6th, 2024 [March 6th, 2024]
- Why the Powerhouses of Cells Evolve Differently in Plants - College of Natural Sciences - March 6th, 2024 [March 6th, 2024]
- Driving the DevOps Evolution: ArgoCD, Tekton and Seamless Migrations - DevOps.com - March 6th, 2024 [March 6th, 2024]
- Finding the Balance: The Evolution of Public Health Guidance Amidst Controversy - Medriva - March 6th, 2024 [March 6th, 2024]
- Insider Podcast: Paolini dishes on her Polish roots and hard-court evolution - WTA Tennis - March 6th, 2024 [March 6th, 2024]
- Interview: Sara Gruen and Rick Elice Talk About the Inspiration and Evolution of the New Musical Water for Elephants - TheaterMania.com - March 6th, 2024 [March 6th, 2024]
- The Evolution of the Laravel Welcome Page - Laravel News - March 6th, 2024 [March 6th, 2024]
- A Serpentine 'Explosion' 125 Million Years Ago Primed Snakes for Rapid, Diverse Evolution - Smithsonian Magazine - March 6th, 2024 [March 6th, 2024]
- The Evolution of Modern Technologies in Car Development - FinSMEs - March 6th, 2024 [March 6th, 2024]
- Milwaukee Transformed: From Bronzeville to Veterans Park, Aerial Timelapses Reveal City's Evolution - BNN Breaking - March 6th, 2024 [March 6th, 2024]
- The eyes are a gateway to evolution of daddy longlegs at least. - University of Wisconsin-Madison - March 6th, 2024 [March 6th, 2024]
- Adrian Newey: RB20 is the next step in Red Bull's design evolution - PlanetSport - March 6th, 2024 [March 6th, 2024]
- LiveScore releases its 'Evolution of Fan' report - Gambling Insider - March 6th, 2024 [March 6th, 2024]
- The loyalty program evolution makes its way to the full-service restaurant category - Nation's Restaurant News - March 6th, 2024 [March 6th, 2024]
- Teenage Mutant Ninja Turtles: The Last Ronin II - Re-Evolution #1 spoiler-free review: goes hard on the action, but ... - Gamesradar - March 6th, 2024 [March 6th, 2024]
- Exploring U.S. Financial Evolution: DAR Hosts Talk on Federal Reserve History in Thomasville - BNN Breaking - March 6th, 2024 [March 6th, 2024]
- Why cloud evolution needs a cohesive approach to succeed - CIO - March 6th, 2024 [March 6th, 2024]
- Gilead Sciences CEO on Company's Evolution and Commitment to the Bay Area - BioSpace - March 6th, 2024 [March 6th, 2024]
- Navigating the AI Quandary: Human Supremacy vs Machine Intelligence Evolution - BNN Breaking - March 6th, 2024 [March 6th, 2024]
- Denis Villeneuve breaks down the evolution of sandworms in 'Dune: Part Two' - Mashable - March 6th, 2024 [March 6th, 2024]
- Continued evolution of law improves governing capacity - Chinadaily.com.cn - China Daily - March 6th, 2024 [March 6th, 2024]
- The Evolution of the DEX Space with dYdX's CEO Antonio Juliano - Blockster - March 6th, 2024 [March 6th, 2024]
- Quick Commerce Evolution: 3PL Firms Aim for Same Day Delivery, Chasing Blinkit and Zepto's Lead - BNN Breaking - March 6th, 2024 [March 6th, 2024]
- What If...? Star Jeffrey Wright Addreses the Watcher's Evolution and 'Epic' Season 2 Finale - CBR - Comic Book Resources - December 31st, 2023 [December 31st, 2023]
- Evolution of the Connected Autonomous Vehicle - Ward's Auto - December 31st, 2023 [December 31st, 2023]
- A project to capture the evolution of human culture. - Psychology Today - December 31st, 2023 [December 31st, 2023]
- The Evolution of a Digital Soul. Beyond Code: A Journey of Heart and | by Mark Randall Havens | Dec, 2023 - Medium - December 31st, 2023 [December 31st, 2023]
- 4 Clues That Reid Is Finally Returning In Criminal Minds: Evolution Season 2 - Screen Rant - December 31st, 2023 [December 31st, 2023]
- Evolution of Samoyed and Kitten's Friendship Delights Internet: 'Wholesome' - Newsweek - December 31st, 2023 [December 31st, 2023]
- Crypto Evolution: Pullix (PLX) vs OKB (OKB) & KuCoin (KCS) - Crypto Reporter - December 31st, 2023 [December 31st, 2023]
- Alfa Romeos mediocre F1 season heralded its era of evolution: Prime Tire - The Athletic - December 31st, 2023 [December 31st, 2023]
- Beyond The Uniform: 10 Years of Evolution in SYNC Performance's Custom Program - SkiRacing.com - December 31st, 2023 [December 31st, 2023]
- Why SZA's evolution into a popstar has earned her recognition as artist of the year - Salon - December 31st, 2023 [December 31st, 2023]
- AI in 2023 Rises, Falls and Evolution - Finance Magnates - December 31st, 2023 [December 31st, 2023]
- Indonesia's Indosat pursues evolution from telecom to tech company - Nikkei Asia - December 31st, 2023 [December 31st, 2023]
- EdTech Evolution: 3 Stocks Educating the Next Generation - InvestorPlace - December 31st, 2023 [December 31st, 2023]
- Informa Tech Interview with Huawei about voice evolution and innovations at 5G Core Summit 2023 - Informa Tech ... - Light Reading - December 31st, 2023 [December 31st, 2023]
- Looking ahead: What will the DeFi evolution look like in 2024? - Ledger Insights - Ledger Insights - December 31st, 2023 [December 31st, 2023]
- Why Cat Bohannon wrote 'Eve, How the Female Body Drove 200 Million Years of Human Evolution' | India News ... - IndiaTimes - December 31st, 2023 [December 31st, 2023]
- The smart-design evolution of the laboratory space - pharmaphorum - December 31st, 2023 [December 31st, 2023]
- The WILD Evolution of Teenage Mutant Ninja Turtles TMNT (VIDEO) - FandomWire - December 31st, 2023 [December 31st, 2023]
- The supernatural invades American museums via indigenous artifacts - Why Evolution Is True - December 31st, 2023 [December 31st, 2023]
- Baleen Whales First Evolved Large Body Size in Cold Southern Waters, New Fossil Shows - Sci.News - December 31st, 2023 [December 31st, 2023]
- The Evolution of Identity in Taiwan The Diplomat - The Diplomat - December 31st, 2023 [December 31st, 2023]
- From the Archive: The Evolution Of Hockey Pools - The Hockey News - December 31st, 2023 [December 31st, 2023]
- 'X-Men: Evolution' Is Better Than 'X-Men: The Animated Series' - Collider - December 31st, 2023 [December 31st, 2023]
- Unveiling the Silver Screen: The Evolution of Celebrity Nudity in Cinema - The Hype Magazine - December 31st, 2023 [December 31st, 2023]
- Are Humans Still Evolving? 'Maybe More Rapidly Than Ever,' Says Scientist - Newsweek - December 31st, 2023 [December 31st, 2023]
- The Intersection of Real Estate and Fintech: Evolution, Impact of Policies, and Global Dynamics - CXOToday.com - December 31st, 2023 [December 31st, 2023]
- Kyle Richards' Style Evolution: Her Best Looks - Us Weekly - December 31st, 2023 [December 31st, 2023]
- Criminal Minds: Evolution Season 2's "Deeper Secrets" Teased By Aisha Tyler - Screen Rant - December 31st, 2023 [December 31st, 2023]
- Saturday: Hili dialogue Why Evolution Is True - Why Evolution Is True - December 31st, 2023 [December 31st, 2023]
- NBA 2K24 MyTEAM New Year Resolution Adds 14 Evolution Cards - ClutchPoints - December 31st, 2023 [December 31st, 2023]
- dive into the history of NASA's logo evolution from the space ... - Designboom - November 8th, 2023 [November 8th, 2023]
- Resolving the puzzle of same-sex sexual interactions - Nature.com - November 8th, 2023 [November 8th, 2023]
- The History and Evolution of Black Friday And How It Got Its Name - Yahoo Life - November 8th, 2023 [November 8th, 2023]
- Evolution of Terran R, with Tim Ellis (Relativity Space) - Payload - November 8th, 2023 [November 8th, 2023]
- Brownell Raves About Breakout Junior's Evolution - The Clemson Insider - November 8th, 2023 [November 8th, 2023]