It has been observed by many that we are at the dawn of the next industrial revolution: The Artificial Intelligence (AI) revolution. The benefits delivered by this intelligence revolution will be many: in medicine, improved diagnostics and precision treatment, better weather forecasting, and self-driving vehicles to name a few. However, one of the costs of this revolution is going to be increased electrical consumption by the data centers that will power it. Data center power usage is projected to double over the next 10 years and is on track to consume 11% of worldwide electricity by 2030. Beyond AI adoption, other drivers of this trend are the movement to the cloud and increased power usage of CPUs, GPUs and other server components, which are becoming more powerful and smart.
AIs two basic elements, training and inference, each consume power differently. Training involves computationally intensive matrix operations over very large data sets, often measured in terabytes to petabytes. Examples of these data sets can range from online sales data to captured video feeds to ultra-high-resolution images of tumors. AI inference is computationally much lighter in nature, but can run indefinitely as a service, which draws a lot of power when hit with a large number of requests. Think of a facial recognition application for security in an office building. It runs continuously but would stress the compute and storage resources at 8:00am and again at 5:00pm as people come and go to work.
However, getting a good handle on power usage in AI is difficult. Energy consumption is not part of standard metrics tracked by job schedulers and while it can be set up, it is complicated and vendor dependent. This means that most users are flying blind when it comes to energy usage.
To map out AI energy requirements, Dr. Miro Hodak led a team of Lenovo engineers and researchers, which looked at the energy cost of an often-used AI workload. The study, Towards Power Efficiency in Deep Learning on Data Center Hardware, (registration required) was recently presented at the 2019 IEEE International Conference on Big Data and was published in the conference proceedings. This work looks at the energy cost of training ResNet50 neural net with ImageNet dataset of more than 1.3 million images on a Lenovo ThinkSystem SR670 server equipped with 4 Nvidia V100 GPUs. AC data from the servers power supply, indicates that 6.3 kWh of energy, enough to power an average home for six hours, is needed to fully train this AI model. In practice, trainings like these are repeated multiple times to tune the resulting models, resulting in energy costs that are actually several times higher.
The study breaks down the total energy into its components as shown in Fig. 1. As expected, the bulk of the energy is consumed by the GPUs. However, given that the GPUs handle all of the computationally intensive parts, the 65% share of energy is lower than expected. This shows that simplistic estimates of AI energy costs using only GPU power are inaccurate and miss significant contributions from the rest of the system. Besides GPUs, CPU and memory account for almost quarter of the energy use and 9% of energy is spent on AC to DC power conversion (this is within line of 80 PLUS Platinum certification of SR670 PSUs).
The study also investigated ways to decrease energy cost by system tuning without changing the AI workload. We found that two types of system settings make most difference: UEFI settings and GPU OS-level settings. ThinkSystem servers provides four UEFI running modes: Favor Performance, Favor Energy, Maximum Performance and Minimum Power. As shown in Table 1, the last option is the best and provides up to 5% energy savings. On the GPU side, 16% of energy can be saved by capping V100 frequency to 1005 MHz as shown in Figure 2. Taking together, our study showed that system tunings can decrease energy usage by 22% while increasing runtime by 14%. Alternatively, if this runtime cost is unacceptable, a second set of tunings, which save 18% of energy while increasing time by only 4%, was also identified. This demonstrates that there is lot of space on system side for improvements in energy efficiency.
Energy usage in HPC has been a visible challenge for over a decade, and Lenovo has long been a leader in energy efficient computing. Whether through our innovative Neptune liquid-cooled system designs, or through Energy-Aware Runtime (EAR) software, a technology developed in collaboration with Barcelona Supercomputing Center (BSC). EAR analyzes user applications to find optimum CPU frequencies to run them at. For now, EAR is CPU-only, but investigations into extending it to GPUs are ongoing. Results of our study show that that is a very promising way to bring energy savings to both HPC and AI.
Enterprises are not used to grappling with the large power profiles that AI requires, the way HPC users have become accustomed. Scaling out these AI solutions will only make that problem more acute. The industry is beginning to respond. MLPerf, currently the leading collaborative project for AI performance evaluation, is preparing new specifications for power efficiency. For now, it is limited to inference workloads and will most likely be voluntary, but it represents a step in the right direction.
So, in order to enjoy those precise weather forecasts and self-driven cars, well need to solve the power challenges they create. Today, as the power profile of CPUs and GPUs surges ever upward, enterprise customers face a choice between three factors: system density (the number of servers in a rack), performance and energy efficiency. Indeed, many enterprises are accustomed to filling up rack after rack with low cost, adequately performing systems that have limited to no impact on the electric bill. Unfortunately, until the power dilemma is solved, those users must be content with choosing only two of those three factors.
Read this article:
Powering the Artificial Intelligence Revolution - HPCwire
- Researchers create innovative verification techniques to increase security in artificial intelligence and image processing - European Research Council - April 29th, 2024 [April 29th, 2024]
- Who in Europe is investing the most in artificial intelligence? - Euronews - April 29th, 2024 [April 29th, 2024]
- 4 Artificial Intelligence (AI) Stocks Members of Congress Can't Stop Buying (and Nvidia Isn't 1 of Them!) - The Motley Fool - April 29th, 2024 [April 29th, 2024]
- The Standard Names Porter Orr Second Vice President of Artificial Intelligence Strategy and Development - NRToday.com - April 29th, 2024 [April 29th, 2024]
- Meet Nvidia CEO Jensen Huang, the man behind the $2 trillion company powering today's artificial intelligence - CBS News - April 29th, 2024 [April 29th, 2024]
- This Warren Buffett Dividend King Stock Just Invested $1.1 Billion Into Artificial Intelligence (AI) - The Motley Fool - April 29th, 2024 [April 29th, 2024]
- Joe Rogan Reveals What Could Be 'Game Over for the Human Race' - Newsweek - April 29th, 2024 [April 29th, 2024]
- LegalTech Artificial Intelligence Market to Soar to $6.4 Billion by 2028: Global Long-term Forecast to 2033 - Strategic ... - Yahoo Finance UK - April 29th, 2024 [April 29th, 2024]
- Artificial Intelligence Tokens Stumble Again! Is The AI Hype Over? - Coinpedia Fintech News - April 29th, 2024 [April 29th, 2024]
- CISA unveils guidelines for AI and critical infrastructure - FedScoop - April 29th, 2024 [April 29th, 2024]
- IBM's Webinar: Cybersecurity In The Era Of Artificial Intelligence Keynotes - AiThority - April 29th, 2024 [April 29th, 2024]
- Being and becoming a good doctor in the age of artificial intelligence - Firstpost - April 29th, 2024 [April 29th, 2024]
- Pope to take part in G7 summit in June to talk Artificial Intelligence - Crux Now - April 29th, 2024 [April 29th, 2024]
- Pope Francis to attend G7 summit to speak on artificial intelligence - Catholic News Agency - April 29th, 2024 [April 29th, 2024]
- 1 Unstoppable Artificial Intelligence (AI) Stock to Buy and Hold Forever - Yahoo Finance - April 29th, 2024 [April 29th, 2024]
- Pope to bring his call for ethical artificial intelligence to G7 summit in June in southern Italy - The Associated Press - April 29th, 2024 [April 29th, 2024]
- Elon Musk Called Tesla an Artificial Intelligence (AI) Robotics Company. Does That Make It a Buy? - The Motley Fool - April 29th, 2024 [April 29th, 2024]
- Emulating neurodegeneration and aging in artificial intelligence systems - Tech Xplore - April 29th, 2024 [April 29th, 2024]
- Pope Francis to participate in G7 session on AI - Vatican News - English - April 29th, 2024 [April 29th, 2024]
- A Once-in-a-Generation Investment Opportunity: 1 Bill Ackman Artificial Intelligence (AI) Stock to Buy Hand Over Fist ... - Yahoo Finance - April 29th, 2024 [April 29th, 2024]
- Tesla Stock Investors Cheer Progress on Its Artificial Intelligence (AI) Future - Yahoo Finance - April 29th, 2024 [April 29th, 2024]
- In Race to Build A.I., Tech Plans a Big Plumbing Upgrade - The New York Times - April 29th, 2024 [April 29th, 2024]
- A Baltimore-area teacher is accused of using AI to make his boss appear racist - NPR - April 29th, 2024 [April 29th, 2024]
- 3 Top Artificial Intelligence (AI) Stocks That Billionaires Jim Simons, Ray Dalio, and Israel Englander Are Buying - The Motley Fool - April 29th, 2024 [April 29th, 2024]
- The iPad Pro can give us a big surprise with its artificial intelligence chips. - Softonic EN - April 29th, 2024 [April 29th, 2024]
- This Artificial Intelligence (AI) Stock Could Soar 70%, According to Wall Street. Time to Buy? - Yahoo Finance - April 29th, 2024 [April 29th, 2024]
- Billionaire Bill Ackman Owns 8 Stocks -- and This Hypergrowth Artificial Intelligence (AI) Stock Isn't One of Them - The Motley Fool - April 29th, 2024 [April 29th, 2024]
- Artificial Intelligence Has Come for Our...Beauty Pageants? - Glamour - April 29th, 2024 [April 29th, 2024]
- 3 Stocks to Grab Now to Ride the Artificial Intelligence Chip Boom to Riches - InvestorPlace - April 29th, 2024 [April 29th, 2024]
- Is ASML's Big Sell-off a Warning Sign to Artificial Intelligence (AI) Investors? - The Motley Fool - April 29th, 2024 [April 29th, 2024]
- 3 Artificial Intelligence (AI) Stocks to Buy With $1,150 and Hold for Decades - Yahoo Finance - March 22nd, 2024 [March 22nd, 2024]
- This combines the power of artificial intelligence with human insights - finews.com - March 22nd, 2024 [March 22nd, 2024]
- Cohere Targets $5 Billion Valuation for ChatGPT Rival - PYMNTS.com - March 22nd, 2024 [March 22nd, 2024]
- IMF: Artificial Intelligence (AI) Will Transform 40% of Jobs. Can Investors Capitalize? - Yahoo Finance - March 22nd, 2024 [March 22nd, 2024]
- Cathie Wood Is Selling These 2 Artificial Intelligence (AI) Stocks. Should You? - Yahoo Finance - March 22nd, 2024 [March 22nd, 2024]
- iShares Robotics and Artificial Intelligence Multisector ETF (NYSEARCA:IRBO) Shares Acquired by Creative Financial ... - Defense World - March 22nd, 2024 [March 22nd, 2024]
- Inaugural Plenary Meeting of States Endorsing the Political Declaration on Responsible Military Use of Artificial ... - Department of State - March 22nd, 2024 [March 22nd, 2024]
- AMD Fell Today -- Is This a Chance to Buy the Artificial Intelligence (AI) Stock? - Yahoo Finance - March 22nd, 2024 [March 22nd, 2024]
- Artificial Intelligence (AI) Stocks Are Red-Hot, but Here's 1 to Avoid (for Now) - The Motley Fool - March 22nd, 2024 [March 22nd, 2024]
- A Once-in-a-Generation Investment Opportunity: 1 Artificial Intelligence (AI) Growth Stock to Buy and Hold Forever - Yahoo Finance - March 22nd, 2024 [March 22nd, 2024]
- 2 Artificial Intelligence (AI) Stocks That Could Go Parabolic - The Motley Fool - March 22nd, 2024 [March 22nd, 2024]
- Artificial intelligence will radically improve health care, but only if managed carefully - The Hill - March 22nd, 2024 [March 22nd, 2024]
- Tennessee Makes A.I. an Outlaw to Protect Its Country Music and More - The New York Times - March 22nd, 2024 [March 22nd, 2024]
- When Using Artificial Intelligence In Pharma R&D, Start With Identifying Problem To Solve - HBW Insight - March 22nd, 2024 [March 22nd, 2024]
- Why Super Micro Computer, Advanced Micro Devices, and Other Artificial Intelligence (AI) Stocks Tumbled on Tuesday - Yahoo Finance - March 22nd, 2024 [March 22nd, 2024]
- 3 Billionaires Are Selling Artificial Intelligence (AI) Stock Nvidia and Buying These 10 AI Stocks Instead - The Motley Fool - March 22nd, 2024 [March 22nd, 2024]
- Nvidia Just Bought 5 Artificial Intelligence (AI) Stocks. These 2 Stand Out the Most. - The Motley Fool - March 22nd, 2024 [March 22nd, 2024]
- UN adopts first global artificial intelligence resolution - ARMENPRESS - March 22nd, 2024 [March 22nd, 2024]
- This New Artificial Intelligence (AI) Chip Is a Massive Game Changer for Nvidia Stock - The Motley Fool - March 22nd, 2024 [March 22nd, 2024]
- Nvidia Led the First Phase of Artificial Intelligence (AI), but These 2 Growth Stocks Will Lead the Next Phases ... - The Motley Fool - March 22nd, 2024 [March 22nd, 2024]
- Meet Wall Street's Newest Stock-Split Stock, Along With the Artificial Intelligence (AI) Stock Likeliest to Follow in Its ... - The Motley Fool - March 22nd, 2024 [March 22nd, 2024]
- Amid Rumors of a Deal With Rivian, Apple Acquired This Artificial Intelligence (AI) Start-Up Instead - Yahoo Finance - March 22nd, 2024 [March 22nd, 2024]
- UN adopts first global artificial intelligence resolution - CGTN - March 22nd, 2024 [March 22nd, 2024]
- 1 No-Brainer Artificial Intelligence (AI) Stock to Buy With $25 and Hold for 10 Years - The Motley Fool - March 22nd, 2024 [March 22nd, 2024]
- 2 Artificial Intelligence (AI) Stocks That Could Make You a Millionaire - The Motley Fool - March 22nd, 2024 [March 22nd, 2024]
- Is Intel the Best Artificial Intelligence (AI) Semiconductor Stock to Buy Before It Skyrockets? - The Motley Fool - February 20th, 2024 [February 20th, 2024]
- The greater social implications of artificial intelligence - New Day NW - KING5.com - February 20th, 2024 [February 20th, 2024]
- AI comes to the world of beauty as eyelash robot uses artificial intelligence to place fake lashes - Fox News - February 20th, 2024 [February 20th, 2024]
- Beyer Appointed To Bipartisan Task Force On Artificial Intelligence - Falls Church News Press - February 20th, 2024 [February 20th, 2024]
- Why Did Nvidia Invest in These 5 Artificial Intelligence (AI) Stocks? Should You Buy Them, Too? - The Motley Fool - February 20th, 2024 [February 20th, 2024]
- Three key themes on artificial intelligence - Research Information - February 20th, 2024 [February 20th, 2024]
- How Artificial Intelligence is transforming consumerism - WFLA - February 20th, 2024 [February 20th, 2024]
- SAP names Philipp Herzig as chief artificial intelligence officer - CIO - February 20th, 2024 [February 20th, 2024]
- Unleashing the Power of Artificial Intelligence: Transforming Web-Based Applications for Enhanced Efficiency and User ... - Financialbuzz.com - February 20th, 2024 [February 20th, 2024]
- Koch Industries continues to accelerate its artificial intelligence initiative - The Business Journals - February 20th, 2024 [February 20th, 2024]
- Forget Nvidia: These 3 Artificial Intelligence (AI) Stocks Can Be the Next Stock-Split Stocks - The Motley Fool - February 20th, 2024 [February 20th, 2024]
- Artificial Intelligence for small business focus of upcoming JWCC Lunch and Learn on Feb. 28 Muddy River News - Muddy River News - February 20th, 2024 [February 20th, 2024]
- Opponents Highlight the Environmental Impact of Artificial Intelligence - News-Press Now - February 20th, 2024 [February 20th, 2024]
- Generative AI's environmental costs are soaring and mostly secret - Nature.com - February 20th, 2024 [February 20th, 2024]
- Chapter Summary: Genesis of Artificial Intelligence and a Scientific Revolution: 1950-1979 - EIN News - February 20th, 2024 [February 20th, 2024]
- What would Thomas Aquinas make of Artificial Intelligence? - ACI Africa - February 20th, 2024 [February 20th, 2024]
- ChatGPT Predicted Bitcoin Price Will "Skyrocket" - Cryptonews - February 20th, 2024 [February 20th, 2024]
- Worried About an Artificial Intelligence (AI) Stock Bubble? Consider This Billionaire Investor's Advice. - Yahoo Finance - February 20th, 2024 [February 20th, 2024]
- This Super Artificial Intelligence (AI) Stock Could Be at the Beginning of a Terrific Bull Run - Yahoo Finance - February 20th, 2024 [February 20th, 2024]
- 5 Artificial Intelligence (AI) Stocks That Could Make You a Millionaire - Yahoo Finance - February 20th, 2024 [February 20th, 2024]
- Will AI replace Colorado teachers? Here's what experts say. - The Colorado Sun - February 20th, 2024 [February 20th, 2024]
- Nvidia Could Be About to Counter a Big Artificial Intelligence (AI) Threat With This Move - Yahoo Finance - February 20th, 2024 [February 20th, 2024]
- The Healthiest U.S. Pharma Companies: Ranked by RealRate's Impressive Artificial Intelligence - Medium - February 20th, 2024 [February 20th, 2024]
- Down 84%, Is This Artificial Intelligence (AI) Stock a Buy After an Earnings Pop? - The Motley Fool - February 20th, 2024 [February 20th, 2024]
- 'AI for Humans' may be the most entertaining way to learn about artificial intelligence - Fast Company - February 20th, 2024 [February 20th, 2024]