With the third round of MLPerf benchmarking results coming out, graphic giant, NVIDIA announced breaking AI performance records becoming the fastest products available commercially for AI training. However, on the other hand, Google has also proclaimed acing the MLPerf tests with the worlds fastest training supercomputer.
Although both companies have showcased significant achievements in creating faster training processes of ML models, which indeed would be critical for research breakthroughs, there is a hidden element that needs to be paid attention to.
Similar to System Performance Evaluation Consortium benchmark and Transaction Processing Council benchmark, MLPerf is an industry-standard benchmark that has been designed to measure the time to train ML models on a few specific tasks. Comprising some 80 companies and universities from all over the world, some other prominent vendors competing in the MLPerf benchmark tests were Intels Xeon processors and Huawei.
Also Read: How MLPerf Will Propel A New Benchmarking Era For Machine Learning
The current result announced by MLPerf was similar to its last two results where NVIDIA stood firm in the top position for all 16 benchmarks with its commercially available hardware and software products for a variety of ML tasks. However, on the other hand, Googles Tensor Processing Unit has surpassed NVIDIAs results in most tasks for the category of research projects.
In a recent tweet, Google AI lead, Jeff Dean shared his excitement on setting records in six out of eight benchmarks in MLPerf tests. According to the results Google topped in training DLRM, Transformer, BERT, SSD, ResNet-50 and Mask R-CNN leveraging its new machine learning supercomputer and TPU chip. In fact, it is believed that Googles supercomputer is 4x larger than Googles Cloud TPU v3 Pod, which made records in the previous results, becoming the first cloud provider to outperform on-premise systems.
While Google has powered up its Cloud TPUs providing faster training time for ML models, NVIDIAs advanced GPU A100 also proved to execute five petaflops of performance excelling on all eight benchmarks of MLPerf. A100 has been the first processor designed on NVIDIAs Ampere architecture, which allows the GPU to address different sized acceleration needs from small to big multi-node workload.
Strangely enough, realising the advanced capabilities of A100 that could easily dominate the field, the only companies to put their submission for commercially available servers were Google and Huawei, that also for only for two categories image classification and natural language processing. Thereby, NVIDIAs results remained high in system design and training models beating Huawei, Googles TPU as well as Intel, which has recently switched to Habanas AI chips.
According to the results, it took about 49 seconds for NVIDIAs DGX A100 to train BERT, which is better than Google with 57 minutes. Understanding its position, Google, therefore submitted its V4 chip in the research category to build its position for the same, with an astounding performance for many training tasks. With that being said, the company hasnt yet released it on the Google Cloud, which again can put it way behind in the race against A100, which is commercially in production.
Also Read: TPU Vs GPU Vs CPU: Which Hardware Should You Choose For Deep Learning
These pointers above highlight how NVIDIA single-handedly dominates the commercial category with other vendor companies like Dell EMC, Alibaba, and even Google using A100 for submitting their performance results. However, even though Googles TPU will still not be available in the market for some time, it indeed showcased intensive performance for many tasks in MLPerf. Interestingly, Intel has also joined the force with its soon-to-be-released CPU, however too early to predict its success in applications.
Home NVIDIA Claims To Have Won MLPerf Benchmarking, But Google Says Otherwise
On another note, MLPerf added a new test to reflect upon the growing application of machine learning in production settings Deep Learning Recommendation Model. Although NVIDIA has performed brilliantly on the newly added benchmarking test, it might still lag in building recommendation engines due to the requirement of a massive amount of supercomputing memory. Google, on the other hand, with its supercomputing abilities, has recently launched a beta version of Recommendation AI, making building recommendation engines super easy for developers.
Having said that, even though MLPerf assesses almost all aspects of AI performance, two vital parameters left out in this benchmarking test are the comparison of the price of chips and computers, and the consumption of energy, which are critical for todays era. It is believed that machines with better accuracy and more chips are going to consume more energy and are going to be heavier on the pockets, which might put Google on the lead once again.
With all that information in hand, it can be established that a usual trend of bigger hardware is indeed gaining traction, which will drastically reduce the amount of training time of machine learning models. However, NVIDIA cannot be the only one dominating the market. Googles supercomputer also showed tremendous results in training ML models to perform tasks.
While NVIDIA and Google are still going to continue the race of achieving the highest AI performance record, NVIDIA stated in their company blog, the real winners are the customers who will now be able to leverage this advancement to transform their businesses faster with AI.
Check full results here.
- As it closes in on Arm, Nvidia announces UK supercomputer dedicated to medical research - TechCrunch - October 8th, 2020
- With Crossroads Supercomputer, HPE Notches Another DOE Win - The Next Platform - October 8th, 2020
- What happens when two planets crash together? This supercomputer has the answer - Digital Trends - October 8th, 2020
- Supermicro Details Its Hardware for MN-3, the Most Efficient Supercomputer in the World - HPCwire - September 2nd, 2020
- I confess, I'm scared of the next generation of supercomputers - TechRadar - September 2nd, 2020
- Bradykinin Hypothesis of COVID-19 Offers Hope for Already-Approved Drugs - BioSpace - September 2nd, 2020
- Stranger than fiction? Why we need supercomputers - TechHQ - September 2nd, 2020
- Google Says It Just Ran The First-Ever Quantum Simulation of a Chemical Reaction - ScienceAlert - September 2nd, 2020
- This Equation Calculates the Chances We Live in a Computer Simulation - Discover Magazine - September 2nd, 2020
- 17 of the best computers and supercomputers to grace the planet - Pocket-lint - August 31st, 2020
- Supercomputer finds best way to air out classroom to ward off virus : The Asahi Shimbun - Asahi Shimbun - August 31st, 2020
- The Supercomputer Breaking Online Gaming Records and Modeling COVID-19 - BioSpace - August 31st, 2020
- When it comes to hurricane models, which one is best? - KHOU.com - August 31st, 2020
- Natural Radiation Including Cosmic Rays From Outer Space Can Wreak Havoc With Quantum Computers - SciTechDaily - August 31st, 2020
- The Tech Field Failed a 25-Year Challenge to Achieve Gender Equality by 2020 Culture Change Is Key to Getting on Track - Nextgov - August 31st, 2020
- Cerebras Systems Expands Global Footprint with Toronto Office Opening - HPCwire - August 31st, 2020
- CSC's Supercomputer Mahti is Now Available to Researchers and Students - HPCwire - August 28th, 2020
- Here's the smallest AI/ML supercomputer ever - TechRadar - August 28th, 2020
- When it comes to hurricane models, which one is best? - 12newsnow.com KBMT-KJAC - August 28th, 2020
- SberCloud's Cloud Platform Sweeps Three International Accolades At IT World Awards - Exchange News Direct - August 28th, 2020
- Supercomputer Market Growth, Future Prospects And Competitive Analysis (2020-2026) - Bulletin Line - August 28th, 2020
- A continent works to grow its stake in quantum computing - University World News - August 28th, 2020
- Supercomputer predicts where Spurs will finish in the 2020/21 Premier League table - The Spurs Web - August 28th, 2020
- Has the world's most powerful computer arrived? - The National - August 28th, 2020
- Galaxy Simulations Could Help Reveal Origins of Milky Way - Newswise - August 28th, 2020
- ALCC Program Awards Computing Time on ALCF's Theta Supercomputer to 24 projects - HPCwire - August 10th, 2020
- A Quintillion Calculations a Second: DOE Calculating the Benefits of Exascale and Quantum Computers - SciTechDaily - August 10th, 2020
- GE plans to give offshore wind energy a supercomputing boost - The Verge - August 10th, 2020
- From WarGames to Terms of Service: How the Supreme Courts Review of Computer Fraud Abuse Act Will Impact Your Trade Secrets - JD Supra - August 10th, 2020
- New Audis To Use Supercomputer That Controls Almost Everything - Motor1 - August 10th, 2020
- Japanese supercomputer ranked as worlds most powerful system - August 10th, 2020
- Top 10 Supercomputers | HowStuffWorks - August 10th, 2020
- What are supercomputers currently used for? | HowStuffWorks - August 10th, 2020
- Microsoft announces new supercomputer, lays out vision for ... - August 10th, 2020
- GE taps into US supercomputer to advance offshore wind - reNEWS - August 10th, 2020
- Summit supercomputer to advance research on wind power for renewable energy - ZDNet - August 10th, 2020
- BSC Researchers Create Spin-Off Platform to Accelerate the Development of New Chemicals - HPCwire - August 10th, 2020
- Supercomputer study of mobility in Spain at the peak of COVID-19 using Facebook and Google data - Science Business - August 9th, 2020
- Julia and PyCaret Latest Versions, arXiv on Kaggle, UK's AI Supercomputer And More In This Week's Top AI News - Analytics India Magazine - August 9th, 2020
- Audi To Over-Complicate Cars With Supercomputers And Repair Costs Could Skyrocket - Top Speed - August 9th, 2020
- Supercomputer COVID-19 insights, ionic spiderwebs, the whiteness of AI TechCrunch - Best gaming pro - August 8th, 2020
- Every Superman Movie Climax, Ranked From Worst To Best - Screen Rant - August 8th, 2020
- Atos Partners with University of Oxford on Largest AI Supercomputer in the UK - HPCwire - August 7th, 2020
- Five Movies Worth Watching About the Threat of Nuclear War - Council on Foreign Relations - August 7th, 2020
- Break it down: A new way to address common computing problem - Washington University in St. Louis Newsroom - August 7th, 2020
- GE Research uses summit supercomputer for study on wind power - Windtech International - August 7th, 2020
- Research: A Survey of Numerical Methods Utilizing Mixed Precision Arithmetic - HPCwire - August 7th, 2020
- Atos signs 5m supercomputing deal to support Oxford University-led AI research push - ComputerWeekly.com - August 6th, 2020
- Atos partners with University of Oxford on largest AI supercomputer in the UK - Yahoo Finance - August 6th, 2020
- How coronavirus antibody testing works - Livemint - August 6th, 2020
- Researchers Use Supercomputers To Discover New Pathway For Covid-19 Inflammation - Forbes - August 6th, 2020
- Supercomputer-Powered Research Uncovers Signs of 'Bradykinin Storm' That May Explain COVID-19 Symptoms - HPCwire - July 31st, 2020
- Celtic and Rangers title race outcome predicted by betting supercomputer - HeraldScotland - July 31st, 2020
- Nvidia reportedly in advanced talks to buy Arm - ZDNet - July 31st, 2020
- Continental is supercharging the development of driver-assistance tech - CNET - July 31st, 2020
- COVID-19 Pandemic Can Help More of Us Learn About Climate Change - UT News | The University of Texas at Austin - July 31st, 2020
- From rocks to icebergs, the natural world tends to break into cubes - Science Magazine - July 31st, 2020
- New Data on Genetic Expression In Severe COVID-19, Pre-Existing Immune Response - Bio-IT World - July 31st, 2020
- Superman's Glasses Are Secretly Used For Mind Control - Screen Rant - July 31st, 2020
- The Israeli company that has come as close as possible to the sun - Haaretz.com - July 31st, 2020
- Celtic and Rangers title race outcome predicted by betting supercomputer - Glasgow Times - July 31st, 2020
- PEARC20 Plenary Introduces Five Upcoming NSF-Funded HPC Systems - HPCwire - July 31st, 2020
- NIH Awards $6M to UConn Health Biological Computer Modeling Teams - HPCwire - July 31st, 2020
- Continental Debuts the Fastest Supercomputer in the Automotive Industry and It's Built for AI - EnterpriseAI - July 30th, 2020
- WATCH: Supercomputer generates 3D videos that show how Earth may have lost half of its atmosphere to create th - Business Insider India - July 26th, 2020
- Repeated intelligence failures: Time to worry - The Sunday Guardian - July 26th, 2020
- Supercomputer Market to witness an impressive growth during the forecast period 2020 - 2026 - CueReport - July 26th, 2020
- What is supercomputer? - Definition from WhatIs.com - July 26th, 2020
- What is supercomputer? - Definition - July 26th, 2020
- Impact of Covid-19 on Supercomputer Market Comprehensive Growth 2020-2027 with Top key vendor IBM Corporation, Cray Inc., Lenovo Inc., Sugon, Inspur -... - July 25th, 2020
- Super-computer Henry Cavill breaks the Internet again with more geek content - KSRO - July 24th, 2020
- How Equity Is Lost When Companies Hire Only Workers With Disabilities - The New York Times - July 24th, 2020
- Earth may have lost half of its atmosphere to create Moon, reveals study using 3D videos - Republic World - Republic World - July 24th, 2020
- "Super"-computer: Henry Cavill breaks the Internet again with more geek content - wcsjnews.com - July 24th, 2020
- Solar Opposites EPs Tease Whats To Come On Season 2 Of Hulu Animated Comedy Comic-Con@Home - Deadline - July 24th, 2020
- NVIDIA and University of Florida Release New AI Curriculum Spanning All Educational Disciplines - Motley Fool - July 21st, 2020
- Supercomputing Pipeline Aids DESI's Quest to Create 3D Map of the Universe - HPCwire - July 21st, 2020
- "Super"-computer Henry Cavill breaks the Internet again with more geek content - 1310kfka.com - July 20th, 2020
- It's time to decide what we want Downriver's future to look like - Southgate News Herald - July 20th, 2020
- The worlds supercomputers joined forces against COVID-19 why such collaborations are critical for tackling future emergencies - The European Sting - July 20th, 2020