Exabyte, a materials discovery cloud specialist, has published a study that compares Linpack performance on four of the largest public cloud providers. Although the studys methodology had some drawbacks, the results suggested that with the right hardware, HPC applications could not only scale well in cloud environments, but could also deliver performance on par with that of conventional supercomputers.
Overall, HPC practitioners have resisted using cloud computing for a variety of reasons, one of the more significant being the lack of performant hardware available in cloud infrastructure. Cluster network performance, in particular, has been found wanting in generic clouds, since conventional Ethernet, both GigE and 10GigE, do not generally have the bandwidth and latency characteristics to keep up with MPI applications running on high core-count nodes. As well see in a moment, it was the network that seemed to matter most in terms of scalability for these cloud environments.
The Exabyte study used high performance Linpack (HPL) as the benchmark metric, measuring its performance on four of the most widely used public clouds in the industry: Amazon Web Service (AWS), Microsoft Azure, IBM SoftLayer, and Rackspace. (Not coincidentally Exabyte, a cloud service provider for materials design, device simulations, and computational chemistry, employs AWS, Azure, SoftLayer and Rackspace as the infrastructure of choice for its customers.) Linpack was measured on specific instances of these clouds to determine benchmark performance across different cluster sizes and its efficiency in scaling from 1 to 32 nodes. The results were compared to those on Edison, a 2.5 petaflop (peak) NERSC supercomputer built by Cray. It currently occupies the number 60 spot on the TOP500 rankings.
To keep the benchmark results on as level a playing field as possible, it looks like the Exabyte team tried to use the same processor technology across systems, in this case, Intel Xeon processors of the Haswell or Ivy Bridge generation. However, the specific hardware profile clock speed, core count, and RAM capacity -- varied quite a bit across the different environments. As it turns out though, the system network was the largest variable across the platforms. The table below shows the node specification for each environment.
Source: Exabyte Inc.
As might be expected, Edison was able to deliver very good results, with a respectable 27-fold speedup as the Linpack run progressed from 1 to 32 nodes, at which point 10.44 teraflops of performance was achieved. That represents decent scalability and is probably typical of a system with a high-performance interconnect, in this case Crays Aries network. Note that Edison had the highest core count per node (48), but one of the slower processor clocks (2.4 GHz) of the environments tested.
The AWS cloud test used the c4.8xlarage instance, but was measured in three different ways: one with hyperthreading enabled, one with hyperthreading disabled, and one with hyperthreading disabled and with the node placements optimized to minimize network latency and maximize bandwidth. The results didnt vary all that much between the three with maximum Linpack performance of 10.74 being recorded for the 32-node setup with hyperthreading disabled and optimal node placement. However the speedup achieved for 32 nodes was just a little over 17 times that of a single node.
The Rackspace cloud instance didnt do nearly as well in the performance department, achieving only 3.04 teraflops on the 32-node setup. Even with just a single node, its performance was much worse than that of the AWS case, despite having more cores, a similar clock frequency, and an identical memory capacity per node. Rackspace did, however, deliver a better than an 18-fold speed as it progressed from 1 to 32 nodes -- slightly better than that of Amazon. That superior speedup is not immediately explainable since AWS instance provides more than twice the bandwidth of the Rackspace setup. Its conceivable the latters network latency is somewhat lower than that of AWS.
IBM SoftLayer fared even worse, delivering just 2.46 Linpack teraflops at 32 nodes and a speedup of just over 4 times that of a single node. No doubt the relatively slow processor clock (2.0 GHz) and slow network speed (1 gigabit/sec) had a lot to do with its poor performance.
Micrsofts Azure cloud offered the most interesting results. Here the Exabyte team decided to test three instances F16s, A9, and H16. The latter two instances were equipped with InfiniBand, the only platforms in the study where this was the case. The A9 instance provided 32 gigabits/sec and the H16 instance provided 54 gigabits/sec nearly as fast as the 64 gigabits/sec of the Aries interconnect on Edison.
Not surprisingly, the A9 and H16 exhibited superior scalability for Linpack, specifically, more than a 28-fold speedup on 32 nodes compared to a single node. Thats slightly better than the 27x speedup Edison achieved. In the performance area, the H16 instance really shined, delivering 17.26 Linpack teraflops in the 32-node configuration. Thats much higher than any of the other environments tested, including the Edison supercomputer. Its probably no coincidence that the H16, which is specifically designed for HPC work, was equipped with the fastest processor of the bunch at 3.2 GHz. Both the A9 and H16 instances also had significantly more memory per node than the other environments.
One of the unfortunate aspects of the Edison measurement is that they enabled hyperthreading for the Linpack runs, something Intel explicitly says not to do if you want to maximize performance on this benchmark. With the exception of one of the AWS tests, none of the others ran the benchmark with hyperthreading enabled.
In fact, the poor Linpack yield on Edison, at just 36 percent of peak in the 32-node test run, suggests the benchmark was not devised very well forthat system. The actual TOP500 run across the entire machine achieved a Linpack yield of more than 64 percent of peak, which is fairly typically of an HPC cluster with a high-performance network. The Azure H16 in this test had a 67 percent Linpack yield.
Theres also no way to tell if other hardware variations things like cache size, memory performance, etc. -- could have affected the results across the different cloud instances. In addition, it was unclear if the benchmark implementations were optimized for the particular environments tested. Teams working on a TOP500 submission will often devote weeks to tweaking a Linpack implementation to maximize performance on a particular system.
It would have been interesting to see Linpack results on other InfiniBand-equipped clouds. In 2014, an InfiniBand option was added to SoftLayer, but the current website doesnt make any mention of such a capability. However, Penguin Computing On Demand, Nimbix, and ProfitBricks all have InfiniBand networks for their purpose-built HPC clouds. Comparing these to the Azure H16 instance could have been instructive. Even more interesting would be to see other HPC benchmarks, like the HPCG metric or specific application kernels tested across these platforms.
Of course, what would be ideal would be some sort of cloud computing tracker that could determine the speed and cost of executing your code on a given cloud platform at any particular time. Thats apt to require a fair amount of AI, not to mention a lot more transparency by cloud providers on how their hardware and software operates underneath the covers. Well, maybe someday...
Go here to read the rest:
Exabyte Measures Linpack Performance Across Major Cloud Vendors - TOP500 News
- New Microsoft Ads Take Aim at Mac Pricing - November 8th, 2009 [November 8th, 2009]
- Adobe Flash Comes to TV - November 8th, 2009 [November 8th, 2009]
- Microsoft Introduces Windows 7 Starter Edition - November 8th, 2009 [November 8th, 2009]
- Mac Viruses and Trojans Becoming More Prevalent - November 8th, 2009 [November 8th, 2009]
- Apple ‘Customer Experience’ Continues to Trounce PCs - November 8th, 2009 [November 8th, 2009]
- Seagate Introduces ‘Replica’ Drive to Backup Entire PC - November 8th, 2009 [November 8th, 2009]
- Still Love XP? Run it on Windows 7! - November 8th, 2009 [November 8th, 2009]
- Is Microsoft Ditching Vista? - November 8th, 2009 [November 8th, 2009]
- The Kindle DX: Not Exactly a Textbook Killer - November 8th, 2009 [November 8th, 2009]
- The Smart Shopper’s Guide to Buying a Wireless Router - May 19th, 2010 [May 19th, 2010]
- iTunes 10: So Long, Ringtone Creator - Thanks for the Memories - October 17th, 2010 [October 17th, 2010]
- iTunes 10: So Long, Ringtone Creator – Thanks for the Memories - February 14th, 2011 [February 14th, 2011]
- How to Make Your Laptop Last Longer - February 14th, 2011 [February 14th, 2011]
- Client Build 5 UPDATE: Personal Super Computer 2011 (SR-2 X5690 OCZ Vertex 3 GTX590 Nvidia Tesla) - Video - March 29th, 2012 [March 29th, 2012]
- Super Micro Computer, Inc. Announces 3rd Quarter 2012 Financial Results - April 25th, 2012 [April 25th, 2012]
- Super Micro Computer Q3 Profit Slips - Quick Facts - April 25th, 2012 [April 25th, 2012]
- Super Computer Maker Cray and Intel strike Partnership - April 25th, 2012 [April 25th, 2012]
- Super Micro Computer Q3 12 Earnings Conference Call At 5:00 PM ET - April 25th, 2012 [April 25th, 2012]
- Herd mentallity and the information super highway - Video - April 25th, 2012 [April 25th, 2012]
- Brain vs. Computer - Video - May 4th, 2012 [May 4th, 2012]
- Minecraft World First - Most wanted redstone device - Video - May 4th, 2012 [May 4th, 2012]
- PS3 Jailbreak Tutorial 4.11 WORKING - Video - May 4th, 2012 [May 4th, 2012]
- China's Tianhe-1 supercomputer begins operations - Video - May 4th, 2012 [May 4th, 2012]
- June 2011 TOP500 Review looks at Japan's K Supercomputer - Video - May 4th, 2012 [May 4th, 2012]
- Super Vision for Soldiers - May 5th, 2012 [May 5th, 2012]
- The Super Sonic Show Episode 0-Computer Help - Video - May 7th, 2012 [May 7th, 2012]
- Why Super Micro Computer's Earnings May Be Less Than Awesome - May 10th, 2012 [May 10th, 2012]
- Magnetic bacteria may help build computer hard drives - May 10th, 2012 [May 10th, 2012]
- SUPER WHY! Around the World Adventure Kicks off PBS KIDS Summer Learning Initiative This June - May 10th, 2012 [May 10th, 2012]
- Tutorial SUPER COMPUTER girl 3750 sylvia Vs fem game 4 (3550) - Video - May 10th, 2012 [May 10th, 2012]
- SUPER COMPUTER Wii best 3750 sylvia Vs learn chess 4 (3550) - Video - May 10th, 2012 [May 10th, 2012]
- SUPER COMPUTER girls city 3750 sylvia Vs RYBKA 4 (3550) - Video - May 10th, 2012 [May 10th, 2012]
- John Laban - Open University Super Computer Room - Video - May 10th, 2012 [May 10th, 2012]
- Can A Super Computer Save Banking? Part 2 of 2 - Video - May 10th, 2012 [May 10th, 2012]
- Supermicro® Launches Widest Range of UP Server Platforms Supporting Intel® Xeon® E3-1200 v2 - May 16th, 2012 [May 16th, 2012]
- Supermicro® Debuts New X9 DP and 4-Way MP Platforms - May 16th, 2012 [May 16th, 2012]
- Supermicro® Launches Widest Range of Server Platforms Supporting Intel® Xeon® E3-1200 v2 - May 16th, 2012 [May 16th, 2012]
- Invention kit for banana pianos, alphabet soup keyboards - May 16th, 2012 [May 16th, 2012]
- A few errors could be key to super-efficient computer chips - May 20th, 2012 [May 20th, 2012]
- Supermicro® Highlights Latest GPU SuperServer®, SuperBlade® and ... - May 20th, 2012 [May 20th, 2012]
- Kontron HPEC Platform Chosen by Military Embedded Systems Magazine for Editor's Choice Award - May 20th, 2012 [May 20th, 2012]
- Raspberry Pi to rebirth an era of Woz-like super creativity? - May 20th, 2012 [May 20th, 2012]
- Taste and tale of success - May 20th, 2012 [May 20th, 2012]
- 1 Reason to Expect Big Things From Super Micro Computer - May 25th, 2012 [May 25th, 2012]
- Bump's Super Popular App Just Got A Million Times Cooler With Its Latest Update - May 25th, 2012 [May 25th, 2012]
- Is The Computer 'Cloud' Compromising You Privacy? - May 26th, 2012 [May 26th, 2012]
- Super MP3 Download 4.8.2.6 - May 28th, 2012 [May 28th, 2012]
- Radiohead's Kid A and OK Computer, Now in 8-Bit - May 29th, 2012 [May 29th, 2012]
- ASUS P6T7 WS Super Computer MoBo - Video - May 29th, 2012 [May 29th, 2012]
- Photonic Super Computer 2012 - Video - May 29th, 2012 [May 29th, 2012]
- Kaspersky discovers super-complex Flame malware - May 30th, 2012 [May 30th, 2012]
- Supermicro® X9 5x GPU SuperWorkstation Delivers Maximum Performance with NVIDIA Maximus Certification - May 30th, 2012 [May 30th, 2012]
- Super-virus Flame raises the cyberwar stakes - May 30th, 2012 [May 30th, 2012]
- Super-stealthy ‘Flame' computer virus spies on Iran - May 31st, 2012 [May 31st, 2012]
- Super-stealthy ‘Flame' computer virus spies on Iranians - May 31st, 2012 [May 31st, 2012]
- Was flame virus written by gamers? Code similar to apps such as Angry Birds - May 31st, 2012 [May 31st, 2012]
- Massive cyber attack on Iran came from U.S., report says - June 2nd, 2012 [June 2nd, 2012]
- Massive cyber attack on Iran came from US, report says - June 2nd, 2012 [June 2nd, 2012]
- Supermicro® Exhibits its Latest X9 Server and Storage Innovations at Computex, Taiwan - June 5th, 2012 [June 5th, 2012]
- Supermicro® Hadoop Solutions Accelerate Innovation with Launch of EMC® ... - June 5th, 2012 [June 5th, 2012]
- Super 57000 Video Game (Family Computer) - Video - June 5th, 2012 [June 5th, 2012]
- Security Cameras Turn into Super-Fast Sleuths - June 7th, 2012 [June 7th, 2012]
- Quantum computers move closer to reality, thanks to highly enriched and highly purified silicon - June 7th, 2012 [June 7th, 2012]
- Research Makes Ultrafast Quantum Computer Concept a Reality - June 9th, 2012 [June 9th, 2012]
- Supermicro's New Compact Embedded Server Appliance Supports 3rd Generation Intel® Core™ i7/i5/i3 Processors - June 11th, 2012 [June 11th, 2012]
- The PC which is truly personal: 'Computer' on a memory stick offers COMPLETE privacy for browsing and documents - June 11th, 2012 [June 11th, 2012]
- 'Purified' silicon nudges quantum computing ahead - June 11th, 2012 [June 11th, 2012]
- Apple serves up 15.4-inch MacBook Pro with Retina Display - June 11th, 2012 [June 11th, 2012]
- Apple debuts next-gen MacBook Pro, iOS 6 - June 11th, 2012 [June 11th, 2012]
- How to Invest Like the Super-Rich - June 13th, 2012 [June 13th, 2012]
- Super Computer for Sale - Video - June 13th, 2012 [June 13th, 2012]
- Supermicro® Launches FatTwin™ Architecture - June 15th, 2012 [June 15th, 2012]
- Computer Workstation utilizes NVIDIA® Maximus(TM) technology. - June 15th, 2012 [June 15th, 2012]
- Supermicro® Launches FatTwinâ„¢ Architecture - June 15th, 2012 [June 15th, 2012]
- Acer: Aspire S5, super-thin Ultrabook, coming to U.S. in late June - June 15th, 2012 [June 15th, 2012]
- Supermicro(R) Launches FatTwin(TM) Architecture - June 15th, 2012 [June 15th, 2012]
- Sheldon Adelson: 7 surprising facts about 2012's biggest donor - June 15th, 2012 [June 15th, 2012]
- lego super computer - Video - June 17th, 2012 [June 17th, 2012]
- Age of Empires: The Conqurors - vsing Duke AI 1.6 - Super computer - Video - June 17th, 2012 [June 17th, 2012]
- Supermicro® FatTwin™ Takes Center Stage at International Supercomputing Conference 2012 - June 18th, 2012 [June 18th, 2012]