July 21, 2020 As neuroscientists work to better understand the complex inner workings of the brain, a focus of their efforts lies in reimagining and reinventing one of their most basic research tools: the microscope. Likewise, as astrophysicists and cosmologists strive to gain new insights into the universe and its origins, they are eager to observe farther, faster, and with increasing detail via enhancements to their primary instrument: the telescope.
In each case, to unravel scientific mysteries that are either too big or too small to see with a physical instrument alone, they must work in conjunction with yet another critical piece of equipment: the computer. This means more data and increasingly complex datasets, which in turn impacts how quickly scientists can sift through these datasets to find the most relevant clues about where their research should go next.
Fortunately, being able to do this sort of data collection and processing in near real time is becoming a reality for projects like theDark Energy Spectroscopic Instrument(DESI), a multi-facility collaboration led by Lawrence Berkeley National Laboratory whose goal is to produce the largest 3D map of the universe ever created. Installed on the Mayall Telescope at Kitt Peak National Observatory near Tucson, Arizona, DESI is bringing high-speed automation, high-performance computing, and high-speed networking to its five-year galaxy-mapping mission, capturing light from 35 million galaxies and 2.4 million quasars and transmitting that data to the National Energy Research Scientific Computing Center (NERSC), a U.S. Department of Energy user facility based at Berkeley Lab that serves as DESIs primary computing center.
We turn the raw data into useful data, said Stephen Bailey, a physicist at Berkeley Lab who is the technical lead and manager of the DESI data systems. The raw data coming off the telescope isnt the map, so we have to take that data, calibrate it, process it, and turn it into a 3D map that the scientists within the broader collaboration (some 600 worldwide) use for their analyses.
Over the last several years the DESI team has been using NERSC to build catalogues of the most interesting observational targets, modeling the shapes and colors of more than 1.6 billion individual galaxies detected in 4.3 million images collected bythree large-scale sky surveys. The resultingDESI Legacy Imaging Surveys, hosted at NERSC, have performed their catalogue generation at NERSC over the course of eight data releases. The DESI project also leverages the Cosmology Data Repository hosted at NERSC, which contains about 900TB of data, and NERSCs Community File System, scratch, and HPSS storage systems.
The previous big survey was a few million objects, but now we are going up to 35-50 million objects, Bailey said. Its a big step forward in the size of the map and the science you can do with.
But storage is only part of the services NERSC delivers for DESI. The supercomputing center has also been instrumental in developing and supporting DESIs data processing pipeline, which facilitates the transfer of data from the surveys to the computing center and to users. The project uses 10 dedicated nodes on the Cori supercomputer, enabling the pipeline to run throughout each night during a survey and ensure that the results are available to users by morning for same-day analysis, often helping to inform the next nights observation plan. The DESI team also uses hundreds of nodes for other processing and expects to scale to thousands of nodes as the dataset increases. To facilitate data I/O, DESI depends on the NERSC data transfer nodes, which are managed as part of a collaborative effort between ESnet and NERSC to enable high performance data movement over the high-bandwidth 100Gb ESnet wide-area network.
DESI is using the full NERSC ecosystem: computing services, storage, the real-time queue, and real-time data transfer, Bailey said. Its a real game changer for being able to keep up with the data.
Optimizing Python for CPUs and GPUs
While gearing up for the five-year DESI survey, which is expected to begin in late 2020, NERSC worked with the DESI team to identify the most computationally intensive parts of the data processing pipeline and implement changes to speed them up. Through the NERSC Exascale Science Applications Program (NESAP), Laurie Stephey, then a postdoctoral researcher and now a data analytics engineer at NERSC, began examining the code.
The pipeline is written almost exclusively in Python a specialty of Stepheys which enables domain scientists to write readable and maintainable scientific code in a relatively short amount of time. Stepheys goal was to improve the pipelines performance while satisfying the DESI teams requirement that the software remain in Python. The challenge, she explained, was in staying true to the original code while finding new and efficient ways to speed its performance.
It was my job to keep their code readable and maintainable and to speed it up on the Cori supercomputers KNL manycore architecture, Stephey said. In the end, we increased their processing throughput 5 to 7 times, which was a big accomplishment bigger than Id expected. This means that something that previously took up to 48 hours now happens overnight, thus enabling analysis during the day and feedback to the following nights observations, Bailey noted. It also saves the DESI project tens of millions of compute hours at NERSC annually.
New experiments funded by DOE approach NERSC for support all the time, said Rollin Thomas who runs NESAP for Data. And experiments that already use NERSC are capitalizing on our diverse capabilities to do new and exciting things with data. DESIs sustained engagement with NERSC, through NESAP for Data, the Superfacility initiative and so on, is a model for other experiments. What we learn from these engagements helps us serve the broader experimental and observational data science community better.
And the optimization effort isnt over yet. The next challenge is to make the DESI code compatible with the GPUs in NERSCs Perlmutter system, which is slated to arrive in late 2020. Bailey and Stephey began this process last year Stephen was instrumental in rewriting the algorithm in a GPU-friendly way, Stephey noted but in April NERSC hired one of its newest NESAP postdocs, Daniel Margala, to take over. As a graduate student, Margala had previously worked with Bailey on the Baryon Oscillation Spectroscopic Survey, a DESI predecessor project, so Im familiar with a lot of the data processing that needs to be done for DESI, he said.
So far, Margalas focus is on preparing DESIs code for GPUs so that it will be ready to leverage the full potential of the Perlmutter system. He is currently working with a small subset of DESI data on Coris GPU testbed nodes; the long-term goal is to make sure the software is ready to handle DESIs entire five-year dataset.
The astrophysicists and scientists on DESI are pretty comfortable using Python, so we are trying to do all of this in Python so that they will be able to understand the code we are writing and learn from it, contribute back to it, and maintain it going forward, Margala said.
Over the next few years, NERSC resources will also be critical to another, larger goal of the DESI project: reprocessing and updating the data.
Every year we are going to reprocess our data from the very beginning using the latest version of all of our code, and those will become our data assemblies that will then flow into the science papers for the collaboration, Bailey said. We only need 10 nodes at NERSC to keep up with the data in real time through the night, but if you want to go back and process 2, 3, 5 years of data, thats where being able to use hundreds or thousands of nodes will allow us to quickly catch up on all that processing.
Originally posted here:
- As it closes in on Arm, Nvidia announces UK supercomputer dedicated to medical research - TechCrunch - October 8th, 2020
- With Crossroads Supercomputer, HPE Notches Another DOE Win - The Next Platform - October 8th, 2020
- What happens when two planets crash together? This supercomputer has the answer - Digital Trends - October 8th, 2020
- Supermicro Details Its Hardware for MN-3, the Most Efficient Supercomputer in the World - HPCwire - September 2nd, 2020
- I confess, I'm scared of the next generation of supercomputers - TechRadar - September 2nd, 2020
- Bradykinin Hypothesis of COVID-19 Offers Hope for Already-Approved Drugs - BioSpace - September 2nd, 2020
- Stranger than fiction? Why we need supercomputers - TechHQ - September 2nd, 2020
- Google Says It Just Ran The First-Ever Quantum Simulation of a Chemical Reaction - ScienceAlert - September 2nd, 2020
- This Equation Calculates the Chances We Live in a Computer Simulation - Discover Magazine - September 2nd, 2020
- 17 of the best computers and supercomputers to grace the planet - Pocket-lint - August 31st, 2020
- Supercomputer finds best way to air out classroom to ward off virus : The Asahi Shimbun - Asahi Shimbun - August 31st, 2020
- The Supercomputer Breaking Online Gaming Records and Modeling COVID-19 - BioSpace - August 31st, 2020
- When it comes to hurricane models, which one is best? - KHOU.com - August 31st, 2020
- Natural Radiation Including Cosmic Rays From Outer Space Can Wreak Havoc With Quantum Computers - SciTechDaily - August 31st, 2020
- The Tech Field Failed a 25-Year Challenge to Achieve Gender Equality by 2020 Culture Change Is Key to Getting on Track - Nextgov - August 31st, 2020
- Cerebras Systems Expands Global Footprint with Toronto Office Opening - HPCwire - August 31st, 2020
- CSC's Supercomputer Mahti is Now Available to Researchers and Students - HPCwire - August 28th, 2020
- Here's the smallest AI/ML supercomputer ever - TechRadar - August 28th, 2020
- When it comes to hurricane models, which one is best? - 12newsnow.com KBMT-KJAC - August 28th, 2020
- SberCloud's Cloud Platform Sweeps Three International Accolades At IT World Awards - Exchange News Direct - August 28th, 2020
- Supercomputer Market Growth, Future Prospects And Competitive Analysis (2020-2026) - Bulletin Line - August 28th, 2020
- A continent works to grow its stake in quantum computing - University World News - August 28th, 2020
- Supercomputer predicts where Spurs will finish in the 2020/21 Premier League table - The Spurs Web - August 28th, 2020
- Has the world's most powerful computer arrived? - The National - August 28th, 2020
- Galaxy Simulations Could Help Reveal Origins of Milky Way - Newswise - August 28th, 2020
- ALCC Program Awards Computing Time on ALCF's Theta Supercomputer to 24 projects - HPCwire - August 10th, 2020
- A Quintillion Calculations a Second: DOE Calculating the Benefits of Exascale and Quantum Computers - SciTechDaily - August 10th, 2020
- GE plans to give offshore wind energy a supercomputing boost - The Verge - August 10th, 2020
- From WarGames to Terms of Service: How the Supreme Courts Review of Computer Fraud Abuse Act Will Impact Your Trade Secrets - JD Supra - August 10th, 2020
- New Audis To Use Supercomputer That Controls Almost Everything - Motor1 - August 10th, 2020
- Japanese supercomputer ranked as worlds most powerful system - August 10th, 2020
- Top 10 Supercomputers | HowStuffWorks - August 10th, 2020
- What are supercomputers currently used for? | HowStuffWorks - August 10th, 2020
- Microsoft announces new supercomputer, lays out vision for ... - August 10th, 2020
- GE taps into US supercomputer to advance offshore wind - reNEWS - August 10th, 2020
- Summit supercomputer to advance research on wind power for renewable energy - ZDNet - August 10th, 2020
- BSC Researchers Create Spin-Off Platform to Accelerate the Development of New Chemicals - HPCwire - August 10th, 2020
- Supercomputer study of mobility in Spain at the peak of COVID-19 using Facebook and Google data - Science Business - August 9th, 2020
- Julia and PyCaret Latest Versions, arXiv on Kaggle, UK's AI Supercomputer And More In This Week's Top AI News - Analytics India Magazine - August 9th, 2020
- Audi To Over-Complicate Cars With Supercomputers And Repair Costs Could Skyrocket - Top Speed - August 9th, 2020
- Supercomputer COVID-19 insights, ionic spiderwebs, the whiteness of AI TechCrunch - Best gaming pro - August 8th, 2020
- Every Superman Movie Climax, Ranked From Worst To Best - Screen Rant - August 8th, 2020
- Atos Partners with University of Oxford on Largest AI Supercomputer in the UK - HPCwire - August 7th, 2020
- Five Movies Worth Watching About the Threat of Nuclear War - Council on Foreign Relations - August 7th, 2020
- Break it down: A new way to address common computing problem - Washington University in St. Louis Newsroom - August 7th, 2020
- GE Research uses summit supercomputer for study on wind power - Windtech International - August 7th, 2020
- Research: A Survey of Numerical Methods Utilizing Mixed Precision Arithmetic - HPCwire - August 7th, 2020
- Atos signs 5m supercomputing deal to support Oxford University-led AI research push - ComputerWeekly.com - August 6th, 2020
- Atos partners with University of Oxford on largest AI supercomputer in the UK - Yahoo Finance - August 6th, 2020
- How coronavirus antibody testing works - Livemint - August 6th, 2020
- Researchers Use Supercomputers To Discover New Pathway For Covid-19 Inflammation - Forbes - August 6th, 2020
- Supercomputer-Powered Research Uncovers Signs of 'Bradykinin Storm' That May Explain COVID-19 Symptoms - HPCwire - July 31st, 2020
- Celtic and Rangers title race outcome predicted by betting supercomputer - HeraldScotland - July 31st, 2020
- Nvidia reportedly in advanced talks to buy Arm - ZDNet - July 31st, 2020
- NVIDIA Claims To Have Won MLPerf Benchmarking, But Google Says Otherwise - Analytics India Magazine - July 31st, 2020
- Continental is supercharging the development of driver-assistance tech - CNET - July 31st, 2020
- COVID-19 Pandemic Can Help More of Us Learn About Climate Change - UT News | The University of Texas at Austin - July 31st, 2020
- From rocks to icebergs, the natural world tends to break into cubes - Science Magazine - July 31st, 2020
- New Data on Genetic Expression In Severe COVID-19, Pre-Existing Immune Response - Bio-IT World - July 31st, 2020
- Superman's Glasses Are Secretly Used For Mind Control - Screen Rant - July 31st, 2020
- The Israeli company that has come as close as possible to the sun - Haaretz.com - July 31st, 2020
- Celtic and Rangers title race outcome predicted by betting supercomputer - Glasgow Times - July 31st, 2020
- PEARC20 Plenary Introduces Five Upcoming NSF-Funded HPC Systems - HPCwire - July 31st, 2020
- NIH Awards $6M to UConn Health Biological Computer Modeling Teams - HPCwire - July 31st, 2020
- Continental Debuts the Fastest Supercomputer in the Automotive Industry and It's Built for AI - EnterpriseAI - July 30th, 2020
- WATCH: Supercomputer generates 3D videos that show how Earth may have lost half of its atmosphere to create th - Business Insider India - July 26th, 2020
- Repeated intelligence failures: Time to worry - The Sunday Guardian - July 26th, 2020
- Supercomputer Market to witness an impressive growth during the forecast period 2020 - 2026 - CueReport - July 26th, 2020
- What is supercomputer? - Definition from WhatIs.com - July 26th, 2020
- What is supercomputer? - Definition - July 26th, 2020
- Impact of Covid-19 on Supercomputer Market Comprehensive Growth 2020-2027 with Top key vendor IBM Corporation, Cray Inc., Lenovo Inc., Sugon, Inspur -... - July 25th, 2020
- Super-computer Henry Cavill breaks the Internet again with more geek content - KSRO - July 24th, 2020
- How Equity Is Lost When Companies Hire Only Workers With Disabilities - The New York Times - July 24th, 2020
- Earth may have lost half of its atmosphere to create Moon, reveals study using 3D videos - Republic World - Republic World - July 24th, 2020
- "Super"-computer: Henry Cavill breaks the Internet again with more geek content - wcsjnews.com - July 24th, 2020
- Solar Opposites EPs Tease Whats To Come On Season 2 Of Hulu Animated Comedy Comic-Con@Home - Deadline - July 24th, 2020
- NVIDIA and University of Florida Release New AI Curriculum Spanning All Educational Disciplines - Motley Fool - July 21st, 2020
- "Super"-computer Henry Cavill breaks the Internet again with more geek content - 1310kfka.com - July 20th, 2020
- It's time to decide what we want Downriver's future to look like - Southgate News Herald - July 20th, 2020
- The worlds supercomputers joined forces against COVID-19 why such collaborations are critical for tackling future emergencies - The European Sting - July 20th, 2020