July 21, 2020 As neuroscientists work to better understand the complex inner workings of the brain, a focus of their efforts lies in reimagining and reinventing one of their most basic research tools: the microscope. Likewise, as astrophysicists and cosmologists strive to gain new insights into the universe and its origins, they are eager to observe farther, faster, and with increasing detail via enhancements to their primary instrument: the telescope.
In each case, to unravel scientific mysteries that are either too big or too small to see with a physical instrument alone, they must work in conjunction with yet another critical piece of equipment: the computer. This means more data and increasingly complex datasets, which in turn impacts how quickly scientists can sift through these datasets to find the most relevant clues about where their research should go next.
Fortunately, being able to do this sort of data collection and processing in near real time is becoming a reality for projects like theDark Energy Spectroscopic Instrument(DESI), a multi-facility collaboration led by Lawrence Berkeley National Laboratory whose goal is to produce the largest 3D map of the universe ever created. Installed on the Mayall Telescope at Kitt Peak National Observatory near Tucson, Arizona, DESI is bringing high-speed automation, high-performance computing, and high-speed networking to its five-year galaxy-mapping mission, capturing light from 35 million galaxies and 2.4 million quasars and transmitting that data to the National Energy Research Scientific Computing Center (NERSC), a U.S. Department of Energy user facility based at Berkeley Lab that serves as DESIs primary computing center.
We turn the raw data into useful data, said Stephen Bailey, a physicist at Berkeley Lab who is the technical lead and manager of the DESI data systems. The raw data coming off the telescope isnt the map, so we have to take that data, calibrate it, process it, and turn it into a 3D map that the scientists within the broader collaboration (some 600 worldwide) use for their analyses.
Over the last several years the DESI team has been using NERSC to build catalogues of the most interesting observational targets, modeling the shapes and colors of more than 1.6 billion individual galaxies detected in 4.3 million images collected bythree large-scale sky surveys. The resultingDESI Legacy Imaging Surveys, hosted at NERSC, have performed their catalogue generation at NERSC over the course of eight data releases. The DESI project also leverages the Cosmology Data Repository hosted at NERSC, which contains about 900TB of data, and NERSCs Community File System, scratch, and HPSS storage systems.
The previous big survey was a few million objects, but now we are going up to 35-50 million objects, Bailey said. Its a big step forward in the size of the map and the science you can do with.
But storage is only part of the services NERSC delivers for DESI. The supercomputing center has also been instrumental in developing and supporting DESIs data processing pipeline, which facilitates the transfer of data from the surveys to the computing center and to users. The project uses 10 dedicated nodes on the Cori supercomputer, enabling the pipeline to run throughout each night during a survey and ensure that the results are available to users by morning for same-day analysis, often helping to inform the next nights observation plan. The DESI team also uses hundreds of nodes for other processing and expects to scale to thousands of nodes as the dataset increases. To facilitate data I/O, DESI depends on the NERSC data transfer nodes, which are managed as part of a collaborative effort between ESnet and NERSC to enable high performance data movement over the high-bandwidth 100Gb ESnet wide-area network.
DESI is using the full NERSC ecosystem: computing services, storage, the real-time queue, and real-time data transfer, Bailey said. Its a real game changer for being able to keep up with the data.
Optimizing Python for CPUs and GPUs
While gearing up for the five-year DESI survey, which is expected to begin in late 2020, NERSC worked with the DESI team to identify the most computationally intensive parts of the data processing pipeline and implement changes to speed them up. Through the NERSC Exascale Science Applications Program (NESAP), Laurie Stephey, then a postdoctoral researcher and now a data analytics engineer at NERSC, began examining the code.
The pipeline is written almost exclusively in Python a specialty of Stepheys which enables domain scientists to write readable and maintainable scientific code in a relatively short amount of time. Stepheys goal was to improve the pipelines performance while satisfying the DESI teams requirement that the software remain in Python. The challenge, she explained, was in staying true to the original code while finding new and efficient ways to speed its performance.
It was my job to keep their code readable and maintainable and to speed it up on the Cori supercomputers KNL manycore architecture, Stephey said. In the end, we increased their processing throughput 5 to 7 times, which was a big accomplishment bigger than Id expected. This means that something that previously took up to 48 hours now happens overnight, thus enabling analysis during the day and feedback to the following nights observations, Bailey noted. It also saves the DESI project tens of millions of compute hours at NERSC annually.
New experiments funded by DOE approach NERSC for support all the time, said Rollin Thomas who runs NESAP for Data. And experiments that already use NERSC are capitalizing on our diverse capabilities to do new and exciting things with data. DESIs sustained engagement with NERSC, through NESAP for Data, the Superfacility initiative and so on, is a model for other experiments. What we learn from these engagements helps us serve the broader experimental and observational data science community better.
And the optimization effort isnt over yet. The next challenge is to make the DESI code compatible with the GPUs in NERSCs Perlmutter system, which is slated to arrive in late 2020. Bailey and Stephey began this process last year Stephen was instrumental in rewriting the algorithm in a GPU-friendly way, Stephey noted but in April NERSC hired one of its newest NESAP postdocs, Daniel Margala, to take over. As a graduate student, Margala had previously worked with Bailey on the Baryon Oscillation Spectroscopic Survey, a DESI predecessor project, so Im familiar with a lot of the data processing that needs to be done for DESI, he said.
So far, Margalas focus is on preparing DESIs code for GPUs so that it will be ready to leverage the full potential of the Perlmutter system. He is currently working with a small subset of DESI data on Coris GPU testbed nodes; the long-term goal is to make sure the software is ready to handle DESIs entire five-year dataset.
The astrophysicists and scientists on DESI are pretty comfortable using Python, so we are trying to do all of this in Python so that they will be able to understand the code we are writing and learn from it, contribute back to it, and maintain it going forward, Margala said.
Over the next few years, NERSC resources will also be critical to another, larger goal of the DESI project: reprocessing and updating the data.
Every year we are going to reprocess our data from the very beginning using the latest version of all of our code, and those will become our data assemblies that will then flow into the science papers for the collaboration, Bailey said. We only need 10 nodes at NERSC to keep up with the data in real time through the night, but if you want to go back and process 2, 3, 5 years of data, thats where being able to use hundreds or thousands of nodes will allow us to quickly catch up on all that processing.
Originally posted here:
- COVID-19: Supercomputer to support research on the pandemic - Taipei Times - June 10th, 2021
- Space Weather Prediction Gets a Supercomputing Boost - HPCwire - June 10th, 2021
- Worlds Fastest AI Supercomputer Perlmutter Will Help Create Largest-Ever 3D Map Of The Universe! - Mashable India - June 10th, 2021
- Supercomputer predicts Euro 2020 with England beating Spain and Portugal but losing on penalties to... - The Sun - June 10th, 2021
- Super Computer predicts Euro 2020 winner England stun Spain but its penalty heartbreak for the Three Lio... - talkSPORT.com - June 10th, 2021
- Looking to the future of quantum cloud computing - Siliconrepublic.com - Siliconrepublic.com - June 10th, 2021
- Hate to break it to you, but football's not coming home if this AI pundit is to be believed - The Register - June 10th, 2021
- Euro 2020: England only have 5.2% chance of winning tournament, says supercomputer - GIVEMESPORT - June 10th, 2021
- Wyoming Supercomputer Upgrade Will Make It One Of Top 25 Fastest In The World - Wyoming Public Media - February 7th, 2021
- Team Led by PPPL Physicist Wins Major Supercomputer Time to Help Develop Fusion Energy - HPCwire - February 7th, 2021
- Super Bowl 2021: What time is the game, how to watch Bucs vs. Chiefs on CBS for free - CNET - February 7th, 2021
- Meet the billionaire commanding SpaceXs all-civilian missionhe dropped out of high school to start his business - CNBC - February 7th, 2021
- Super Computer rates Newcastle United chances of beating Southampton and relegation probability | NUFC The Mag - The Mag - February 7th, 2021
- Supercomputer Market to Exhibit Impressive Growth of CAGR during the period 202 - Business-newsupdate.com - February 7th, 2021
- HORIZON BLOG: Research and innovation in the new seven-year budget - Science Business - February 7th, 2021
- Is this real life? Glitch in the Matrix has its doubts - Boston Herald - February 7th, 2021
- Global Supercomputer Market Is Expected To Show Significant Growth over the Forecast Period 2020-2027 The Courier - The Courier - February 7th, 2021
- What is a supercomputer? - CNBC - December 5th, 2020
- Supercomputer may give us COVID meds to join vaccines - al.com - December 5th, 2020
- Singapore Researchers Plug in to World's Fastest Supercomputer - HPCwire - December 5th, 2020
- Pawsey's Galaxy Supercomputer Aids Telescope in Creating New Atlas of the Universe - HPCwire - December 5th, 2020
- GENCI Supercomputer Simulation Illuminates the Dark Universe - HPCwire - December 5th, 2020
- Cerebras CS-1 supercomputer uses the worlds largest chip - Inceptive Mind - December 5th, 2020
- Supercomputer Market Overview with Qualitative analysis, Competitive landscape & Forecast by 2027 - The Market Feed - December 5th, 2020
- New IBM encryption tools head off quantum computing threats - TechTarget - December 5th, 2020
- As it closes in on Arm, Nvidia announces UK supercomputer dedicated to medical research - TechCrunch - October 8th, 2020
- With Crossroads Supercomputer, HPE Notches Another DOE Win - The Next Platform - October 8th, 2020
- What happens when two planets crash together? This supercomputer has the answer - Digital Trends - October 8th, 2020
- Supermicro Details Its Hardware for MN-3, the Most Efficient Supercomputer in the World - HPCwire - September 2nd, 2020
- I confess, I'm scared of the next generation of supercomputers - TechRadar - September 2nd, 2020
- Bradykinin Hypothesis of COVID-19 Offers Hope for Already-Approved Drugs - BioSpace - September 2nd, 2020
- Stranger than fiction? Why we need supercomputers - TechHQ - September 2nd, 2020
- Google Says It Just Ran The First-Ever Quantum Simulation of a Chemical Reaction - ScienceAlert - September 2nd, 2020
- This Equation Calculates the Chances We Live in a Computer Simulation - Discover Magazine - September 2nd, 2020
- 17 of the best computers and supercomputers to grace the planet - Pocket-lint - August 31st, 2020
- Supercomputer finds best way to air out classroom to ward off virus : The Asahi Shimbun - Asahi Shimbun - August 31st, 2020
- The Supercomputer Breaking Online Gaming Records and Modeling COVID-19 - BioSpace - August 31st, 2020
- When it comes to hurricane models, which one is best? - KHOU.com - August 31st, 2020
- Natural Radiation Including Cosmic Rays From Outer Space Can Wreak Havoc With Quantum Computers - SciTechDaily - August 31st, 2020
- The Tech Field Failed a 25-Year Challenge to Achieve Gender Equality by 2020 Culture Change Is Key to Getting on Track - Nextgov - August 31st, 2020
- Cerebras Systems Expands Global Footprint with Toronto Office Opening - HPCwire - August 31st, 2020
- CSC's Supercomputer Mahti is Now Available to Researchers and Students - HPCwire - August 28th, 2020
- Here's the smallest AI/ML supercomputer ever - TechRadar - August 28th, 2020
- When it comes to hurricane models, which one is best? - 12newsnow.com KBMT-KJAC - August 28th, 2020
- SberCloud's Cloud Platform Sweeps Three International Accolades At IT World Awards - Exchange News Direct - August 28th, 2020
- Supercomputer Market Growth, Future Prospects And Competitive Analysis (2020-2026) - Bulletin Line - August 28th, 2020
- A continent works to grow its stake in quantum computing - University World News - August 28th, 2020
- Supercomputer predicts where Spurs will finish in the 2020/21 Premier League table - The Spurs Web - August 28th, 2020
- Has the world's most powerful computer arrived? - The National - August 28th, 2020
- Galaxy Simulations Could Help Reveal Origins of Milky Way - Newswise - August 28th, 2020
- ALCC Program Awards Computing Time on ALCF's Theta Supercomputer to 24 projects - HPCwire - August 10th, 2020
- A Quintillion Calculations a Second: DOE Calculating the Benefits of Exascale and Quantum Computers - SciTechDaily - August 10th, 2020
- GE plans to give offshore wind energy a supercomputing boost - The Verge - August 10th, 2020
- From WarGames to Terms of Service: How the Supreme Courts Review of Computer Fraud Abuse Act Will Impact Your Trade Secrets - JD Supra - August 10th, 2020
- New Audis To Use Supercomputer That Controls Almost Everything - Motor1 - August 10th, 2020
- Japanese supercomputer ranked as worlds most powerful system - August 10th, 2020
- Top 10 Supercomputers | HowStuffWorks - August 10th, 2020
- What are supercomputers currently used for? | HowStuffWorks - August 10th, 2020
- Microsoft announces new supercomputer, lays out vision for ... - August 10th, 2020
- GE taps into US supercomputer to advance offshore wind - reNEWS - August 10th, 2020
- Summit supercomputer to advance research on wind power for renewable energy - ZDNet - August 10th, 2020
- BSC Researchers Create Spin-Off Platform to Accelerate the Development of New Chemicals - HPCwire - August 10th, 2020
- Supercomputer study of mobility in Spain at the peak of COVID-19 using Facebook and Google data - Science Business - August 9th, 2020
- Julia and PyCaret Latest Versions, arXiv on Kaggle, UK's AI Supercomputer And More In This Week's Top AI News - Analytics India Magazine - August 9th, 2020
- Audi To Over-Complicate Cars With Supercomputers And Repair Costs Could Skyrocket - Top Speed - August 9th, 2020
- Supercomputer COVID-19 insights, ionic spiderwebs, the whiteness of AI TechCrunch - Best gaming pro - August 8th, 2020
- Every Superman Movie Climax, Ranked From Worst To Best - Screen Rant - August 8th, 2020
- Atos Partners with University of Oxford on Largest AI Supercomputer in the UK - HPCwire - August 7th, 2020
- Five Movies Worth Watching About the Threat of Nuclear War - Council on Foreign Relations - August 7th, 2020
- Break it down: A new way to address common computing problem - Washington University in St. Louis Newsroom - August 7th, 2020
- GE Research uses summit supercomputer for study on wind power - Windtech International - August 7th, 2020
- Research: A Survey of Numerical Methods Utilizing Mixed Precision Arithmetic - HPCwire - August 7th, 2020
- Atos signs 5m supercomputing deal to support Oxford University-led AI research push - ComputerWeekly.com - August 6th, 2020
- Atos partners with University of Oxford on largest AI supercomputer in the UK - Yahoo Finance - August 6th, 2020
- How coronavirus antibody testing works - Livemint - August 6th, 2020
- Researchers Use Supercomputers To Discover New Pathway For Covid-19 Inflammation - Forbes - August 6th, 2020
- Supercomputer-Powered Research Uncovers Signs of 'Bradykinin Storm' That May Explain COVID-19 Symptoms - HPCwire - July 31st, 2020
- Celtic and Rangers title race outcome predicted by betting supercomputer - HeraldScotland - July 31st, 2020
- Nvidia reportedly in advanced talks to buy Arm - ZDNet - July 31st, 2020
- NVIDIA Claims To Have Won MLPerf Benchmarking, But Google Says Otherwise - Analytics India Magazine - July 31st, 2020