Greg Kurtzer invented software called Singularity to enable the use of containers in high-performance computing (Credit: Marillyn Chung/Berkeley Lab)
Scientists used to come to Gregory Kurtzer of Lawrence Berkeley National Laboratorys (Berkeley Labs) IT department a lot, asking for a better way to use software containers in a high-performance computing (HPC) environment. After a while he got tired of saying, Sorry, not possible. So he invented a solution and named it Singularity.
Within a few months of its release last year, Singularity took off. Computing-heavy scientific institutions worldwidefrom Stanford University to the Massachusetts Institute of Technology to various sites on the European Grid e-Infrastructureflocked to the software. Singularity was also recently recognized by HPCwire editors as one of five new technologies to watch.
Singularity has been making huge strides in the computing community, Kurtzer said, with some surprise, adding that Open Science Grid, a consortium that provides distributed computing resources for scientific research, has served over 20 million containers with Singularity.
Its now on its seventh release (version 2.2.1) and has caught on so quickly that Kurtzer has launched SingularityWare LLC to further develop and support the open-source software. The company is being funded by RStor Inc., a startup based in Saratoga, California. Kurtzer, the long-time technical lead and architect for the HPC Services group at Berkeley Lab with a joint appointment at UC Berkeley, will shift to an advisory role at the Lab in order to focus on Singularity.
Berkeley Lab makes some Lab-developed software available at no cost to maximize its impact and to participate in the open-source software community, said ElsieQuaite-Randall, Berkeley Labs Chief Technology Transfer Officer. Singularity fosters innovation as open-source software, and now SingularityWare LLClike other Berkeley Lab startupswill set out to expand the reach and adoption of an important technology.
A typical case where users might need Singularity is if they want to run an application such as Googles TensorFlow. They may need a very specific version of Tensor Flow installed, Kurtzer said. They can create a container to do that in about five minutes. Then they can take that container, bring it to our environment and run it, even if we dont have that version of Tensor Flow installed.
Software containers make it possible to take your entire computing environment, including your files and all the applications you want to run, and encapsulate it so it can be easily replicated on another machine without worrying whether the new machine has a compatible operating system, libraries, applications, and so forth.
Containers share some of the use cases of virtual machines but without the code redundancy and performance hit associated with virtualization, Kurtzer said. Singularity containers allow a user to encapsulate an entire OS (operating system) environment and use it on a shared HPC system like any other program, without an admin doing anything.
Another example where Singularity would be useful would be allowing other scientists to reproduce experiments. Say you just published an article. Wouldnt it be nice to have a location you can cite where someone can download the Singularity container and replicate all the experiments? Kurtzer asked. Someone can enter the container, and now theyre sitting in the exact same environment as you were.
Containerization was developed for enterprise environments, where it has become very popular, especially with the rise of Dockers container technology. Dockers container solution is for the enterprise. But the scientific use case is quite different, Kurtzer said. Our goal isnt to run as many containers as we can on a single host, with each having the illusion of sole occupancy and isolation, but to run maybe one, and enable it to utilize all the resources on that host. Its kind of the opposite of isolation!
So Kurtzer started working on his own solution, and four months later, the first version was released last spring. When I started working on it, I asked, what do scientists really need from containers? They need reproducibility, mobility, and also freedomthe ability to install their own applications and run in their own environment, and store it just like any other data file, Kurtzer said. Thats what Singularity solves for scientific computing.
Kurtzer chose the name Singularity for its meaning in astronomy. As I understand it, its the culmination of a whole bunch of matter in the universe forming a single infinitely dense point, he said. Thats what I was thinking when I was creating Singularitytaking everything necessary to create a reproducible scientific environment and putting it in one file.
Singularity also enables users to run legacy workflows easily. Kurtzer cites one example of how his group saved an 18-year-old workflow from failing hardware and was able to convert it to a Singularity container that is still being used in production today.
Kurtzer believes Singularity will benefit scientists who may not even know they need it. Were trying to reach out to more scientists and engage with additional groups, especially those who are not traditional HPC users, also known as the computational long tail of science, Kurtzer said. We have a lot of users that are running computationally intensive jobs on their laptops and workstations and not making use of the dedicated computational cycles that are designed specifically for computing and available to them. With Singularity we can easily make these large computing resources tangible.
# # #
Lawrence Berkeley National Laboratory addresses the worlds most urgent scientific challenges by advancing sustainable energy, protecting human health, creating new materials, and revealing the origin and fate of the universe. Founded in 1931, Berkeley Labs scientific expertise has been recognized with 13 Nobel Prizes. The University of California manages Berkeley Lab for the U.S. Department of Energys Office of Science. For more, visit http://www.lbl.gov.
DOEs Office of Science is the single largest supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time. For more information, please visit science.energy.gov.
More here:
- Downloads - Singularity Viewer - December 25th, 2016 [December 25th, 2016]
- What is Singularity (the)? - Definition from WhatIs.com - January 5th, 2017 [January 5th, 2017]
- When Electronic Witnesses Are Everywhere, No Secret's Safe - Singularity Hub - February 7th, 2017 [February 7th, 2017]
- Report: AMD Ryzen Performance in Ashes of the Singularity Benchmark - PC Perspective - February 7th, 2017 [February 7th, 2017]
- Do you believe in the Singularity? - Patheos (blog) - February 7th, 2017 [February 7th, 2017]
- Editorial Note From the Singularity Hub Team - Singularity Hub - February 7th, 2017 [February 7th, 2017]
- Discover the Most Advanced Industrial Technologies at Exponential Manufacturing - Singularity Hub - February 7th, 2017 [February 7th, 2017]
- 10th Letter looks at nature in the time of the Singularity - Creative Loafing Atlanta - February 7th, 2017 [February 7th, 2017]
- Donald Trump Is the Singularity - Bloomberg View - Bloomberg.com - Bloomberg - February 7th, 2017 [February 7th, 2017]
- Wearable Devices Can Actually Tell When You're About to Get Sick - Singularity Hub - February 7th, 2017 [February 7th, 2017]
- AMD 8-core Ryzen benchmark show up on Ashes Of The Singularity ... - VR-Zone - February 8th, 2017 [February 8th, 2017]
- Robot Cars Can Teach Themselves How to Drive in Virtual Worlds - Singularity Hub - February 8th, 2017 [February 8th, 2017]
- Physicists Unveil Blueprint for a Quantum Computer the Size of a Soccer Field - Singularity Hub - February 10th, 2017 [February 10th, 2017]
- How Robots Helped Create 100000 Jobs at Amazon - Singularity Hub - February 10th, 2017 [February 10th, 2017]
- Ready to Change the World? Apply Now for Singularity University's 2017 Global Solutions Program - Singularity Hub - February 10th, 2017 [February 10th, 2017]
- Singularity Containers for Science, Reproducibility, and HPC - Linux.com (blog) - February 10th, 2017 [February 10th, 2017]
- Families Finally Hear From Completely Paralyzed Patients Via New Mind-Reading Device - Singularity Hub - February 12th, 2017 [February 12th, 2017]
- artificial intelligence: the fear of a technological singularity ... - ETtech.com - February 13th, 2017 [February 13th, 2017]
- Holograms Aren't The Stuff of Science Fiction Anymore - Singularity Hub - February 15th, 2017 [February 15th, 2017]
- How the World Has Changed From 1917 to 2017 - Singularity Hub - February 16th, 2017 [February 16th, 2017]
- Preparing for the Singularity - Inverse - February 16th, 2017 [February 16th, 2017]
- Our Health Data Can Save Lives, But We Have to Be Willing to Share - Singularity Hub - February 17th, 2017 [February 17th, 2017]
- Ashes of the Singularity merges with standalone expansion Escalation, no upgrade fee - PCGamesN - February 17th, 2017 [February 17th, 2017]
- Just Stand Inside this Room and it Will Wirelessly Charge Your Phone - Singularity Hub - February 18th, 2017 [February 18th, 2017]
- AMD bundles Ashes of the Singularity with FX processors ahead of Ryzen's launch - PCWorld - February 18th, 2017 [February 18th, 2017]
- Ashes of the Singularity: Escalation being merged with the original game - PC Invasion (blog) - February 18th, 2017 [February 18th, 2017]
- Singularity - GameSpot - February 20th, 2017 [February 20th, 2017]
- The roots of technological singularity can be traced backed to the Stone Age - Wired.co.uk - February 20th, 2017 [February 20th, 2017]
- Jide's new OS is like an Android version of Windows 10's Continuum - The Verge - February 22nd, 2017 [February 22nd, 2017]
- Jide Announces Remix Singularity: The Continuum Alternative for Android - XDA Developers (blog) - February 22nd, 2017 [February 22nd, 2017]
- New Tech Makes Brain Implants Safer and Super Precise - Singularity Hub - February 23rd, 2017 [February 23rd, 2017]
- One Android company wants to use smartphones to make PCs truly dead - BGR - February 23rd, 2017 [February 23rd, 2017]
- Remix tries its hand at the mobile-desktop hybrid OS with Singularity - Android Police - February 23rd, 2017 [February 23rd, 2017]
- Financial Leaders: Make Your Mark on the Future at Exponential Finance - Singularity Hub - February 23rd, 2017 [February 23rd, 2017]
- Remix Singularity is Jide's Android answer to Windows Continuum - SlashGear - February 24th, 2017 [February 24th, 2017]
- AMD Radeon RX 580 Ashes of the Singularity Benchmarks Leaked 4K, Ryzen Combo, CrossFire and More! - Wccftech - February 24th, 2017 [February 24th, 2017]
- Damon Wayans Jr. Joins FX Sci-Fi Comedy Singularity - Den of Geek US - February 24th, 2017 [February 24th, 2017]
- After Man? From Singularity to Specificity - Peace Research Institute Oslo (PRIO) (press release) (blog) - February 24th, 2017 [February 24th, 2017]
- Why the Potential of Augmented Reality Is Greater Than You Think - Singularity Hub - February 24th, 2017 [February 24th, 2017]
- Damon Wayans Jr In Evan Goldberg & Seth Rogen AI comedy - /FILM - February 24th, 2017 [February 24th, 2017]
- Ashes of Singularity: Escalation Gets an Update - CGMagazine - February 24th, 2017 [February 24th, 2017]
- Google Updates: Scuba, Singularity, SMS and suing - The INQUIRER - February 24th, 2017 [February 24th, 2017]
- Singularity Art Show Tonight In San Francisco! - February 25th, 2017 [February 25th, 2017]
- Stardock celebrate v2.1 of Ashes of the Singularity: Escalation with a ... - PCGamesN - February 27th, 2017 [February 27th, 2017]
- Video: AI Is Getting Smarter, Says Singularity University's Neil ... - Wall Street Journal (subscription) (blog) - February 28th, 2017 [February 28th, 2017]
- This Neural Probe Is So Thin, The Brain Doesn't Know It's There - Singularity Hub - March 1st, 2017 [March 1st, 2017]
- Citizen Science Means Anyone Could Discover Planet NineEven You - Singularity Hub - March 1st, 2017 [March 1st, 2017]
- Singularity University establishes new organisation in Denmark - Ministry of Foreign Affairs of Denmark - March 1st, 2017 [March 1st, 2017]
- Singularity University opening organisation in Denmark - The Copenhagen Post - Danish news in english - March 1st, 2017 [March 1st, 2017]
- Does Zapping Your Brain Actually Help You Learn Faster? - Singularity Hub - March 2nd, 2017 [March 2nd, 2017]
- What You Need to Know About Elon Musk's Plan to Fly People to the Moon - Singularity Hub - March 2nd, 2017 [March 2nd, 2017]
- Singularity: Explain It to Me Like I'm 5-Years-Old - Futurism - Futurism - March 3rd, 2017 [March 3rd, 2017]
- Singularity for PC Reviews - Metacritic - March 4th, 2017 [March 4th, 2017]
- Singularity (mathematics) - Wikipedia - March 4th, 2017 [March 4th, 2017]
- See How This House Was 3D Printed in Just 24 Hours - Singularity Hub - March 6th, 2017 [March 6th, 2017]
- NYC's Metrograph theater is running a sci-fi film series featuring Blade Runner, Ex Machina, and Metropolis - The Verge - March 8th, 2017 [March 8th, 2017]
- 3 Exciting Biotech Trends to Watch Closely in 2017 - Singularity Hub - March 9th, 2017 [March 9th, 2017]
- New Burger Robot Will Take Command of the Grill in 50 Fast Food Restaurants - Singularity Hub - March 9th, 2017 [March 9th, 2017]
- Are These Giant Neurons the Seat Of Consciousness in the Brain? - Singularity Hub - March 10th, 2017 [March 10th, 2017]
- How Fully Synthetic Complex Life Just Got a Lot Closer - Singularity Hub - March 12th, 2017 [March 12th, 2017]
- Singularity University launches inaugural Canada Summit | BetaKit - BetaKit - April 8th, 2017 [April 8th, 2017]
- Singularity - Everything2.com - April 8th, 2017 [April 8th, 2017]
- Singularity (Game) - Giant Bomb - April 8th, 2017 [April 8th, 2017]
- Ashes of the Singularity: Escalation on Steam - April 8th, 2017 [April 8th, 2017]
- Approaching the World of Collaboration Singularity - CommsTrader - June 6th, 2017 [June 6th, 2017]
- Beyond Politics: Innovating for a Sustainable Future - Singularity Hub - June 7th, 2017 [June 7th, 2017]
- Tune Into the Future of Fintech at Exponential Finance This Week - Singularity Hub - June 7th, 2017 [June 7th, 2017]
- Experts Weigh in on AI and the Singularity - Futurism - June 7th, 2017 [June 7th, 2017]
- Singularity | Mass Effect Wiki | Fandom powered by Wikia - June 7th, 2017 [June 7th, 2017]
- Quantum Computers Will Analyze Every Financial Model at Once - Singularity Hub - June 8th, 2017 [June 8th, 2017]
- Deloitte and Singularity University Extend Their Relationship To ... - PR Newswire (press release) - June 8th, 2017 [June 8th, 2017]
- Ashes of the Singularity: Escalation 2.3 update adds a new campaign today - PC Gamer - June 8th, 2017 [June 8th, 2017]
- Singularity and Docker | Singularity - June 8th, 2017 [June 8th, 2017]
- Ashes of the Singularity gets a new fully-voiced campaign - PCGamesN - June 9th, 2017 [June 9th, 2017]
- At Exponential Finance, the Singularity University Explores Visionary Applications of Blockchains - Crypto Insider (press release) (blog) - June 9th, 2017 [June 9th, 2017]
- Get It While It's Hot: Why Fintech Is a Goldmine for Investors - Singularity Hub - June 10th, 2017 [June 10th, 2017]
- Forget Police Sketches: Researchers Perfectly Reconstruct Faces by Reading Brainwaves - Singularity Hub - June 14th, 2017 [June 14th, 2017]
- Singularity Summit comes to SA | IT-Online - IT-Online - June 16th, 2017 [June 16th, 2017]
- These 7 Disruptive Technologies Could Be Worth Trillions of Dollars - Singularity Hub - June 17th, 2017 [June 17th, 2017]
- Why Interstellar Travel Will Be Possible Sooner Than You Think - Singularity Hub - June 18th, 2017 [June 18th, 2017]