Extreme Mobility of Compute
Singularity containers let users run applications in a Linux environment of their choosing.
Possible uses for Singularity on Biowulf:
Additional Learning Resources
Example definition files written by the NIH HPC staff
These definition files can all be found on GitHub, and the containers built from them are hosted on Singularity hub.
Additionally, a large number of staff maintained definition files and associated helper scripts can be found at this GitHub repo. These are files that staff members use to install containerized apps on the NIH HPC systems.
Please Note: Singularity gives you the ability to install and run applications in your own Linux environment with your own customized software stack. With this ability comes the added responsibility of managing your own Linux environment. While the NIH HPC staff can provide guidance on how to create and use singularity containers, we do not have the resources to manage containers for individual users. If you decide to use Singularity, it is your responsibility to build and manage your own containers.
Creating Singularity containers
To use Singularity on Biowulf, you either need to create your own Singularity container, or use one created by someone else. You have several options to build Singularity containers:
You can find information about installing Singularity on your Linux build system here for 2.x and here for the current 3.x series.
In addition to your own Linux environment, you will also need a definition file to build a Singularity container from scratch. You can find some simple definition files for a variety of Linux distributions in the /example directory of the source code. You can also find a small list of definition files containing popular applications at the top of this page.Detailed documentation about building Singularity container images is available at the Singularity website.
Troubleshooting containers that hang when they run out of memory
A few containers have caused issues on Biowulf by triggering a kernel level bug described in detail here and here. These include fmriprep and nanodisco. The problems follow a predictable pattern:
Binding external directories
Binding a directory to your Singularity container allows you to access files in a host system directory from within your container. By default, Singularity will bind your $HOME directory (along with a few other directories such as /tmp and /dev). You can also bind other directories into your Singularity container yourself.The process is described in detail in the Singularity documentation.
While $HOME is bound to the container by default, there are several filesystems on the NIH HPC systems that you may also want to include.Furthermore, if you are running a job and have allocated local scratch space, you might like to bind mount your lscratch directory to /tmp in the container.
The following command opens a shell in a container while bind-mounting your data directory, /fdb, /scratch, and /lscratch into the same path inside the containerIf you have access to shared data directories, you'll want to add them to the list as well (for example, /data/$USER,/data/mygroup1,/data/mygroup2,/fdb,...).
NIH HPC Staff recommendations for bind mounts on Biowulf
When building containers for use on Biowulf, there are two steps you should take to ensure that users of the container can read and write data to all normal directories. These are the same steps that NIH HPC staff take when containerizing applications for general use.
Step 1. Add directories and symlinks at build time
Data directories hosted on the GPFS file system rely on a series of symbolic links. Singularity can't follow these symlinks if they don't exist within the container, so you need to create them at build time. Other data directories can be automatically created within the container by recent versions of Singularity, but it's still a good idea to create them.
Include the following lines in the %post section of your definition file.
Step 2. Set the --bind option or the SINGULARITY_BIND variable appropriately at run time
Use the following option/argument pair when you run your container.
Singularity as an Installation Medium: faking a native installation
One use case of Singularity is to transparently use software in a container as through it were directly installed on the host system.To accomplish this on our systems, you need to be aware of the shared filesystem locations and bind mount the corresponding directories inside the container, which is more complicated than it seems because we use symbolic links to refer to some of our network storage systems.As a result, you will need to specify some directories in addition to the ones you use directly to ensure that the symbolic link destinations are also bound into the container.
If you wanted to take advantage of a Debian package this way and use it to install software into your home directory, for example samtools and bcftools, you would use a definition file, Singularity, with these contents:
After finalizing the definition file, you can proceed to build the container (of course, on a system where you have sudo or root access):
You can then set up your installation prefix (here, it's $HOME/opt/hts) as follows, making use of symbolic links and a wrapper script:
So if you have added the installation prefix $HOME/opt/hts/bin to your PATH, then calling samtools or bcftools will run those programs from within your container.And because we have arranged to bind mount all the necessary filesystems into the container, the path names you provide for input and output into the programs will be available to the container in the same way.
Interactive Singularity containers
Singularity cannot be run on the Biowulf login node.
To run a Singularity container image on Biowulf interactively, you need to allocate an interactive session, and load the Singularity module. In this sample session (user input in bold), an Ubuntu 16.04 Singularity container is downloaded and run from Docker Hub. If you want to run a local Singularity container instead of downloading one, just replace the DockerHub URL with the path to your container image file.
Expand the tab below to view a demo of interactive Singularity usage.
Singularity interactive container demo
Singularity containers in batch
In this example, singularity will be used to run a TensorFlow example in an Ubuntu 16.04 container. (User input in bold).
First, create a container image on a machine where you have root privileges. These commands were run on a Google Cloud VM instance running an Ubuntu 16.04 image, and the Singularity container was created using this definition file that includes a TensorFlow installation.
Next, copy the TensorFlow script that you want to run into your home directory, or another directory that will be visible from within the container at runtime. (See 'binding external directories' above). In this case, this example script from the TensorFlow website was copied to /home/$USER, and the container was moved to the user's data directory
Then ssh to Biowulf and write a batch script to run the singularity command similar to this:
Submit the job like so:
After the job finishes executing you should see the following output in the slurm*.out file.
Expand the tab below to watch a quick demo of Singularity in batch mode.
Singularity containers in batch demo
Singularity containers on GPU nodes
With the release of Singularity v2.3 it is no longer necessary to install NVIDIA drivers into your Singularity container to access the GPU on a host node. If you still want the deprecated gpu4singularity script that was used to install NVIDIA drivers within containers for use on our GPU nodes you can find it on GitHub.
Now, you can simply use the --nv option to grant your containers GPU support at runtime. Consider the following example in which we will download some TensorFlow models to the user's home directory and then run the latest TensorFlow container from DockerHub to train a model on the MNIST handwritten digit data set using a GPU node.
Expand the tab below to see a demo of installing and using GPU support in a Singularity container.
Using the GPU demo
Using Docker containers with Singularity
Singularity can import, bootstrap, and even run Docker images directly from Docker Hub. For instance, the following commands will start an Ubuntu container running on a compute node with no need for a definition file or container image!And, of course, we remember to set SINGULARITY_BINDPATH appropriately to be able to access all our files.
In this example, we will create a Singularity container image starting from the official continuumio miniconda container on Docker Hub. Then we'll install a number of RNASeq tools. This would allow us to write a pipelinewith, for example, Snakemake and distribute it along with theimage to create an easily shared, reproducible workflow. This definition file also installs a runscript enabling us to treat our container like an executable.
Assuming this file is called rnaseq.def, we can create a Singularity container called rnaseq on our build system with the following commands:
This image contains miniconda and our rnaseq tools and can be called directly as an executable like so:
After copying the image to the NIH HPC systems, allocate an sinteractivesession and test it there
This could be used with a Snakemake file like this
Expand the tab below to see an example of creating a Singularity container to be used as an executable from a Docker image on DockerHub.
Singularity with Docker demo
Follow this link:
- Breakthrough CRISPR Gene Therapy Could Be a 'One and Done' Injection - Singularity Hub - July 5th, 2021
- A COMPASS FOR THE MURKY WATERS OF POLITICS - THISDAY Newspapers - July 5th, 2021
- Pause and beware of 'separate but equal' - News from southeastern Connecticut - theday.com - July 5th, 2021
- Oops I let the humans die: being god of the galaxy in The Fermi Paradox is tricky - Eurogamer.net - July 5th, 2021
- Confronting the Threat of QAnon - The New York Times - June 30th, 2021
- Did the universe begin with a big bang or a little pop? - Herald Review - June 30th, 2021
- Fermi Paradox: Here's What an Alien Civilization Settling the Galaxy Looks Like - Singularity Hub - June 30th, 2021
- Wolfgang Review: Light as a Souffl, and About as Substantial - The New York Times - June 30th, 2021
- In awe of Simone Biles' greatness, with my daughter and 25,000 others - ESPN - June 30th, 2021
- The Heineken beer bot is too helpful, too cute, and too advanced - Yahoo Lifestyle - June 30th, 2021
- Irish music is an ongoing artistic conversation made all the more impactful by the diverse voices engaged in it TLMT's The Month That Was June 2021 -... - June 30th, 2021
- BlackBerry refocuses priorities towards IoT and cybersecurity - TechHQ - June 30th, 2021
- Could a building collapse happen here? One of Utahs top engineers explains - ABC 4 - June 24th, 2021
- An 'Uncrashable' Car? Luminar Says Its Lidar Can Get There - Singularity Hub - June 24th, 2021
- New Study Finds that EV Charge Automation Could Reduce Carbon Impact by up to 14% in US and by 43% in California - PRNewswire - June 24th, 2021
- Sense Debuts Carbon Intensity Tracking in the Sense App - PRNewswire - June 24th, 2021
- Why Flying Cars Could Be Here Within the Decade - Singularity Hub - June 24th, 2021
- CERN: How We're Probing the Universe's Origins Using Record Precision Measurements - Singularity Hub - June 24th, 2021
- SentinelOne driven to stop attacks in their tracks - Tahawul Tech - June 24th, 2021
- '1970s tech and forecasts in the bin': why renewables need digitalisation more than ever | Recharge - Recharge - June 24th, 2021
- Inflation will burst this tech bubble and good riddance to its New Age cranks - Telegraph.co.uk - June 24th, 2021
- Poni is the latest to join Sable Valley's roster with engulfing remix of Deadcrow's 'Fallout' - Dancing Astronaut - June 24th, 2021
- Digital anthropologist and futurist will share how your company can thrive in a digital economy, on next episode of 21st Century Business Forum -... - June 20th, 2021
- These Mice Were Born From Sperm That Spent Almost 6 Years in Space - Singularity Hub - June 20th, 2021
- Anime Review: Vivy Flourite Eyes Song - The Outerhaven - June 20th, 2021
- What Is A Singularity? - WorldAtlas - June 4th, 2021
- 'Naked Singularity,' Thriller With John Boyega, Sells to ... - June 4th, 2021
- Singularity 6 Reveals First Game, Palia, a Genre-Defining ... - June 4th, 2021
- Singularity | Mass Effect Wiki | Fandom - June 4th, 2021
- Singularity 6 reveals friendly MMO community sim Palia as ... - June 4th, 2021
- Palia, New MMO Sim From Singularity 6, Revealed The ... - June 4th, 2021
- Singularity 6s Palia Is A Relaxing MMO - Game Informer - June 4th, 2021
- Quirk Singularity | My Hero Academia Wiki | Fandom - June 4th, 2021
- Singularity Tutorial - SDSC - June 4th, 2021
- SS&C Singularity - June 4th, 2021
- GeForce RTX 3070 Ti performance Leaks in Ashes of the Singularity - guru3d.com - June 4th, 2021
- RTX 3070 Ti Benchmark Results Leaked for Ashes of The Singularity and Geekbench - Tom's Hardware - June 4th, 2021
- Fate/Grand Order Final Singularity Release Date, Cast And Plot - What We Know So Far - Looper - June 4th, 2021
- Modis bulldozing of parliament shows him as the architect of a Hindu Taliban - The Guardian - June 4th, 2021
- Japan Wants to Build Intercontinental Passenger Spaceships by the Early 2040s - Singularity Hub - June 4th, 2021
- Singularity Is Relaunching the 'Singularity Radio' Podcast Network - Singularity Hub - May 20th, 2021
- Scientists Say Black Holes Might Lead to the Birth of New Universes - Interesting Engineering - May 20th, 2021
- Attack the Block sequel will have John Boyega return as Moses - SlashGear - May 20th, 2021
- Lotus to be celebrated at Goodwood Festival of Speed 2021 - AM - May 20th, 2021
- Ferguson: Nothing like a heated quarterback battle - CFL.ca - May 20th, 2021
- Can Darwinian Theory Explain the Rise and Fall of Businesses? - Walter Bradley Center for Natural and Artificial Intelligence - May 20th, 2021
- 8K TVs at everyday prices signal that theres far less reasons not to buy 8K - RedShark News - May 14th, 2021
- How One Round of Gene Therapy Fixed 48 Kids' Immune Systems - Singularity Hub - May 14th, 2021
- Review: The Woman in the Window is one of director Joe Wrights stinkers, even with Amy Adams as star - SF Chronicle Datebook - May 14th, 2021
- The US Just Approved Its First Big Offshore Wind Farm, and It's a Breakthrough for the Industry - Singularity Hub - May 14th, 2021
- SentinelOne Receives Highest Score for Type B Use Case in the Gartner 2021 Critical Capabilities for Endpoint Protection Platforms - StreetInsider.com - May 14th, 2021
- Watch a Jet Suit Pilot Fly Onto a Ship to Trial the Tech for Fighting Pirates - Singularity Hub - May 14th, 2021
- How AI could give us a four-day work week and shape the future of business - The CEO Magazine - May 14th, 2021
- SentinelOne is the Only Vendor to Score Highest Across All Three Gartner Critical Capabilities Use Cases - Business Wire - May 14th, 2021
- Intelligent enterprises and the importance of intellectual property - Express Computer - May 14th, 2021
- Richard Mille just revealed the most extreme watch its ever made - British GQ - May 14th, 2021
- Gravitational singularity - Wikipedia - May 9th, 2021
- Create an Image | Singularity - May 9th, 2021
- GitHub - hpcng/singularity: Singularity: Application ... - May 9th, 2021
- BTS: ARMY makes Vs Singularity top Billboard World Digital Song Sales chart on its third anniversary; V crea - Bollywood Life - May 9th, 2021
- BTS's Singularity Makes Them 1st Artist Ever To Achieve 28 No. 1s On Billboard World Digital Song Sales Chart - soompi - May 9th, 2021
- IBM's New Chip Technology Shows Off the Next Big Step in Moore's Law - Singularity Hub - May 9th, 2021
- The Infinite Rebirth of the Universe - The Great Courses Daily News - May 9th, 2021
- Here are the best Phantom skins in VALORANT - Dot Esports - May 9th, 2021
- Funding Roundup: Derivatives and Defi Draw a Fresh Influx of VC Firms and Private Investors Finance Bitcoin News - Bitcoin News - May 9th, 2021
- Elon Musk Believes 1/10th of a Humans Body is 13.8 Billion Years Old - News18 - May 9th, 2021
- China fans of BTS' V raise $1.1 million in one hour for his birthday - The Straits Times - May 9th, 2021
- Massive Flare Seen Close to Our Solar System: What It Means for Chances of Alien Neighbors - Singularity Hub - May 9th, 2021
- Builder and Breaker - The American Prospect - May 9th, 2021
- Planet Earth Report Biological Fukushima Threatens the Planet to Mutation that Rewired the Human Mind - The Daily Galaxy --Great Discoveries Channel - May 9th, 2021
- Singularity | technology | Britannica - April 13th, 2021
- Anticipate a Return to Live With Singularitys Socially Distanced Pod Experience - The Nocturnal Times - April 13th, 2021
- Scientists Completed the First Human Trial of a Wireless High-Bandwidth Brain-Computer Interface - Singularity Hub - April 13th, 2021
- Soon, the internet will make its own cat photos and then it wont need us - The Verge - April 13th, 2021
- This Robot Taught Itself to Walk in a SimulationThen Went for a Stroll in Berkeley - Singularity Hub - April 13th, 2021
- GPT 3: Behind the Hype - Analytics India Magazine - April 13th, 2021
- Business unusual: wearable tech, touchless surfaces and office 'mud rooms' - MarketWatch - April 13th, 2021
- This Week's Awesome Tech Stories From Around the Web (Through April 10) - Singularity Hub - April 11th, 2021
- Professor of Particle Physics, University of Liverpool - Singularity Hub - April 11th, 2021
- 'Deep Nostalgia' Uses AI to Make Old Photos of Your Relatives Wink, Nod, Dance, and More - Singularity Hub - April 11th, 2021