Singularity is good friends with Docker. The reason is because the developers use and really like using Docker, and scientists have already put much resources into creating Docker images. Thus, one of our early goals was to support Docker. What can you do?
You can shell, import, run, and exec.
The core of a Docker image is basically a compressed set of files, a set of .tar.gz that (if you look in your Docker image folder on your host machine, you will see. The Docker Registry, which you probably interact with via Docker Hub, serves these layers. These are the layers that you see downloading when you interact with the docker daemon. We are going to use these same layers for Singularity!
The Docker engine communicates with the Docker Hub via the Docker Remote API, and guess what, we can too! The easiest thing to do is create an image, and then pipe a Docker image directly into it from the Docker Registry. You dont need Docker installed on your machine, but you will need a working internet connection. Lets create an ubuntu operating system, from Docker:
Note that the default size is 768MB, you can modify this by adding the --size or -s argument like:
If you arent sure about the size? Try building into a folder first.
Next, lets import a Docker image into it!
Just like Docker has the Dockerfile, Singularity has a file called Singularity that (currently) applications like Singularity Hub know to sniff for. For reproducibility of your containers, our strong recommendation is that you build from these files. Any command that you issue to change a container with --writable is by default not recorded, and your container loses its reproducibility. So lets talk about how to make these files! First, lets look at the absolute minimum requirement:
We would save this content to a file called Singularity and then issue the following commands to bootstrap the image from the file
but just those two lines and doing bootstrap is silly, because we would achieve the same thing by doing:
The power of bootstrap comes with the other stuff that you can do! This means running specific install commands, specifying your containers runscript (what it does when you execute it), adding files, labels, and customizing the environment. Here is a full Singularity file:
In the example above, I am overriding any Dockerfile ENTRYPOINT or CMD because I have defined a %runscript. If I want the Dockerfile ENTRYPOINT to take preference, I would remove the %runscript section. If I want to use CMD instead of ENTRYPOINT, I would again remove the runscript, and add IncludeCmd to the header:
Did you know that you can commit this Singularity file to a Github repo and it will automatically build for you when you push to Singularity Hub?. This will ensure maximum reproducibility of your work.
Docker has two commands in the Dockerfile that have something to do with execution, CMD and ENTRYPOINT. The differences are subtle, but the best description Ive found is the following:
A CMD is to provide defaults for an executing container.
and
An ENTRYPOINT helps you to configure a container that you can run as an executable.
Given the definition, the ENTRYPOINT is most appropriate for the Singularity %runscript, and so using the default bootstrap (whether from a docker:// endpoint or a Singularity spec file) will set the ENTRYPOINT variable as the runscript. You can change this behavior by specifying IncludeCmd: yes in the Spec file (see below). If you provide any sort of %runscript in your Spec file, this overrides anything provided in Docker. In summary, the order of operations is as follows:
In the example above, you probably saw that we referened the docker image first with the uri docker:// and that is important to tell Singularity that it will be pulling Docker layers. To ask for ubuntu, we asked for docker://ubuntu. This uri that we give to Singularity is going to be very important to choose the following Docker metadata items:
When we put those things together, it looks like this:
By default, the minimum requirement is that you specify a repository name (eg, ubuntu) and it will default to the following:
If you provide a version instead of a tag, that will be used instead:
You can have one or the other, both are considered a digest in Docker speak.
If you want to change any of those fields, then just specify what you want in the URI.
For both import and bootstrap using a build spec file, by default we use the Docker Registry index.docker.io. Singularity first tries the call without a token, and then asks for one with pull permissions if the request is defined. However, it may be the case that you want to provide a custom token for a private registry. You have two options. You can either provide a Username and Password in the build specification file (if stored locally and there is no need to share), or (in the case of doing an import or needing to secure the credentials) you can export these variables to environmental variables. We provide instructions for each of these cases:
You can simply specify your additional authentication parameters in the header with the labels Username and Password:
Again, this can be in addition to specification of a custom registry with the Registry parameter.
You can export your username, and password for Singularity as follows:
If you are having trouble, you can test your token by obtaining it on the command line and putting it into an environmental variable, CREDENTIAL:
This should place the token in the environmental variable TOKEN. To test that your token is valid, you can do the following
The above call should return the tags list as expected. And of course you should change the repo name to be one that actually exists that you have credentials for.
Why wont my image bootstrap work? If you cant find an answer on this site, please ping us an issue. If youve found an answer and youd like to see it on the site for others to benefit from, then post to us here.
This entire process will hopefully change in two ways. First, we hope to collapse the image creation and bootstrapping, so you have the option to do them both in one full swing. Second, we hope to eventually figure out some kind of solution to import Docker containers without needing sudo.
Continue reading here:
Singularity and Docker | Singularity
- Downloads - Singularity Viewer - December 25th, 2016 [December 25th, 2016]
- What is Singularity (the)? - Definition from WhatIs.com - January 5th, 2017 [January 5th, 2017]
- When Electronic Witnesses Are Everywhere, No Secret's Safe - Singularity Hub - February 7th, 2017 [February 7th, 2017]
- Report: AMD Ryzen Performance in Ashes of the Singularity Benchmark - PC Perspective - February 7th, 2017 [February 7th, 2017]
- Do you believe in the Singularity? - Patheos (blog) - February 7th, 2017 [February 7th, 2017]
- Editorial Note From the Singularity Hub Team - Singularity Hub - February 7th, 2017 [February 7th, 2017]
- Discover the Most Advanced Industrial Technologies at Exponential Manufacturing - Singularity Hub - February 7th, 2017 [February 7th, 2017]
- 10th Letter looks at nature in the time of the Singularity - Creative Loafing Atlanta - February 7th, 2017 [February 7th, 2017]
- Donald Trump Is the Singularity - Bloomberg View - Bloomberg.com - Bloomberg - February 7th, 2017 [February 7th, 2017]
- Wearable Devices Can Actually Tell When You're About to Get Sick - Singularity Hub - February 7th, 2017 [February 7th, 2017]
- AMD 8-core Ryzen benchmark show up on Ashes Of The Singularity ... - VR-Zone - February 8th, 2017 [February 8th, 2017]
- Robot Cars Can Teach Themselves How to Drive in Virtual Worlds - Singularity Hub - February 8th, 2017 [February 8th, 2017]
- Physicists Unveil Blueprint for a Quantum Computer the Size of a Soccer Field - Singularity Hub - February 10th, 2017 [February 10th, 2017]
- How Robots Helped Create 100000 Jobs at Amazon - Singularity Hub - February 10th, 2017 [February 10th, 2017]
- Ready to Change the World? Apply Now for Singularity University's 2017 Global Solutions Program - Singularity Hub - February 10th, 2017 [February 10th, 2017]
- Singularity Containers for Science, Reproducibility, and HPC - Linux.com (blog) - February 10th, 2017 [February 10th, 2017]
- Families Finally Hear From Completely Paralyzed Patients Via New Mind-Reading Device - Singularity Hub - February 12th, 2017 [February 12th, 2017]
- artificial intelligence: the fear of a technological singularity ... - ETtech.com - February 13th, 2017 [February 13th, 2017]
- Holograms Aren't The Stuff of Science Fiction Anymore - Singularity Hub - February 15th, 2017 [February 15th, 2017]
- How the World Has Changed From 1917 to 2017 - Singularity Hub - February 16th, 2017 [February 16th, 2017]
- Preparing for the Singularity - Inverse - February 16th, 2017 [February 16th, 2017]
- Our Health Data Can Save Lives, But We Have to Be Willing to Share - Singularity Hub - February 17th, 2017 [February 17th, 2017]
- Ashes of the Singularity merges with standalone expansion Escalation, no upgrade fee - PCGamesN - February 17th, 2017 [February 17th, 2017]
- Just Stand Inside this Room and it Will Wirelessly Charge Your Phone - Singularity Hub - February 18th, 2017 [February 18th, 2017]
- AMD bundles Ashes of the Singularity with FX processors ahead of Ryzen's launch - PCWorld - February 18th, 2017 [February 18th, 2017]
- Ashes of the Singularity: Escalation being merged with the original game - PC Invasion (blog) - February 18th, 2017 [February 18th, 2017]
- Singularity - GameSpot - February 20th, 2017 [February 20th, 2017]
- The roots of technological singularity can be traced backed to the Stone Age - Wired.co.uk - February 20th, 2017 [February 20th, 2017]
- Jide's new OS is like an Android version of Windows 10's Continuum - The Verge - February 22nd, 2017 [February 22nd, 2017]
- Jide Announces Remix Singularity: The Continuum Alternative for Android - XDA Developers (blog) - February 22nd, 2017 [February 22nd, 2017]
- New Tech Makes Brain Implants Safer and Super Precise - Singularity Hub - February 23rd, 2017 [February 23rd, 2017]
- One Android company wants to use smartphones to make PCs truly dead - BGR - February 23rd, 2017 [February 23rd, 2017]
- Remix tries its hand at the mobile-desktop hybrid OS with Singularity - Android Police - February 23rd, 2017 [February 23rd, 2017]
- Financial Leaders: Make Your Mark on the Future at Exponential Finance - Singularity Hub - February 23rd, 2017 [February 23rd, 2017]
- Remix Singularity is Jide's Android answer to Windows Continuum - SlashGear - February 24th, 2017 [February 24th, 2017]
- AMD Radeon RX 580 Ashes of the Singularity Benchmarks Leaked 4K, Ryzen Combo, CrossFire and More! - Wccftech - February 24th, 2017 [February 24th, 2017]
- Damon Wayans Jr. Joins FX Sci-Fi Comedy Singularity - Den of Geek US - February 24th, 2017 [February 24th, 2017]
- After Man? From Singularity to Specificity - Peace Research Institute Oslo (PRIO) (press release) (blog) - February 24th, 2017 [February 24th, 2017]
- Why the Potential of Augmented Reality Is Greater Than You Think - Singularity Hub - February 24th, 2017 [February 24th, 2017]
- Damon Wayans Jr In Evan Goldberg & Seth Rogen AI comedy - /FILM - February 24th, 2017 [February 24th, 2017]
- Ashes of Singularity: Escalation Gets an Update - CGMagazine - February 24th, 2017 [February 24th, 2017]
- Google Updates: Scuba, Singularity, SMS and suing - The INQUIRER - February 24th, 2017 [February 24th, 2017]
- Singularity Art Show Tonight In San Francisco! - February 25th, 2017 [February 25th, 2017]
- Stardock celebrate v2.1 of Ashes of the Singularity: Escalation with a ... - PCGamesN - February 27th, 2017 [February 27th, 2017]
- Video: AI Is Getting Smarter, Says Singularity University's Neil ... - Wall Street Journal (subscription) (blog) - February 28th, 2017 [February 28th, 2017]
- This Neural Probe Is So Thin, The Brain Doesn't Know It's There - Singularity Hub - March 1st, 2017 [March 1st, 2017]
- Citizen Science Means Anyone Could Discover Planet NineEven You - Singularity Hub - March 1st, 2017 [March 1st, 2017]
- Singularity University establishes new organisation in Denmark - Ministry of Foreign Affairs of Denmark - March 1st, 2017 [March 1st, 2017]
- Singularity University opening organisation in Denmark - The Copenhagen Post - Danish news in english - March 1st, 2017 [March 1st, 2017]
- Does Zapping Your Brain Actually Help You Learn Faster? - Singularity Hub - March 2nd, 2017 [March 2nd, 2017]
- What You Need to Know About Elon Musk's Plan to Fly People to the Moon - Singularity Hub - March 2nd, 2017 [March 2nd, 2017]
- Singularity: Explain It to Me Like I'm 5-Years-Old - Futurism - Futurism - March 3rd, 2017 [March 3rd, 2017]
- Singularity for PC Reviews - Metacritic - March 4th, 2017 [March 4th, 2017]
- Singularity (mathematics) - Wikipedia - March 4th, 2017 [March 4th, 2017]
- See How This House Was 3D Printed in Just 24 Hours - Singularity Hub - March 6th, 2017 [March 6th, 2017]
- NYC's Metrograph theater is running a sci-fi film series featuring Blade Runner, Ex Machina, and Metropolis - The Verge - March 8th, 2017 [March 8th, 2017]
- 3 Exciting Biotech Trends to Watch Closely in 2017 - Singularity Hub - March 9th, 2017 [March 9th, 2017]
- New Burger Robot Will Take Command of the Grill in 50 Fast Food Restaurants - Singularity Hub - March 9th, 2017 [March 9th, 2017]
- Are These Giant Neurons the Seat Of Consciousness in the Brain? - Singularity Hub - March 10th, 2017 [March 10th, 2017]
- How Fully Synthetic Complex Life Just Got a Lot Closer - Singularity Hub - March 12th, 2017 [March 12th, 2017]
- Singularity University launches inaugural Canada Summit | BetaKit - BetaKit - April 8th, 2017 [April 8th, 2017]
- Singularity - Everything2.com - April 8th, 2017 [April 8th, 2017]
- Singularity (Game) - Giant Bomb - April 8th, 2017 [April 8th, 2017]
- Ashes of the Singularity: Escalation on Steam - April 8th, 2017 [April 8th, 2017]
- Approaching the World of Collaboration Singularity - CommsTrader - June 6th, 2017 [June 6th, 2017]
- Berkeley Lab's Open-Source Spinoff Serves Science | Berkeley Lab - Lawrence Berkeley National Laboratory - June 7th, 2017 [June 7th, 2017]
- Beyond Politics: Innovating for a Sustainable Future - Singularity Hub - June 7th, 2017 [June 7th, 2017]
- Tune Into the Future of Fintech at Exponential Finance This Week - Singularity Hub - June 7th, 2017 [June 7th, 2017]
- Experts Weigh in on AI and the Singularity - Futurism - June 7th, 2017 [June 7th, 2017]
- Singularity | Mass Effect Wiki | Fandom powered by Wikia - June 7th, 2017 [June 7th, 2017]
- Quantum Computers Will Analyze Every Financial Model at Once - Singularity Hub - June 8th, 2017 [June 8th, 2017]
- Deloitte and Singularity University Extend Their Relationship To ... - PR Newswire (press release) - June 8th, 2017 [June 8th, 2017]
- Ashes of the Singularity: Escalation 2.3 update adds a new campaign today - PC Gamer - June 8th, 2017 [June 8th, 2017]
- Ashes of the Singularity gets a new fully-voiced campaign - PCGamesN - June 9th, 2017 [June 9th, 2017]
- At Exponential Finance, the Singularity University Explores Visionary Applications of Blockchains - Crypto Insider (press release) (blog) - June 9th, 2017 [June 9th, 2017]
- Get It While It's Hot: Why Fintech Is a Goldmine for Investors - Singularity Hub - June 10th, 2017 [June 10th, 2017]
- Forget Police Sketches: Researchers Perfectly Reconstruct Faces by Reading Brainwaves - Singularity Hub - June 14th, 2017 [June 14th, 2017]
- Singularity Summit comes to SA | IT-Online - IT-Online - June 16th, 2017 [June 16th, 2017]
- These 7 Disruptive Technologies Could Be Worth Trillions of Dollars - Singularity Hub - June 17th, 2017 [June 17th, 2017]
- Why Interstellar Travel Will Be Possible Sooner Than You Think - Singularity Hub - June 18th, 2017 [June 18th, 2017]