COSMA6 can be used for galactic simulations

Durham University has built itself a secondhand supercomputer from recycled parts and beefed up its contribution to DiRAC (distributed research utilising advanced computing), the integrated facility for theoretical modelling and HPC-based research in particle physics, astronomy and cosmology.

The Institute for Cosmological Computing (ICC) at Durham, which is in North East England, runs a COSMA5 system as its DiRAC contribution.

There are five DiRAC installations in the UK, which is a world leader in these HPC fields:

The Durham cluster listed above is a COSMA5 system, which features 420 IBM iDataPlex dx360 M4 servers with a 6m720 2.6 GHz Intel Sandy Bridge Xeon E5-2670 CPU cores. There is 53.76TB of DDR3 RAM and Mellanox FDR10 Infiniband in a 2:1 blocking configuration.

It has 2.5PB of DDN storage with two SD12K controllers configured in fully redundant mode. It's served by six GPFS servers connected into the controllers over full FDR and using RDMA over the FDR10 network into the compute cluster. COSMA5 uses the GPSF file system with LSF as its job scheduler.

The ICC and DiRAC needed to strengthen this system and found that the Hartree Centre at Daresbury had a supercomputer it needed rid of. This HPC system was installed in April 2012 but had to go because Daresbury had newer kit.

Durham had a machine room with power and cooling that could take it. Even better, its configuration was remarkably similar to COSMA5.

So HPC, storage and data analytics integrator OCF, and server relocation and data centre migration specialist Technimove dismantled, transported, and rebuilt the machine at the ICC. The whole exercise was funded by the Science and Technology Facilities Council.

COSMA6 arrived at Durham in April 2016, and was installed and tested at the ICC. It now extends Durham's DiRAC system as part of DiRAC 2.5.

COSMA6 has:

The Lustre filesystem and SLURM are used for its job submission system.

COSMA6 racks

Lydia Heck, ICC technical director, said: "While it was quite an effort to bring it to its current state, as it is the same architecture and the same network layout as our previous system, we expect this to run very well."

Durham now has both COSMA5 (6,500 cores) and COSMA6 (8,000 cores) contributing to DiRAC and available for researchers.

Find out how to access and use DiRAC here.

Sponsored: The Joy and Pain of Buying IT - Have Your Say

Read the original:

Durham Uni builds supercomputer from secondhand parts - The Register