Deutsches Elektronen-Synchrotron (DESY), an accelerator that is used to analyse molecular structures, is deploying to an IBM-built software-defined storage infrastructure to handle its growing data load.
DESY runs big data experiments for biology and chemistry, using massive detectors, each of which can generate a data stream of about 5Gbps..
Its 1.7-mile-long Petra III accelerator, which is known as a "super microscope", speeds up electrically charged particles – nearly to the speed of light – to generate x-ray radiation that is picked up by the detectors.
The IT project is part of an upgrade to the accelerator, which will be restarted in April 2015. With up to 24 detectors capable of producing 20GB of data every second, the upgrade will generate more data than DESY’s existing IT infrastructure can currently handle.
More articles on big data
Speaking to Computer Weekly, head of DESY IT Volker Gülzow, said one problem it would have to overcome is data injection.
This is important, because DESY scientists need to check the experiment is working. Gülzow said the dataset is then either analysed locally, or DESY offers analysis as a service and remote access to the datasets.
Along with the size of the dataset, he said another big challenge for DESY is input/output (I/O) – the issue of getting data to and from the detectors and the computer rooms for computational analysis.
"There is a lot of I/O and networking strain. But 100Gbps is available today over the WAN. What we usually do is use bandwidth at the network edge by taking multiple 10Gbps links to make a 100Gbps link."
Gülzow said the parallelism offered by the file system in IBM's software-defined storage infrastructure will give DESY the speed and flexibility it requires.