The Big Data Bang

How do you store 15 million gigabytes of data?

Photo: CERN

CERN's Large Hadron Collider, when it is switched on in 2007, will record the results of collisions between two groups of protons, each traveling in the opposite direction with the energy equivalent to that of a car moving 25 000 kilometers per hour.

The collisions, which will occur at four points on the LHC's circular tunnel [photo] where gargantuan detectors are being assembled, will allow physicists to get a glimpse of particles that have long since disappeared from our universe, such as the hypothetical Higgs boson, considered to be the last major missing piece in the so‑called Standard Model of particle physics.

The largest of the detectors, ATLAS, fills an underground cavern over six stories high. It is expected to detect 1 billion collisions per second. Given that a single collision results in essentially an image of hundreds of particle tracks produced in the detector, equivalent to several megabytes of digital data, the data rate is mindboggling. Only by extremely selective filtering of about one in a million events do the scientists stand a chance of being able to store the data. And even then, the cumulative data expected during a year will amount to about 15 million gigabytes.

CERN has no way to store so much data centrally, and even if it did, a central repository would be inconvenient for most of the far-flung institutions involved in the LHC project. So building a global infrastructure for storing and analyzing the LHC data is necessary. This infrastructure began with the LHC Computing Grid, launched in 2002, and now encompasses multiple grids, including the Enabling Grids for E‑sciencE (EGEE) project.

The LHC data provide an ideal application for the EGEE project to start with, but ultimately, if this initiative's ambitions are fulfilled, particle physics will become just one of a broad range of scientific projects running on this general-purpose grid. —F.G. & F.G.