Central Data Recording

The Central Data Recording service (CDR) is based on a series of shell scripts to transfer experimental data files from data acquisition systems (DAQ) to CASTOR, the CERN hierarchical mass storage system. Data files are transferred over the network and CASTOR manages the migration of files between disk pools and cartridge tapes. Disk pools are groups of Unix file systems located on one or several disk servers and managed by a storage manager called a "stager". A central database holds the names and attributes of all files in CASTOR

CDR is targeted at LHC test beams but several high data rate experiments like NTOF and HARP have used CDR to transfer data at up to 50MB/sec using multiple data streams. Variants of the basic scripts are also used to manage data produced in Monte Carlo productions and mock data challenges for LHC experiments. Large experiments have dedicated CASTOR disk servers whereas small experiments and testbeams share two public CDR disk servers.

CDR execution is controlled by a configuration file which specifies all relevant parameters. The main control daemon wakes up at periodic intervals and check for new files to transfer. Once transferred, files may be deleted on the DAQ system according to high and low water marks in the configuration file. DAQ files are only candidates for deletion once they are physically written to tape. All CDR data transfers are logged and a list of all files copied is created.

CDR aims to provide a reliable and resilient system for fast and efficient storage data files whilst ensuring data integrity. An aim has been to keep the scripts are standard as possible and incorporate customisation for different experiments by means of the configuration file. The service is operated by the FIO group in the CERN IT Dept. CDR may be used by any CERN experiment and best effort support is provided outside normal working hours.  

Central Data Recording was first deployed at CERN by Bernd Panzer-Steindel in the context of CERN's participation in the Esprit project.  The current Service Manager and others who have contributed include:

  • Gordon Lee: CERN staff, CDR Service Manager
  • Charles Curran: CERN staff, author of copious documentation
  • Leanne Guy: CERN staff, CDR Service Manager 1999 - 2001
  • Laura Bertolotto: CERN fellow 1997 -> 1999

 

 

 

 

The European laboratory for particle physics
Feedback and questions concerning this site should be directed to cdr.support@cern.ch