The Central Data Recording service (CDR) is based
on a series of shell scripts to transfer experimental data
files from
data acquisition systems (DAQ) to CASTOR,
the CERN hierarchical mass storage system. Data files are
transferred over the network and CASTOR manages the migration of
files between disk pools and cartridge tapes. Disk pools are groups of
Unix file systems located on one or several disk servers and managed by a
storage manager called a "stager".
A central database holds
the names and attributes of all files in CASTOR
CDR is targeted at LHC test beams but several high data rate experiments like
NTOF and HARP have used CDR to transfer data at up to 50MB/sec using
multiple data streams. Variants of the basic scripts are also
used to manage data produced in Monte Carlo productions and
mock data challenges for LHC experiments. Large experiments have
dedicated CASTOR disk servers whereas small experiments
and testbeams share two public CDR disk servers.
CDR execution is controlled by a configuration
file which specifies all relevant parameters. The main control
daemon wakes up at periodic intervals and check for new files to
transfer. Once transferred, files may be deleted on the DAQ system
according to high and low water marks in the
configuration file. DAQ files are only candidates
for deletion once they are physically written to tape. All CDR data
transfers are logged and a list of all files copied is created.
CDR aims to provide a reliable and resilient system
for fast and efficient storage data files whilst ensuring data
integrity. An aim has been to keep the scripts are standard as
possible and incorporate customisation for different experiments by
means of the configuration file. The service is operated by the FIO
group in the CERN IT Dept. CDR may be used by any CERN experiment
and best effort support is provided outside normal working hours.
Central Data Recording was first deployed at CERN by Bernd Panzer-Steindel
in the context of CERN's participation in the Esprit project.
The current Service Manager and others who have contributed include:
- Gordon Lee: CERN staff, CDR Service Manager
- Charles Curran: CERN staff, author of copious
documentation
- Leanne Guy: CERN staff, CDR Service Manager
1999 - 2001
- Laura Bertolotto: CERN fellow 1997 -> 1999
|