DELPHI 1998 DATA FLOW SCHEME

A new idea of data processing has been proposed and accepted for this year: try reduce as much as possible the quantity of unnecessary events in the final data sample produced for the analysis. This idea comes from the following facts and necessities:

many analysis in DELPHI need only a small fraction of events in the total sample, hadronic analysis could be done with a few percents of events of the total sample;
need to speed up analysis jobs;
need to speed up reprocessings.

The tagging of the selected events is done by the event-filter in DELANA. After the rejection of the events not tagged by the event-filter the users have a set of data reduced by a factor ~15, so much faster to be analysed and reprocessed. This sample has to satisfy the majority of people doing analysis. The event-filter is also run on Monte Carlo events and its result is put in the pilot record. In this way you can evaluate the effect on the efficiency in your analysis due to the rejection of the events.

The proposal is then to have three different type of processings:

standard processing: here events are processed as usual by DELANA, DSTANA and DAFNE. The removal of not necessary events, the one not tagged by the event-filter is done at the level of DAFNE;
emergency reprocessings: here only the RARE events or the previously event-filtered events are reprocessed;
last reprocessing: all the events are reprocessed once DELANA and DSTANA will reach the final status of tuning and calibration, no events rejection is done here.

IMPORTANT: You are invited to check if the actual event-filter rejects interesting events. If you think that it is necessary to add a new selection in the filter please send an e-mail to pierpaolo.rebecchi@cern.ch.

The standard processing chain is shown in the nearby figure. RAW data come from the pit after the 4th level trigger and they are stored on tape and processed by DELANA as usual. Here DELANA outputs events in two streams depending if the event passed or not the selection of the event-filter:

a stream of the event-filter selected events, containing only RAW data. This events are then used in the emergency reprocessings;
a stream with all the events, containing RAW+TAN+DST for the event-filter selected events, DST only for the non selected events.

Both the two streams are backup-ed on tape. At this level, the event-filter does simply a tag, no rejection is done.

The events are then processed in one single step by DSTANA and DAFNE, no tapes (as the last year) are written between the two packages.

The next step is the removal of events not selected by the event-filter, i.e. the ones stored only with the DST information. This is done in DAFNE in order to monitor the rejection. Once we are confident about the rejection then it will probably be done at DELANA level.

Then DAFNE writes out the usual four streams:

calibration and RARE events stream (RAW+TAN+XSDST on FARM disks) ~2000 events per pb^-1. These events are used for graphical analysis and emergency reprocessings;
analysis stream (XSDST on SHIFT disks), ~6000 events per pb^-1;
HADRONIC events stream (XSDST on SHIFT disks), ~500 events per pb^-1;
selected STIC events stream (XSDST on SHIFT disks), ~30000 events per pb^-1.

In addition DAFNE outputs as usual the scan lists for all and RARE events.

Emergency reprocessings could be done in the following cases:

to re-calibrate detectors or to test new features of the processing codes. This kind of reprocessings can be done from the calibration and RARE events stored in the FARM disks. The time needed is ~1 day per pb^-1;
for conferences or LEPC meetings. This kind of reprocessings can be done from the the event-filter selected events stored by DELANA on tape. The time needed is ~5 days per pb^-1.

In the emergency reprocessings DAFNE does not reject events and the selected STIC events stream is suppressed.

The last reprocessing of the year is done on all the events, no rejection is done. That is achieved when DELANA and DSTANA will reach a reasonable quality and stability in data processing.

DELANA writes always events in DST format and DAFNE doesn't write out the calibration and RARE events stream in the FARM disks.

Last update 22 May 1998

For any problem/question contact :
Pierpaolo Rebecchi (pierpaolo.rebecchi@cern.ch)