AtlantisWithDPD

Introduction

The instructions here are for real data DPDs/ESDs. Some instructions on how to produce XML from Monte Carlo AODs/ESDs are also given below.

Before you follow these instruction, make sure that you have set up your account as described in WorkBookSetAccount

XML Files from ESDs/DPDs with Athena 14.5.X

1. Check out and compile the CaloJiveXML package

   cmt co -r CaloJiveXML-00-00-81 Calorimeter/CaloCnv/CaloJiveXML
   cd Calorimeter/CaloCnv/CaloJiveXML/cmt
   source setup.sh
   gmake

2. Copy this job option to your working directory

3. Change the name of the input file and the number of events to be processed

athenaCommonFlags.PoolESDInput.set_Value_and_Lock(['DPD.pool.root'] 
athenaCommonFlags.EvtMax.set_Value_and_Lock(10)

4. You have to be careful of using the same Data Base Tag that was used to produce the DPD. For the last reprocessing, for instance, you should use the following Conditions DB tag

globalflags.ConditionsTag.set_Value_and_Lock('COMCOND-REPC-002-13')

5. Detailed information (like raw data, calibrations factors, etc) can be included or not in he XML file by turning on/off this option

ATLASCosmicFlags.CaloDetailedJiveXML = True

6. To run

athena readesdusingtag.py

7. When you run this JO you may find an error like this:

GeoModelSvc                                          ERROR *** *** Geometry configured through jobOpti ons does not match TagInfo tags! *** 
GeoModelSvc                                          INFO ** Job Option configuration:
GeoModelSvc                                          INFO * ATLAS tag: ATLAS-GEO-04-00-00
GeoModelSvc                                          INFO * InDet tag:
GeoModelSvc                                          INFO * Pixel tag:
GeoModelSvc                                          INFO * SCT   tag:
GeoModelSvc                                          INFO * TRT   tag:
GeoModelSvc                                          INFO * LAr   tag:
GeoModelSvc                                          INFO * Tile  tag:
GeoModelSvc                                          INFO * Muon  tag:
GeoModelSvc                                          INFO * Calo  tag:
GeoModelSvc                                          INFO ** TAG INFO configuration:
GeoModelSvc                                          INFO * ATLAS tag: ATLAS-Comm-09-00-00

This means that you should change the detector description that you are using(ATLAS-GEO-04-00-00 ) by that one suggested by Athena (ATLAS-Comm-09-00-00)

XML Files from ESDs/DPDs with Athena 15.5.3.3

1. Copy this job option to your working directory

2. In the beginning of the the JO, put the path to the ESD files

DIR='/tmp/asevedo/data09_900GeV.00140541.physics_MinBias.recon.ESD.f174/*'

3. Set the number of events to be processed

athenaCommonFlags.EvtMax.set_Value_and_Lock(2)

4. In the very end of the JO, you can specify the directory to where you want to dump the xml files (Note that JiveXML is the file prefix)

AlgoJiveXML.StreamTools+= [JiveXML__StreamToFileTool(FileNamePrefix="/tmp/asevedo/colisions/JiveXML")]

5. Get the JO CaloJiveXML_DataTypes.py. In this JO you can set the level of details that you want to have in the XML files (like digits, calibrations factores, etc)

get_files CaloJiveXML_DataTypes.py

6. To run

athena xmlFromESD_15.5.py

XML Files with TAG_COMM with 15.5.3.3

Commissioning TAGs can be used to produce XML files for specific events. Detailed information about TAGs can be found here: CommissioningTag.

For now, the TAGs can be used to read ESDs, but not DPD. This option might be available in the future.

If you have the ESD data set in your local disk, do

1. Get the TAG_COMM files from the grid

source /afs/cern.ch/atlas/offline/external/GRID/ddm/DQ2Clients/setup.sh
voms-proxy-init -voms atlas
dq2-get data08_cosmag.00088464.physics_L1Calo.recon.TAG_COMM.o4_r653/

2. Make a pool file catalogue

pool_insertFileToCatalog path_to_ESDs/*

A file called PoolFileCatalog.xml will be created in the directory.

3. Copy this job option to your working directory. In the beginning of the the JO, put the path to the TAG files

DIR='/tmp/asevedo/data09_cos.00134772.physics_L1Calo.merge.TAG_COMM.f153_m185/*'

and put the event number you want to select in the line just below athenaCommonFlags.PoolESDInput

athenaCommonFlags.PoolInputQuery.set_Value_and_Lock("EventNumber==545940 || EventNumber==460325")

Only XML files for these specified events will be produced . It is also possible to use other variables available in the TAG_COMM to select events, like total MET

athenaCommonFlags.PoolInputQuery.set_Value_and_Lock("CellMET > 200000")

4. Set the number of events to be processed

athenaCommonFlags.EvtMax.set_Value_and_Lock(2)

5. In the very end of the JO, you can specify the directory to where you want to dump the xml files (Note that JiveXML is the file prefix)

AlgoJiveXML.StreamTools+= [JiveXML__StreamToFileTool(FileNamePrefix="/tmp/asevedo/colisions/JiveXML")]

6. Get the JO CaloJiveXML_DataTypes.py. In this JO you can set the level of details that you want to have in the XML files (like digits, calibrations factors, etc)

get_files CaloJiveXML_DataTypes.py

7. To run

athena xmlFromTAG_15.5.py

If the ESD files are not in the local disk, one can use ELSSI to find out in which ESD file the desired event is, and get just that ESD from the grid. Note that you need to have your grid certificate loaded in your browser to access this site. If you are not familiar with ELSSI, please take a look at this tutorial

1. Get the TAG_COMM files from the grid

source /afs/cern.ch/atlas/offline/external/GRID/ddm/DQ2Clients/setup.sh
voms-proxy-init -voms atlas
dq2-get data08_cosmag.00088464.physics_L1Calo.recon.TAG_COMM.o4_r653/

2. In ELSSI, select the dataset you want. In the "Physics Attributes" panel, you will find a list of variable that you can use to apply the selection, like CellMET, jet_pt, EventNumber, etc. See the link tutorial above for details.

3. Once you have applied your selections, go to "Display Results Panel" and click on "AOD/guids" to see the ID of the Files which contain the events that passed the selection. An example is given below.

4. With the File ID, you can get the specific ESDs that you need. You can use dq2-ls command to see the files. For this example, we have

dq2-ls -f data09_cos.00135356.physics_L1Calo.recon.ESD.f157

data09_cos.00135356.physics_L1Calo.recon.ESD.f157._lb0000._SFO-2._0024.1   44A07431-D9BB-DE11-B550-003048D3765A
data09_cos.00135356.physics_L1Calo.recon.ESD.f157._lb0000._SFO-4._0038.1   12BC4519-D7BB-DE11-BF02-00304896B9A0
data09_cos.00135356.physics_L1Calo.recon.ESD.f157._lb0000._SFO-1._0040.1   907A55D0-D8BB-DE11-BFF5-003048D37510
data09_cos.00135356.physics_L1Calo.recon.ESD.f157._lb0000._SFO-2._0023.1   CC3C2E21-D9BB-DE11-BAB1-00304867811E
data09_cos.00135356.physics_L1Calo.recon.ESD.f157._lb0000._SFO-4._0025.1   F8D27005-D7BB-DE11-A04D-003048D2BB50
data09_cos.00135356.physics_L1Calo.recon.ESD.f157._lb0000._SFO-4._0040.1   74023630-D7BB-DE11-9F45-003048D37458   

5. Once you the get the ESDs, follow the same steps from step 2 of previous instructions

XML Files with TAG_COMM with 15.6.3.6

A tag file is a list of event references. The Commissioning TAGs can be used to read selected events from ESD and to produce XML file for event display. Note that the files to which they point (i.e. its ESD) must be accessible, and a catalog that can translate the references into local file names must be provided when tags are used.

1. Get the TAG_COMM files from the grid

source /afs/cern.ch/atlas/offline/external/GRID/ddm/DQ2Clients/setup.sh
voms-proxy-init -voms atlas
dq2-ls -r data09_900GeV.00142191.physics_MinBias.merge.TAG_COMM.r988*
dq2-get data09_900GeV.00142191.physics_MinBias.merge.TAG_COMM.r988_p63/

2. Create the POOL File catalogue from a dataset (not a container what dataset name ends with '/')

dq2-ls -r data09_900GeV.00142191.physics_MinBias.recon.ESD.r988*
dq2-ls -L CERN-PROD_DATADISK -P -R srm://srm-atlas.cern.ch^rfio: data09_900GeV.00142191.physics_MinBias.recon.ESD.r988*

Please find first what replica the data is on using dq2-ls -r, and give it to the dq2-ls -L replica to write the PoolFileCatalog.xml.
If you have the ESD data set in your local disk, then use "pool_insertFileToCatalog path_to_ESDs/*"

3. Set stage for CERN-PROD_DATADISK

export STAGE_SVCCLASS=atldata

Please use atlcal with CERN-PROD_TZERO.

4. Update the JiveXML package(s) before producing xml-files, and get the CaloJiveXML_DataTypes.py

cmt co -r CaloJiveXML-00-01-25 Calorimeter/CaloCnv/CaloJiveXML (and compile it)
get_files -jo CaloJiveXML_DataTypes.py (and modify it)

In this JO you can set the level of details that you want to have in the XML files (like digits, calibrations factors, etc) by True or False. Particular information about how to run JiveXML in different releases can be found here: JiveXMLReleseIssues

5. Run with this this job option in your working directory

athena readesdusingtag_wiki.py

Since release 15.6.x and/or with data10, it is recommended to use RecExCommon with Autoconfiguration. See details here: JiveXMLWithAODorESD

Don't forget to update the path to the TAG files to your working directory:
DIR='/tmp/minsuk/data09_900GeV.00142191.physics_MinBias.merge.TAG_COMM.r988_p63_tid105048_00/*'

The followings will produce XML file(s) for the specified event(s) and the selected event(s) by variables available in the TAG_COMM:
athenaCommonFlags.PoolInputQuery.set_Value_and_Lock("EventNumber==319475")
athenaCommonFlags.PoolInputQuery.set_Value_and_Lock("abs(LArECtimeDiff) < 5 && abs(MBTStimeDiff) < 10")

6. Download Atlantis and run

java -jar atlantis.jar JiveXML_142191_319475.xml

Event display for 900 GeV Collision

XML Files with TAG_COMM with 15.6.7.8

1. Get the TAG_COMM files from the grid

source /afs/cern.ch/atlas/offline/external/GRID/ddm/DQ2Clients/setup.sh
voms-proxy-init -voms atlas
dq2-ls -r data10_7TeV.00152166.physics_MinBias.merge.TAG_COMM.*
dq2-get data10_7TeV.00152166.physics_MinBias.merge.TAG_COMM.f236_m426

2. Create the POOL File catalogue from a dataset (not a container what dataset name ends with '/')

dq2-ls -r data10_7TeV.00152166.physics_MinBias.recon.ESD.f236*
dq2-ls -L CERN-PROD_DATADISK -P -R srm://srm-atlas.cern.ch^rfio: data10_7TeV.00152166.physics_MinBias.recon.ESD.f236

Please find first what replica the data is on using dq2-ls -r, and give it to the dq2-ls -L replica to write the PoolFileCatalog.xml.
If you have the ESD data set in your local disk, then use "pool_insertFileToCatalog path_to_ESDs/*"

3. Set stage for CERN-PROD_DATADISK

export STAGE_SVCCLASS=atldata

Please use atlcal with CERN-PROD_TZERO.

4. Update all possible JiveXML packages before producing xml-files and compile them. For example,

cmt co -r CaloJiveXML-00-01-25 Calorimeter/CaloCnv/CaloJiveXML
cmt co -r AnalysisJiveXML-00-02-11 PhysicsAnalysis/AnalysisEventDisplay/AnalysisJiveXML

Note: update AnalysisJiveXML to mask CaloMuonCollection, which causes lots of confusion in high-pT collision events.

5. Use this CaloJiveXML_DataTypes.py with switching-on pulse shape

In this JO you can set the level of details that you want to have in the XML files (like digits, calibrations factors, etc) by True or False. Particular information about how to run JiveXML in different releases can be found here: JiveXMLReleseIssues

6. Run with this readesdusingtag_try.py in your working directory

athena readesdusingtag_fun.py

Since release 15.6.x and/or with data10, it is recommended to use RecExCommon with Autoconfiguration. See details here: JiveXMLWithAODorESD

Don't forget to update the path to the TAG files to your working directory:

7. Download Atlantis and run

java -jar atlantis.jar JiveXML_152166_810258.xml

XML Files on 15.6.9.x

It is recommended to submit a job to Grid to get XML files. If you would like to access data in CASTOR and run locally, please find the Lumi Block Number (LBN) for the selected event in the Run. The PoolFileCatalog.xml what you produce allows to access data over CASTOR, i.e., you don't need to copy every data in your computer. It should be noted that it works ONLY if the selected dataset is located in CERN-PROD_DATADISK. On Aug 11, 2010, found that data10_7TeV.00158548.physics_L1Calo.recon.ESD.f273 doesn't exist in CERN-PROD_DATADISK any more. So please use other datasets which exist in CERN-PROD_DATADISK for your test because it is impossible to specify which datasets will remain in CERN-PROD_DATADISK.

pathena \
     --eventPickEvtList selevt.txt\
     --eventPickDataType ESD\
     --eventPickStreamName physics_L1Calo\
     --eventPickDS '*r129*,*f259*,*f26*,*f27*'\
     --supStream THIST\
     --outDS user10.MinsukKim.20100712\
     --extOutFile 'JiveXML_*.xml'\
     --individualOutDS\
     --dbRelease LATEST\
     readesd_wiki.py

where

• readesd_wiki.py is any jobOptions to run on ESD, with the additional flag rec.doJiveXML=True. (It is OK to use copyesd_wiki.py which is a jobOptions to extract single event. In this case do not need to specify JiveXML*.xml files in the grid output directory.)
• selevt.txt is a simple ascii file specifying the run and event number. Here, selevt.txt has one line: 158548 2486978

You need to copy these jobOptions readesd_wiki.py, copyesd_wiki.py and CaloJiveXML_DataTypes.py to your working directory.

1. Get the TAG file(s) from the grid to get the LBN

source /afs/cern.ch/atlas/offline/external/GRID/ddm/DQ2Clients/setup.sh
voms-proxy-init -voms atlas
dq2-ls -r data10_7TeV.00158548.physics_L1Calo.merge.TAG.f27*
dq2-get data10_7TeV.00158548.physics_L1Calo.merge.TAG.f27*
cd data10_7TeV.00158116.physics_L1Calo.merge.TAG.f273_m543_m542
root -l 
   root [0] TChain ch("POOLCollectionTree")
   root [1] ch.Add("data10_7TeV.00158548.physics_L1Calo.merge.TAG*")
   root [2] POOLCollectionTree->Scan("EventNumber:LumiBlockN","EventNumber==2486978")

This will give the Lumi Block Number = 120.

2. Create the POOL File catalogue from a dataset (not a container what dataset name ends with '/')

export DQ2_LOCAL_SITE_ID=CERN-PROD_DATADISK
export STAGE_SVCCLASS=atldata
dq2-ls -r data10_7TeV.00158548.physics_L1Calo.recon.ESD.f27*
dq2-ls -P -R srm://srm-atlas.cern.ch^rfio: data10_7TeV.00158548.physics_L1Calo.recon.ESD.f273
grep lb0120 PoolFileCatalog.xml | grep castor > temp.dat

This will list the 13 files in CASTOR.

3. Run this in your working directory to filter out the event. Note that the PoolFileCatalog.xml should be in the same directory. This is an example how to run with one file. Or run with this copyesd_wiki.py with all files. The command acmd.py exists after setting up athena.

acmd.py filter-files -o filtered.esd.pool.root -s '[(158548,2486978)]' "rfio:/castor/cern.ch/grid/atlas/atlasdatadisk/data10_7TeV/ESD/f273/data10_7TeV.00158548.physics_L1Calo.recon.ESD.f273/data10_7TeV.00158548.physics_L1Calo.recon.ESD.f273._lb0120._0009.1"

4. Run this readesd_wiki.py to make the XML file

athena -c 'PoolESDFile="filtered.esd.pool.root"' readesd_wiki.py

5. Download Atlantis and run it with this config.xml. It is recommened to update every JiveXML package in the Atlantis if you want to have no problem with tracking and the pulse shpaes.

java -jar atlantis.jar -c config.xml JiveXML_158548_2486978.xml

Jets and MET in Atlantis

The Jets and MET containers that one wants to retrieve can be chosen in the JO. Look at the previous jobObption for an example. In the very end of it, you will see the options to set the Jets and MET containers

from RecJiveXML.RecJiveXMLConf import JiveXML__JetRecJetRetriever
theJetRecJetRetriever = JiveXML__JetRecJetRetriever (name = "JetRecJetRetriever")
theJetRecJetRetriever.FavouriteJetCollection="AntiKt4H1TopoJets"
### read all other jet collections (or select one like "Cone4H1TopoJets"): 
theJetRecJetRetriever.OtherJetCollections=[]

from RecJiveXML.RecJiveXMLConf import JiveXML__MissingETRetriever
theMissingETRetriever = JiveXML__MissingETRetriever (name = "MissingETRetriever")
theMissingETRetriever.FavouriteMissingETCollection="MET_RefFinal"
theMissingETRetriever.OtherMissingETCollections=["MET_RefMuon","MET_Final"]

XML Files from MC AODs

The instructions on how to run JiveXML with MC AODs can be found in this link JiveXMLWithAODorESD.

Atlantis Event Display

The Atlantis Event Display can be download from the Atlantis web page. Two examples of event display are shown below: a single beam event (run 87861) and a MC QCD event from AOD.

The LAr and Tile pulse shapes can be displayed by clicking on a cell with the “Pick” function from the Atlantis menu. In order to display only the digits, without the red curve, one should press d and then click on a cell.

• Single Beam Event:
  

• AOD Dijet Event: