Get SINGLE EVENT data

WorkBook 9.6 How to Pick Events (Interactive and CRAB3)

To avoid the current (Aug 2016 onwards) edmPickEvents.py issues (it doesnt seem to parse correctly), do the output lines directly with edmCopyPickMerge ie:

edmCopyPickMerge outputFile=pickevents.root eventsToProcess=248038:12714964,XXX:YYYY,MMM:ZZZZ inputFiles=/store/data/Run2015A/MET/RECO/PromptReco-v1/000/248/038/00000/546E74FC-5D14-E511-9DCE-02163E0145BA.root

27 Sep 2016

 
edmCopyPickMerge outputFile=pickevents.root eventsToProcess=278817:66:3428691 inputFiles=/store/data/Run2016G/DoubleEG/RECO/PromptReco-v1/000/278/817/00000/C0276B9E-9F63-E611-AF6C-02163E011976.root

Gave:

27-Sep-2016 17:22:59 CEST  Initiating request to open file root://eoscms.cern.ch//eos/cms/store/data/Run2016G/DoubleEG/RECO/PromptReco-v1/000/278/817/00000/C0276B9E-9F63-E611-AF6C-02163E011976.root
[2016-09-27 17:23:01.081517 +0200][Error  ][XRootD            ] [eoscms.cern.ch:1094] Handling error while processing kXR_open (file: /eos/cms/store/data/Run2016G/DoubleEG/RECO/PromptReco-v1/000/278/817/00000/C0276B9E-9F63-E611-AF6C-02163E011976.root, mode: 0660, flags: kXR_open_read kXR_async kXR_retstat ): [ERROR] Error response.
%MSG-w XrdAdaptorInternal:  file_open 27-Sep-2016 17:23:01 CEST pre-events
Failed to open file at URL root://eoscms.cern.ch:1094//eos/cms/store/data/Run2016G/DoubleEG/RECO/PromptReco-v1/000/278/817/00000/C0276B9E-9F63-E611-AF6C-02163E011976.root.
%MSG
[2016-09-27 17:23:01.082416 +0200][Error  ][XRootD            ] [eoscms.cern.ch:1094] Handling error while processing kXR_open (file: /eos/cms/store/data/Run2016G/DoubleEG/RECO/PromptReco-v1/000/278/817/00000/C0276B9E-9F63-E611-AF6C-02163E011976.root?tried=, mode: 0660, flags: kXR_open_read kXR_async kXR_retstat ): [ERROR] Error response.
%MSG-w XrdAdaptorInternal:  file_open 27-Sep-2016 17:23:01 CEST pre-events
Failed to open file at URL root://eoscms.cern.ch:1094//eos/cms/store/data/Run2016G/DoubleEG/RECO/PromptReco-v1/000/278/817/00000/C0276B9E-9F63-E611-AF6C-02163E011976.root?tried=.
%MSG
27-Sep-2016 17:23:01 CEST  Fallback request to file root://xrootd-cms.infn.it//store/data/Run2016G/DoubleEG/RECO/PromptReco-v1/000/278/817/00000/C0276B9E-9F63-E611-AF6C-02163E011976.root
%MSG-w XrdAdaptor:  file_open 27-Sep-2016 17:23:05 CEST pre-events
Data is served from gridpp.rl.ac.uk instead of original site infn.it
%MSG
27-Sep-2016 17:23:08 CEST  Successfully opened file root://xrootd-cms.infn.it//store/data/Run2016G/DoubleEG/RECO/PromptReco-v1/000/278/817/00000/C0276B9E-9F63-E611-AF6C-02163E011976.root
Begin processing the 1st record. Run 278817, Event 3428691, LumiSection 66 at 27-Sep-2016 17:23:15.544 CEST
27-Sep-2016 17:23:16 CEST  Closed file root://xrootd-cms.infn.it//store/data/Run2016G/DoubleEG/RECO/PromptReco-v1/000/278/817/00000/C0276B9E-9F63-E611-AF6C-02163E011976.root

=============================================

MessageLogger Summary

 type     category        sev    module        subroutine        count    total
 ---- -------------------- -- ---------------- ----------------  -----    -----
    1 XrdAdaptor           -w file_open                              1        1
    2 XrdAdaptorInternal   -w file_open                              2        2
    3 fileAction           -s file_close                             1        1
    4 fileAction           -s file_open                              3        3

 type    category    Examples: run/evt        run/evt          run/evt
 ---- -------------------- ---------------- ---------------- ----------------
    1 XrdAdaptor           pre-events                        
    2 XrdAdaptorInternal   pre-events       pre-events       
    3 fileAction           PostEndRun                        
    4 fileAction           pre-events       pre-events       pre-events

Severity    # Occurrences   Total Occurrences
--------    -------------   -----------------
Warning                 3                   3
System                  4                   4

with output file in /afs/cern.ch/user/d/davec/CMSSW_8_0_17/src/Reco/RecoAnalyzer as:
-rw-r--r--. 1 davec zh 2.7M Sep 27 17:23 pickevents.root

NOTE: would not run edmCopyPickMerge in /afs/cern.ch/user/d/davec/CMSSW_7_4_7/src/ since data created with later version of CMSSW (CMSSW_8_0_17).

23 Aug 2016

edmPickevents.py with CRAB3 works (must use CMSSW_7_4_7 or higher), BUT the output file now gets stored at a web address given by the job log file.

INFO 2016-08-22 12:28:09,431:     
Log file is /afs/cern.ch/user/d/davec/CMSSW_7_4_7/src/crab_pickevents_20160819_160735/crab_pickEvents-260540/crab.log

Output dataset:         /SinglePhoton/davec-crab_pickEvents-260540-27375c23c7a5bb979eff0a77ed96ed1b/USER

Output dataset DAS URL:      
https://cmsweb.cern.ch/das/request?input=%2FSinglePhoton%2Fdavec-crab_pickEvents-260540-27375c23c7a5bb979eff0a77ed96ed1b%2FUSER&instance=prod%2Fphys03

Enter, into DAS, the output dataset url:
dataset=/SinglePhoton/davec-crab_pickEvents-260540-27375c23c7a5bb979eff0a77ed96ed1b/USER
or, into the web page url locator

https://cmsweb.cern.ch/das/request?input=%2FSinglePhoton%2Fdavec-crab_pickEvents-260540-27375c23c7a5bb979eff0a77ed96ed1b%2FUSER&instance=prod%2Fphys03

Click on 'files' (see DAS screen below)

Get new screen for:
/store/user/davec/SinglePhoton/crab_pickEvents-260540/160822_095745/0000/pickevents_1.root

DAS-screenview:

Comments from DAS page

CMS provides several ways to download invidual LFNs:
using xrootd CMS tool. Here is an example how you can do it (you may use any CMSSW release):

    source /afs/cern.ch/cms/cmsset_default.sh
    source /afs/cern.ch/cms/LCG/LCG-2/UI/cms_ui_env.sh ===> NOT NEEDED with CRAB3
    cmsrel CMSSW_6_0_0
    cd CMSSW_6_0_0
    cmsenv
    xrdcp root://cms-xrd-global.cern.ch//store/user/davec/SinglePhoton/crab_pickEvents-260540/160822_095745/0000/pickevents_1.root /some/local/path

    opened directly in CMSSW with:

    fileNames = cms.untracked.vstring('root://cms-xrd-global.cern.ch//store/user/davec/SinglePhoton/crab_pickEvents-260540/160822_095745/0000/pickevents_1.root')

My commands to get file:

In area CMSSW_7_4_7/src/
bash
source /afs/cern.ch/cms/cmsset_default.sh
csh ====> since cmsenv not set up as an alias in bash but only in csh
cmsenv
xrdcp root://cms-xrd-global.cern.ch//store/user/davec/SinglePhoton/crab_pickEvents-260540/160822_095745/0000/pickevents_1.root /afs/cern.ch/work/d/davec
[3.226MB/3.226MB][100%][==================================================][550.6kB/s] 

/<2>work/d/davec $ ls *pick*.root
-rw-r--r--. 1 davec zh 3.3M Aug 23 11:02 pickevents_1.root

22 Aug 2016

Using the interactive version of edmPickEvents.py does not work. However using the edmPickEvents.py expected screen output format does work !

WorkBook comments:
To run the script interactively do the following, for example
edmPickEvents.py  "/MET/Run2015A-PromptReco-v1/RECO" 248038:25:12714964
where
*/MET/Run2015A-PromptReco-v1/RECO = is the dataset you want to pick events from. You can only do one per job.
248038:25:12714964 is one event you want to pick. The syntax is Run:Lumi:Event 
(one could also put more than one event by separating the different events with a comma: Run1:Lumi1:Event1, Run2:Lumi2:Event2. 
or
edmPickEvents.py "/MET/Run2015A-PromptReco-v1/RECO" events.txt
where events.txt is a text file that contains = 176163:41:69046624 = ( and others if desired, but one by line ) 

The screen output would look like this:
edmCopyPickMerge outputFile=pickevents.root \
eventsToProcess=248038:12714964 \
inputFiles=/store/data/Run2015A/MET/RECO/PromptReco-v1/000/248/038/00000/546E74FC-5D14-E511-9DCE-02163E0145BA.root


In this case, the user can either paste the above edmCopyPickMerge output 
or 
run edmPickEvents.py with the --runInteractive flag which will run it for you (warning, this can take a long time).

This will create a ROOT file called pickevents.root in the same directory you executed the command from. 
Also note that edmCopyPickMerge script locates edmPickEvents.py configuration file and then uses it with cmsRun. 

Pre Aug 2016

Use single-event.txt with pickEvent_cfg.py.txt
NOTE NOTE - may need to run the single-event.txt file more than once if trying to get events from CASTOR, staging issue perhaps???

more single-event.txt:

cmsRun pickEvent_cfg.py \
inputFiles=/store/data/Run2010B/Photon/RAW/v1/000/146/944/2CB08BA6-34CC-DF11-99F8-003048F118AC.root \
eventsToProcess=146944:254:200578359,146944:254:200582259,146944:255:201078861,146944:255:201078862,146944:255:201135456 \
outputFile=/afs/cern.ch/user/d/davec/scratch0/Run146944-Lumi255-ev78861.root

• This picks three events using the format run:lumi:event with each event separated by a comma.
• change '/store/' which is for castor data to 'file:full....afs...address' for local files

more pickEvent_cfg.py:

import FWCore.ParameterSet.Config as cms
from FWCore.ParameterSet.VarParsing import VarParsing

options = VarParsing ('analysis')
# add a list of strings for events to process
options.register ('eventsToProcess',
                                  '',
                                  VarParsing.multiplicity.list,
                                  VarParsing.varType.string,
                                  "Events to process")
options.parseArguments()

process = cms.Process("PickEvent")
process.source = cms.Source ("PoolSource",
          fileNames = cms.untracked.vstring (options.inputFiles),
          eventsToProcess = cms.untracked.VEventRange (options.eventsToProcess)                               
)

process.Out = cms.OutputModule("PoolOutputModule",
        fileName = cms.untracked.string (options.outputFile)
)

process.end = cms.EndPath(process.Out)

To run single-event.txt and get the data:

chmod u+x single-event.txt  <= to make it an executable
./single-event.txt                     <= to run and get single event data

Locating files to access a single event or lumi section

Use command line syntax on lxplus, ie:

dbsql "find file where dataset=/Photon/Run2011A-May10ReReco-v1/RECO and run=163657 and lumi=34"

Answer:
Using DBS instance at: http://cmsdbsprod.cern.ch/cms_dbs_prod_global/servlet/DBSServlet file
-------------------------------------------------------

/store/data/Run2011A/Photon/RECO/May10ReReco-v1/0000/A6E40CC1-987C-E011-85F6-001A6478A824.root

Old DBS data search methods

DBS notes on searching for data


-- DavidCockerill - 2016-09-29