Tutorial on How to Analyze Pixel Data

Pixel Reconstruction Flow

Steps (described in detail here: https://twiki.cern.ch/twiki/bin/view/CMS/PixelOfflineSoftware)

• Raw2Digi:
  • performs conversion between raw data from Pixel FEDs to Digi data format (row, column, pixel charge)
  • details: https://twiki.cern.ch/twiki/bin/view/CMS/PixelRaw2Digi
• Pixel Clusterizer:
  • Forms clusters from pixel digis
  • It loops over all detector modules. In each detector module selects all pixels above certain threshold.
  • Forms clusters around pixel seeds (pixels above certain seed threshold)
  • details: http://cmslxr.fnal.gov/lxr/source/RecoLocalTracker/SiPixelRecHits/
  • source code: http://cmslxr.fnal.gov/lxr/source/RecoLocalTracker/SiPixelClusterizer/
• RecHits:
  • Takes clusters from previous step and determines the hit position: (x +/- error_x, y +/- error_y)
  • details: http://cmslxr.fnal.gov/lxr/source/RecoLocalTracker/SiPixelRecHits/
  • source code: http://cmslxr.fnal.gov/lxr/source/RecoLocalTracker/SiPixelRecHits/
• Finally, PixelRecHits are used in tracking reconstruction

What is the Pixel Ntuplizer

• As soon as CRAFT data is reprocessed with a new version of CMSSW and/or global tag, the "standard" pixel ntuple is produced.
• The code that produces the pixel ntuple resides in DPGAnalysis/SiPixelTools and is refered to as "pixel ntuplizer"
  • for some reason I cannot see the source code in lxr, but it can be seen in the CVS browser: http://cmssw.cvs.cern.ch/cgi-bin/cmssw.cgi/CMSSW/DPGAnalysis/SiPixelTools/
• Detailed description of all variable in the pixel ntuple can be found by clicking on "CMSSiPixelNtupleDocumentation": https://twiki.cern.ch/twiki/bin/view/CMS/CMSSiPixelNtupleDocumentation
  • This program takes tracks as input, loops over all PixelRecHits of each track and stores information about each PixelRecHit. Information about PixelRecHits which are off-track is also stored, but only for detector modules on which a track passes through and has a track PixelRecHit.
  • Information about tracks is stored in a separate Tree

How to Find the Pixel Ntuple

• Information about the most recent pixel ntuple (both data and MC) is kept up to date here: https://twiki.cern.ch/twiki/bin/view/CMS/CMSSiPixelCosmicCommData

• The latest data and MC ntuples, as of 04/24/2009, are:
  • Data Ntuple
    • Location: /castor/cern.ch/cms/store/group/tracker/pixel/PixelNtuplizer/
    • File: SuperPointing_CRAFT_ALL_V11_NtuplizerV00-01-03_Data_AllTrkAlgos.root
    • made from /Cosmics/Commissioning08_CRAFT_ALL_V11_227_Tosca090216_ReReco_FromSuperPointing_v2/RAW-RECO in DBS
    • Global Tag: CRAFT_ALL_V11
    • TrackRefitter: All algorithms: ctfWithMaterialTracksP5, rsWithMaterialTracksP5, cosmictrackfinderP5
    • CMSSW release: CMSSW_2_2_7
    • Version of Ntuplizer: DPGAnalysis/SiPixelTools V00-01-03
    • produced by: Matthew Searle
  • MC Ntuple with increased thresholds bpix/fpix = 5200/4500
    • Location: /castor/cern.ch/cms/store/group/tracker/pixel/PixelNtuplizer/
    • Files: SuperPointing_COSMMC_22X_V1_newthresholds_NtuplizerV00-01-03_MC_alltrkalgos.root
    • made from /TkAlCosmics4T/Summer08_COSMMC_21X_V1_SuperPointing_v4/RAW-RECO in DBS
    • global Tag: COSMMC_22X_V1
    • TrackRefitter: All algorithms: ctfWithMaterialTracksP5, rsWithMaterialTracksP5, cosmictrackfinderP5
    • Version of Ntuplizer: V00-01-03 DPGAnalysis/SiPixelTools
    • release used for ntupling CMSSW_2_2_7
    • produced by: GavrilGiurgiu

• To access the pixel ntuple, log into the lxplus machines at CERN:
  • [ggiurgiu@ggiurgiu ~]$ ssh lxplus.cern.ch
• List all files in the PixelNtuplizer area:
  • [lxcmsf2] nsls /castor/cern.ch/cms/store/group/tracker/pixel/PixelNtuplizer/
    • ...
    • SuperPointing_CRAFT_ALL_V11_NtuplizerV00-01-03_Data_AllTrkAlgos.root
    • ...
• Copy the pixel ntuple to your scratch (or tmp) area:
  • [lxcmsf2] /localscratch/g/ggiurgiu > rfcp /castor/cern.ch/cms/store/group/tracker/pixel/PixelNtuplize /SuperPointing_CRAFT_ALL_V11_NtuplizerV00-01-03_Data_AllTrkAlgos.root ./
• The file is 329M so it's not hard to copy
• It takes 10 minutes to copy it by simple scp from CASTOR at cern to my scratch area at FNAL. (Fancier methods might be used to copy (maybe Phedex ?) but not worth the complication)
• You can find the latest data pixel ntuple on FNAL machine:
  • cmslpc.fnal.gov:/uscms_data/d2/ggiurgiu/SuperPointing_CRAFT_ALL_V11_NtuplizerV00-01-03_Data_AllTrkAlgos.root
• You can find the latest MC pixel ntuple in the same place:
  • cmslpc.fnal.gov:/uscms_data/d2/ggiurgiu/SuperPointing_COSMMC_22X_V1_newthresholds_NtuplizerV00-01-03_MC_alltrkalgos.root

How to Run the Pixel Ntuplizer Yourself

• Get a kerberos ticket and log into a cmslpc.fnal.gov machine:

[ggiurgiu@ggiurgiu ~]$ kinit ... [ggiurgiu@ggiurgiu ~]$ ssh cmslpc.fnal.gov

• Setup a CMSSW project somewhere in your area:

[ggiurgiu@cmslpc03 ~/work]$ scramv1 project CMSSW CMSSW_2_2_7

[ggiurgiu@cmslpc03 ~/work]$ cd CMSSW_2_2_7/

[ggiurgiu@cmslpc03 CMSSW_2_2_7]$ eval `scramv1 runtime -csh`

• Get a CERN kerberos ticket. It is needed to check out packages from CVS

[ggiurgiu@cmslpc07 CMSSW_2_2_7]$ kserver_init ...

• Go the the source area "src":

[ggiurgiu@cmslpc07 CMSSW_2_2_7]$ cd src/

• Check out the pixel ntuplizer package:

[ggiurgiu@cmslpc07 CMSSW_2_2_7]$ cvs co -r V00-01-03 DPGAnalysis/SiPixelTools

• Compile and build the package:

[ggiurgiu@cmslpc07 src]$ scramv1 b

• Go to the "test" area in this package:

[ggiurgiu@cmslpc07 src]$ cd DPGAnalysis/SiPixelTools/test/

• The configuration file used to run the ntuplizer is:

pixelNtuplizer_RealData_cfg38T.py

• Important things in the configuration file:
  • Global tag: process.GlobalTag.globaltag = "CRAFT_ALL_V11::All" (a global tag is a set of Data Base (DB) conditions)
  • Input file: /store/data/Commissioning08/Cosmics/RAW-RECO/CRAFT_ALL_V9_SuperPointing_225-v3/0006/26B79166-7C00-DE11-AA81-0030486790A0.root
  • Output file: SuperPointing_test_threealgos.root

• Run the ntuplizer:

[ggiurgiu@cmslpc07 test]$ cmsRun pixelNtuplizer_RealData_cfg38T.py

• Will see some scary printouts:

2009-04-23 11:57:38 DQMNet[17321]: NOTE: DQM server at localhost:9090 is unavailable. Connection will be established automatically on the background once the server becomes available. Error from the attempt was: Network operation connect() failed (because of System error: Connection refused (<#111,#108>))

• This is OK, I think... as long as we get to see these printouts:

23-Apr-2009 11:58:59 CDT Initiating request to open file dcap://cmsdca3.fnal.gov:24142/pnfs/fnal.gov/usr/cms/WAX/11/store/data/Commissioning08/Cosmics/RAW-RECO/CRAFT_ALL_V9_SuperPointing_225-v3/0006/26B79166-7C00-DE11-AA81-0030486790A0.root

23-Apr-2009 11:59:28 CDT Successfully opened file dcap://cmsdca3.fnal.gov:24142/pnfs/fnal.gov/usr/cms/WAX/11/store/data/Commissioning08/Cosmics/RAW-RECO/CRAFT_ALL_V9_SuperPointing_225-v3/0006/26B79166-7C00-DE11-AA81-0030486790A0.root

...

Begin processing the 1st record. Run 68129, Event 453, LumiSection 1 at 23-Apr-2009 12:01:28 CDT

Begin processing the 11th record. Run 68129, Event 5065, LumiSection 1 at 23-Apr-2009 12:01:33 CDT

Begin processing the 21st record. Run 68129, Event 19065, LumiSection 1 at 23-Apr-2009 12:01:34 CDT

...

Begin processing the 4641st record. Run 68129, Event 10903429, LumiSection 205 at 23-Apr-2009 12:03:25 CDT

23-Apr-2009 12:03:25 CDT Closed file dcap://cmsdca3.fnal.gov:24142/pnfs/fnal.gov/usr/cms/WAX/11/store/data/Commissioning08/Cosmics/RAW-RECO/CRAFT_ALL_V9_SuperPointing_225-v3/0006/26B79166-7C00-DE11-AA81-0030486790A0.root

MessageLogger Summary ...

• Keep in mind that whenever running the ntuplizer you should make sure you use the correct Global Tag:
  • normally the same global tag used to reconstruct the data.
• Detailed information pixel DB files: https://twiki.cern.ch/twiki/bin/view/CMS/PixelDBTags
• Details about Global Tags: https://twiki.cern.ch/twiki/bin/view/CMS/SWGuideFrontierConditions

How to Quickly Look the Pixel Ntuple

• If you got here, you should have the output ntuple in your working directory:
• You can look at the ntuple:

[ggiurgiu@cmslpc07 test]$ root -l SuperPointing_test_threealgos.root

root [1] TBrowser t

• Click on "ROOT Files"
• Click on file name: "SuperPointing...root"
• There are three folders: "cosmtfNtuple", "ctfNtuple", "rsNtuple".
• Each folder corresponds to the tracking algorithm used to reconstruct the cosmic muon tracks:
  • "Cosmic Track Finder = cosmtf"
  • "Combinatorial Track Finder = ctf"
  • "Road Search = rs"

• Cosmic Track Finder is an algorithm specialized for cosmics. It reconstructs one track per event.
• Combinatorial Track Finder and Road Search are used to reconstruct collisions as well

• Ongoing comparison between these three algorithms by Matt Searle: http://indico.cern.ch/conferenceDisplay.py?confId=55035 "Comparing ntuplizer output for the three different tracking algorithms", Matthew Paul Searle (UCD)

• Inside each of the 3 folders:
  • PixNtuple - filled for each cluster
  • TrackNtuple - filled for each track
    • Click on "PixNtuple" yellow folder
    • Click on "RecHit" yellow folder
    • Click on "residualX" green leaf
    • You get a plot of the X residuals. To zoom in and apply some selection cuts, type this in the Root session:

root [5] PixNtuple->Draw("residualX", "abs(residualX)<0.1 && ladder>0")

• • • You get a zoomed in plot of the X residuals for the BPIX detector (units on the horisontal axis are in cm)

How to Write a Root Script to analyze Pixel Ntuple

• A more programmatic approach is to write a Root script to analyze for example the "ctfNtuple":

[ggiurgiu@cmslpc07 test]$ root -l

root [0] TChain chain("ctfNtuple/PixNtuple")

root [1] chain.Add("*.root") - this command adds all .root files in the current directory into the chain. In this case just SuperPointing_test_threealgos.root

root [2] chain.MakeClass("test")

• Now, in your working directory, you have a skeleton code (test.h, test.C) that can be run on the pixel ntuple.
• In the header file "test.h" are all the variable you can look at:
  • Int_t evt_run;
  • Int_t evt_evtnum;
  • ...
  • Float_t Cluster_charge;
  • ...
  • Float_t RecHit_localX;
  • Float_t RecHit_localY;
  • Float_t RecHit_residualX;
  • Float_t RecHit_residualY;
  • ...
  • Float_t track_pt;
  • Float_t track_globalEta;
  • ...

• A detailed description of most variables is here: https://twiki.cern.ch/twiki/bin/view/CMS/CMSSiPixelCosmicCommData
  • Click on Andrew York's page: "CMSSiPixelNtupleDocumentation" to get here: https://twiki.cern.ch/twiki/bin/view/CMS/CMSSiPixelNtupleDocumentation

• Further down in the header file "test.h" is the name of the file that you will analyze:
  • chain->Add("/uscms/home/ggiurgiu/work/CMSSW_2_2_7/src/DPGAnalysis/SiPixelTools/test/SuperPointing_test_threealgos.root/ctfNtuple/PixNtuple");

• In the source file "test.C" you can write your analysis code:
• Add this line at the top of the file after the other "include" statements: #include <iostream>
• Replace: Long64_t nentries = fChain->GetEntriesFast();
• With: Long64_t nentries = fChain->GetEntries(); cout << "nentries = " << nentries << endl;
• Compile and run the Root script:

[ggiurgiu@cmslpc07 test]$ root -l

root [0] .L test.C++

... blah, blah, blah, some Root messages...

root [1] test mmm

root [2] mmm.Loop()

• You just ran over the ntuple, but didn't really do anything yet except for printing out the total number of entries (clusters) in the ntuple.
• A small example on how to plot X and Y residuals for BPIX and FPIX is located here:/uscms/home/ggiurgiu/work/CMSSW_2_2_7/src/DPGAnalysis/SiPixelTools/test/
  • test.h
  • test.C
• Copy and run this example in your work area:

[ggiurgiu@cmslpc07 test]$ root -l

root [0] .L test.C++

... blah, blah, blah, some Root messages...

root [1] test abc

root [2] abc.Loop()

• You should get 4 plots of the X/Y BPIX/FPIX residuals as shown here: http://www.pha.jhu.edu/~ggiurgiu/CMS/PIXEL_TUTORIAL_PLOTS/EPS/tutorial_residuals.eps
  • The same plots in png format are here: http://www.pha.jhu.edu/~ggiurgiu/CMS/PIXEL_TUTORIAL_PLOTS/PNG/
  • Note that these are not intrinsic PixelRecHit residuals. In this case residuals are calculated as distance between the PixelRecHit position and the position of the trajectory intersection with the pixel module. These residuals are dominated by tracking extrapolation effects.
  • You can replace the input file in the header "test.h" with the high statistics file corresponding to the full CRAFT sample and then run again: chain->Add("/uscms_data/d2/ggiurgiu/SuperPointing_CRAFT_ALL_V11_NtuplizerV00-01-03_Data_AllTrkAlgos.root/ctfNtuple/PixNtuple");

• Another example script:
  • /uscms/home/ggiurgiu/work/CMSSW_2_2_7/src/DPGAnalysis/SiPixelTools/test/test_more_advanced.h
  • /uscms/home/ggiurgiu/work/CMSSW_2_2_7/src/DPGAnalysis/SiPixelTools/test/test_more_advanced.C
• This script plots the cluster charge and size as well as the cluster track angle alpha between the track and the

local X axis. It also takes a certain cluster in the ntuple (I just took cluster number 1500) and shows its geometrical shape:

[ggiurgiu@cmslpc07 test]$ root -l

root [0] .L test_more_advanced.C++

... blah, blah, blah, some Root messages...

root [1] test_more_advanced xyz

root [2] xyz.Loop()

• You will see plots like these:
  • http://www.pha.jhu.edu/~ggiurgiu/CMS/PIXEL_TUTORIAL_PLOTS/EPS/tutorial_charge_size_alpha.eps
  • http://www.pha.jhu.edu/~ggiurgiu/CMS/PIXEL_TUTORIAL_PLOTS/EPS/tutorial_charge_size_alpha_2.eps
    • this is the same as above but high statistics
  • http://www.pha.jhu.edu/~ggiurgiu/CMS/PIXEL_TUTORIAL_PLOTS/EPS/tutorial_cluster_shape.eps
    • the charge of each pixel is in electrons and encoded by the color scale on the right hand side
  • http://www.pha.jhu.edu/~ggiurgiu/CMS/PIXEL_TUTORIAL_PLOTS/EPS/tutorial_cluster_shape_zoomin.eps
    • this is the same as above but zoomed in

• You can modify the above script to do your own analysis.

How to modify the Pixel Ntuplizer

• If you need more than the pixel ntuplizer can offer, you can either:
  • Write your own analyzer, see detailed instructions here:
    • https://twiki.cern.ch/twiki/bin/view/CMS/WorkBook
    • https://twiki.cern.ch/twiki/bin/view/CMS/WorkBookWriteFrameworkModule
  • Simply add to the pixel ntuplizer what you need (and remove what you don't...)

• The actual code is in: DPGAnalysis/SiPixelTools/plugins/
• Edit DPGAnalysis/SiPixelTools/plugins/PixelNtuplizer_RealData.cc and .hh files
  • A trivial example: put in some printouts by adding this line at the begining of analyze(...) function
    • cout << "I am adding my stuff here......................................................" << endl;
    • go back to DPGAnalysis/SiPixelTools/
    • compile and build again: scramv1 b
    • go to DPGAnalysis/SiPixelTools/test/
    • now you can run again the ntuplizer which includes your changes:

[ggiurgiu@cmslpc07 test]$ cmsRun pixelNtuplizer_RealData_cfg38T.py

How to Run the Pixel Ntuplizer (or Any Other Module for that Matter) Using CRAB

• Details about CRAB:
  • https://twiki.cern.ch/twiki/bin/view/CMS/WorkBookRunningGrid
  • https://twiki.cern.ch/twiki/bin/view/CMS/SWGuideCrabHowTo

• I only did this using CERN machines, lxplus.cern.ch, (doing it from FERMILAB machines, cmslpc.fnal.gov, should be similar, but did not get to test it yet)
• Log into CERN cluster: ssh lxplus.cern.ch
• source /afs/cern.ch/cms/LCG/LCG-2/UI/cms_ui_env.csh
• cd public/
• scramv1 project CMSSW CMSSW_2_2_7
• cd CMSSW_2_2_7
• eval `scramv1 runtime -csh`
• source /afs/cern.ch/cms/ccs/wm/scripts/Crab/crab.csh
• cd src/
• cvs co -r V00-01-03 DPGAnalysis/SiPixelTools
• scramv1 b
• cd DPGAnalysis/SiPixelTools/test/ntuplizer_scripts/
• cp ../pixelNtuplizer_RealData_cfg38T.py ./
• mv crab_for_CAF.cfg crab.cfg
• Edit crab.cfg as follows:
  • datasetpath=/Cosmics/Commissioning08_CRAFT_ALL_V11_227_Tosca090216_ReReco_FromSuperPointing_v2/RAW-RECO
  • pset=pixelNtuplizer_RealData_cfg38T.py
  • total_number_of_events=1000 # after making sure everything runs fine you can run over all events by setting: total_number_of_events=-1 (-1 means all available events in the input files)
  • number_of_jobs = 5 # when running on all events should split this in more jobs (200 ?)
  • user_remote_dir=/user/g/ggiurgiu/ # set the output location in your castor area
  • return_data = 1 # this setting will return your output ntuples in your home area DPGAnalysis/SiPixelTools/test/ntuplizer_scripts/crab_x_xyz_xyz/res
  • return_data = 0 # this setting will put your output ntuples in castor as specified above;
  • if return_data = 0, then also set copy_data=1
• Save your updates to crab.cfg
• crab -create
  • You should see:
    • crab. Creating 5 jobs, please wait...
    • crab. Total of 5 jobs created.
• crab -submit
  • You should see:
    • crab. Total of 5 jobs submitted.

• Now you can wait and pray Or, every few seconds you can check the status of your jobs:
• crab -status
  • You should see something like this

1 RUN lxb7710

2 RUN lxb8217

3 RUN lxb7708

4 RUN lxb8220

5 DONE lxb7716

• When all jobs are done you will find the output either in your castor area:
  • nsls /castor/cern.ch/user/g/ggiurgiu/
• ... or in your home area:
  • DPGAnalysis/SiPixelTools/test/ntuplizer_scripts/crab_x_xyz_xyz/res
• The output location depends on the setings of "return_data" and "set copy_data" in crab.cfg
• The log files are here: DPGAnalysis/SiPixelTools/test/ntuplizer_scripts/crab_0_090424_181059/res/

• You can look at your output directly from castor:
  • root [0] TFile *f = TFile::Open ( "rfio:/castor/cern.ch/user/g/ggiurgiu/SuperPointing_test_threealgos_4.root" )
• ... or you can copy it from castor to your area:
  • rfcp /castor/cern.ch/user/g/ggiurgiu/my_file.root ./

• If everything looks good in your output files, you can go ahead and change crab.cfg to run on all events in your sample:
  • total_number_of_events=-1
  • number_of_jobs = 100 or 200 or something like that...

-- GavrilGiurgiu - 23 Apr 2009