My Swiss Army Knife of Useful Stufff

Important links

• CMSWeb
• Twiki user guide

Tips

• Getting CERN users info

    $ phonebook -lo <username>
    $ phonebook <full_name>

• To list all files greater than ~10G:

    $ find . -type f -size +10G -exec ls -lh {} \; | awk '{ print $9 ": " $5 }'

• To print only directories

    $ ls -d */

• To convert the EPOCH time in human readable time

    $ date -d @<epoch_time>

• Show all output lines of command except those containing the string str.

    $ command | grep -v <str>

• Command line to comment out all lines in a file (using vim):

    :1,$s/^/#/g

• grep: looking for a pattern (in this case, X86_64) -R: recursively -H: print the filename for each match -n: Prefix each line of output with the line number within its input file. -s: Suppress error messages about nonexistent or unreadable files. -I Process a binary file as if it did not contain matching data; -i: Ignore case distinctions in both the PATTERN and the input files.

    $ grep -R -H -I -n -s X86_64 *

• Commenting/uncommenting files with sed (link):

    sed -i '/<pattern>/s/^/#/g' file (to comment out)
    sed -i '/<pattern>/s/^#//g' file (to uncomment)

LXPLUS

• Setup SSH access via key: https://www.hep.manchester.ac.uk/u/miyagawa/work/athena/ssh.html
• SSH tunneling in lxplus: https://security.web.cern.ch/recommendations/en/ssh_tunneling.shtml
• AFS console for user loads: https://afs-console.cern.ch/cgi-scripts/index.cgi
• Recover user/work area from backup: https://cern.service-now.com/service-portal?id=kb_article&n=KB0000430
• LCG Views: https://indico.cern.ch/event/479888/contributions/1999187/attachments/1217382/1778496/LCG-Views-20160126.pdf
• Check storage node attached to account:

      $ fs whereis /afs/cern.ch/user/s/silveira/ /afs/cern.ch/work/s/silveira/

GRID

• Grid Administration: commands related to proxy:

    $ voms-proxy-init -debug -voms cms       # the cert/key pair of files should be placed under ~/.globus
    $ voms-proxy-init --valid 192:00 -voms cms:/cms/Role=production
    $ globusrun -a -r osgce.hepgrid.uerj.br   # testing authentication
    $ voms-proxy-info -all -file <your proxy>

CERN Batch HTCondor

• DAG manual
• Condor status code
• Condor commands:

    $ condor_hold -reason 'just messing around' 6502.11
    $ condor_rm -const 'CMS_JobType =?= "Merge"'
    $ condor_submit <submit.file>
    $ watch -n 1 'condor_q'
    $ condor_qedit 763.0 RequestMemory 2200
    $ condor_qedit 261.0 DESIRED_Sites "\"T1_US_FNAL\""
    $ condor_qedit -const 'JobStatus=?=1 && (CMS_JobType =?= "Production" || CMS_JobType =?= "Processing")' JobPrio 999999
    $ condor_qedit -const 'JobStatus=?=1 && isUndefined(REQUIRED_OS)' REQUIRED_OS "\"any\""
    $ condor_q -af:h MaxWallTimeMins
    $ condor_q -af:h WMAgent_SubTaskName RequestCpus RequestMemory RequestDisk MachineCount AccountingGroup MaxWallTimeMins
    $ condor_q -global | grep running
    $ condor_q -hold
    $ condor_status -submitters
    $ condor_status -run
    $ condor_rm <job_id>
    $ condor_q -bet <job_id>
    $ condor_q -global -better-analyze <job_id>
    $ condor_prio -p 20 <job_id>  ### Giving the maximum priority in the local queue
    $ condor_userprio -all -allusers

• How to run bare and custom CMSSW code: https://uscms.org/uscms_at_work/computing/setup/batch_systems.shtml#tarCMSSW

CMSSW dev

• Release schedule
• Choosing_CMSSW - Which CMSSW release to use
• cmsDriver guide
• How to locate data samples to analyze
• PR Workflow
• tutorial-collaborating-with-peers

Git

• Commit Good Practices
• Pulling branch from other repo
• Setup GPG key to verify commits
• Changing files from a commit existent in a PR:

    $ git reset --soft HEAD~1
    $ git reset FILENAME    ### to unstage a file
    $ git checkout -- FILENAME ### to rollback that file to its original state before that commit

• Getting stats from the diff between two branches.

    $ git diff --stat --color comp..comp_gcc481

• Retrieving the diff of a file betweem two branches:

    $ git diff --color comp..comp_gcc481 reqmon.spec

• Test PR locally

     $ cmsrel (cmssw-release-IB-or-not)
     $ cmsenv
     $ git cms-addpkg (corresponding-package)
     $ git fetch origin pull/<ID>/head:[BRANCHNAME] (BRANCHNAME local)
     $ git checkout [BRANCHNAME] (PR changes will be local)
   * If PR in another remote:
     $ git fetch <another_remote> pull/[pull-request-id]/head:[BRANCHNAME]
     $ git checkout [BRANCHNAME]
   * for CMSSW:
     $ cmsrel CMSSW_13_3_X_2023-10-27-1100    # the latest available IB
     $ cd CMSSW_13_3_X_2023-10-27-1100/src
     $ git cms-init
     $ git cms-merge-topic <PR_NUMBER>
     $ git cms-checkdeps -a
     $ scram b -j 8

Analysis tools

• Intro to uproot: https://hsf-training.github.io/hsf-training-uproot-webpage/aio/index.html
• Enabling dasgoclient: https://twiki.cern.ch/twiki/bin/view/Main/EnablingDasgoclient

MccM

• GrASP
• PdmVMonteCarloCoordinationMeeting
• SWGuideSubgroupMC
• MccM tickets
• MccM Requests

-- GustavoGilDaSilveira - 2022-05-15