CRAB3-glideinWMS testing

This page describes the very, very alpha testing for CRAB3-glideinWMS

Engineering setup

The known working host environment is:

• RHEL5.
• Download the latest tarball. Unpack into a separate pre-made CRAB3 directory, then source setup.sh.
  • Example:

    mkdir ~/CRAB3; pushd ~/CRAB3
    curl http://hcc-briantest.unl.edu/CRAB3-gWMS.tar.gz | tar zx
    source setup.sh
    popd
    

• A grid proxy that works with cmsweb. cmsweb has several bugs, so I found this precise combination works:

  voms-proxy-init --voms cms:/cms --old --valid 168:00

  . DigiCert certificates do NOT work.

KNOWN BUGS:

• The gsissh environment and the shipped HTCondor libraries conflict. If using GSISSH, do the following:

   mv $CRAB3_BASEPATH/lib/condor $CRAB3_BASEPATH/lib/condor-orig

User Environment

First, setup a CMS user area:

. /cvmfs/cms.cern.ch/cmsset_default.sh
mkdir ~/tmp
cd ~/tmp
cmsrel CMSSW_5_3_7
cd CMSSW_5_3_7
cmsenv

Next, download the attached crabConfig.py. This is equivalent to the old crab.cfg.

Edit this config as desired; make sure to point it at the appropriate pset.py to run your job.

Now, source the setup.sh found inside the CRAB3-gWMS.tar.gz. crab should be in your path.

Things to Try

crab submit

crab status -t <task name>

crab getoutput -t <task name>

crab kill -t <task name>

For direct submission

This version of CRAB3 can submit directly to CRAB3 without GSISSH. However, this requires a very specific HTCondor setup. Be able to submit to a remote HTCondor schedd using GSI auth. Talk to your local HTCondor experts.

• Python2.6 must be installed on the schedd. If you have a RHEL5 box, this can be done by "yum install python26".
• If it's a RHEL5 box, you must also install the classads module for Python 2.6. Do: curl http://hcc-briantest.unl.edu/classad.so > /usr/lib64/python2.6/site-packages/classad.so
• The schedd needs to use full-strength delegations; FTS cannot use the default limited proxy delegation. We also suggest to increase the DAGMan submission rates for large jobs. Add the following config variables to the schedd configuration:

  DAGMAN_MAX_SUBMITS_PER_INTERVAL = 50
  DELEGATE_FULL_JOB_GSI_CREDENTIALS = True
  

Add lines to crabConfig.py to specify the HTCondor schedd location. For example:

config.General.condorPool = "glidein.unl.edu"
config.General.condorScheddList = ["glidein.unl.edu"]