Standalone reader of CoolDozer database.

Installing dozerReader.py

The application for reading CoolDozer database is a part of CoolDozer package since CoolDozer-00-00-39 tag. It is a part of any release which was build after 15.7.0, so you can use it directly on any lxplus node. For all previous releases you have to check it out to your work area

[lxplus217] ~ % svn co $SVNOFF/Tools/CoolDozer/tags/CoolDozer-00-00-39 CoolDozer
[lxplus217] ~ % cd CoolDozer/cmt
[lxplus217] ~ % make

After that the application (dozerReader.py) should be accessible form command line and ready to use.

CLI options

The full list of options could be seen by running dozerReader.py with --help or -h option.

[lxplus217] ~ % dozerReader.py --help
Usage: dozerReader.py [opts]

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -b                    batch mode for PyROOT [always on]
  -p PACKAGE, --package=PACKAGE
                        package name (REQUIRED)
  -l LOGNAME, --logname=LOGNAME
                        log name [default dozerReader_PACKAGE.log]
  -q, --quiet           don't print log messages to console [default False]
  --verbose             print log messages to stdout [default False]
  --framework=FRAMEWORK
                        test framework [default RTT]
  --branch=BRANCH       branch name [default *]
  --install=INSTALL     install type [default build]
  --cmtconfig=CMTCONFIG
                        cmtconfig value [default *]
  --project=PROJECT     project name [default *]
  --jobid=JOBID         job id name
  --since=SINCE         since date in ISO format (YYYY-MM-DD)
  --until=UNTIL         until date in ISO format (YYYY-MM-DD)
  --lastndays=LASTNDAYS
                        last N days of monitoring [default 30]
  --dump-xml            dump payloads to xml files [default False]
  --dump-csv            dump payloads to csv files [default False]
  --output=OUTPUT       output file name [default dozerReader.html]
  --css=CSS             html page style sheet [default Dozer]
  --plots               trigger trends and histograms generation (same as
                        --plot-trend and --plot-histos together) [default
                        False]
  --plot-trends         trigger trend plots generation [default False]
  --plot-histos         trigger histogram plots generation [default False]
  --fit                 trigger plots fitting (same as --fit-trends and --fit-
                        histos together) [default False]
  --fit-trends          trigger trend plots fitting [default False]
  --fit-histos          trigger histogram plots fitting [default False]

--version

This option only prints to stdout version of dozerReader.py app:

[lxplus217] ~ % dozerReader.py --version
dozerReader.py $Revision: 274938 $ by Krzysztof Daniel Ciba (Krzysztof.Ciba@NOSPAMgmail.com)

-h or --help

List all options and switches to the stdout, was used before.

-b

It is used to force batch processing for PyROOT and it is always present, so you don't have to add this one to the command line.

-p PACKAGE or --package PACKAGE

It sets the package name to PACKAGE. When not specified the application quits:

Usage: dozerReader.py [opts]

dozerReader.py: error: --package option ir required

So this is the only one option which is really required to run dozerReader.py app.

-l LOGNAME or --logname LOGNAME

Sets the name of created by dozerReader.py log file. When not specified, the log name is set to the dozerReader_PACKAGE.log, where PACKAGE is set by -p option.

-q or --quiet

This one option close logging to stdout, which is done by default.

--verbose

This one option is opposite to the -q or --quiet - it adds console handler for logging to the stdout/stderr.

-f FRAMEWORK or --framework FRAMEWORK

Those options are setting the test framework. By default it is set to the RTT (RunTimeTester), as only this one is using CoolDozer right now.

--branch BRANCH

This one option sets the branch name to BRANCH. When BRANCH is misspelled, it shows all available values:

[lxplus217] ~ % dozerReader.py -p CoolDozer  --branch fubar
Usage: dozerReader.py [opts]

dozerReader.py: error: Invalid --branch option value 'fubar'

It should be one of:
 - *
 - NN.N.NX
 - NN.N.NX.Y
 - NN.N.NX.Y-VAL
 - NN.N.OLD.Y
 - NN.N.X.Y
 - NN.N.X.Y-VAL
 - bugfix
 - dev
 - devval
 - mig0
 - mig1
 - mig2
 - mig3
 - mig4
 - mig5
 - mig6
 - numbered
 - pcache
 - point1
 - val

Default value is *, which means records would be read from all available branches in database.

--install INSTALL

This one option sets the installation type:

[lxplus217] ~ % dozerReader.py -p CoolDozer --install fubar
Usage: dozerReader.py [opts]

dozerReader.py: error: Invalid --install option value 'fubar'

It should be one of:
 - *
 - build
 - kit

By default it is set to build, as RTT is running at the moment all tests only on afs build nightlies (but I hope running it on the kit is still available...).

--cmtconfig CMTCONFIG

This one option sets CMTCONFIG value:

[lxplus217] ~ % dozerReader.py -p CoolDozer --cmtconfig fubar
Usage: dozerReader.py [opts]

dozerReader.py: error: Invalid --cmtconfig option value 'fubar'

It should be one of:
 - *
 - i686-slc3-gcc323-dbg
 - i686-slc3-gcc323-opt
 - i686-slc4-gcc34-dbg
 - i686-slc4-gcc34-opt
 - i686-slc4-gcc43-dbg
 - i686-slc4-gcc43-opt
 - i686-slc5-gcc34-dbg
 - i686-slc5-gcc34-opt
 - i686-slc5-gcc43-dbg
 - i686-slc5-gcc43-opt
 - x86_64-slc4-gcc34-dbg
 - x86_64-slc4-gcc34-opt
 - x86_64-slc5-gcc34-dbg
 - x86_64-slc5-gcc34-opt
 - x86_64-slc5-gcc43-dbg
 - x86_64-slc5-gcc43-opt

--project PROJECT

When added it sets the project name, by default it is set to * (all available in db).

[lxplus217] ~ % dozerReader.py -p CoolDozer --project fubar  
Usage: dozerReader.py [opts]

dozerReader.py: error: Invalid --project option value 'fubar'

It should be one of:
 - *
 - AtlasAnalysis
 - AtlasConditions
 - AtlasCore
 - AtlasEvent
 - AtlasHLT
 - AtlasOffline
 - AtlasP1HLT
 - AtlasProduction
 - AtlasReconstruction
 - AtlasSimulation
 - AtlasTier0
 - AtlasTrigger

Remember, that RTT is running against full releases (so project is set to AtlasProduction) or for several sub-projects (e.g. AtlasTier0 or AtlasHLT), so specifying --project AtlasCore doesn't make any sense...

--jobid JOBID

When specified it sets reading only records from JOBID RTT jobs. The value of JOBID is exactly the same as trendId in your *_TestConfiguration.xml file.

--since SINCE, --until UNTIL and --lastndays LASTNDAYS

This group of options is used to set interval (IOV) of time for records. You can specify IOV: * by setting start and end date in ISO format (YYYY-MM-DD) using --since SINCE and --until UNTIL options * by setting --lastndays LASTNDAYS, where LASTNDAYS is a integer number (default = 30). In that way UNTIL is equal to today and SINCE is equal to today-30. You can't used the both:

[lxplus217] ~ % dozerReader.py -p CoolDozer  --lastndays 15 --since 2010-01-01 --until 2010-02-01
Usage: dozerReader.py [opts]

dozerReader.py: error: cannot use --since, --until and --lastndays options at the same time!

And when you are using first option, you have to specify both of switches:

[lxplus217] ~ % dozerReader.py -p CoolDozer --since 2010-01-10                                   
Usage: dozerReader.py [opts]

dozerReader.py: error: option --since present but --until is missing

Of course additional checking for dates is switched on:

[lxplus217] ~ % dozerReader.py -p CoolDozer --since 109-1-32
Usage: dozerReader.py [opts]

dozerReader.py: error: --since option value (=109-1-32) error, day is out of range for month

BTW the oldest record in CoolDozer db for RTT framework was added close to the 1st of Jan, 2009.

--output OUTPUT

This one overrides standard html file name (dozerReader.html). You can specify this one as absolute of relative path to the file and the full output (plots, xml or csv dumps) would be created over there.

--dump-xml and --dump-csv

Those two switches trigger dumping record sets to the XML files and CSV (comma separated text files). Those files are stored in OUTPUTDIR/xml and OUTPUTDIR/csv and their names corresponds to the full path inside CoolDozer database. e.g. PACKAGE__BRANCH__INSTALL_CMTCONFIG_PROJECT_JOBID_CHANNEL.[xml|csv]. They could be used for some further processing in some other applications.

--css CSS

It sets style for plots and css for output html:

[lxplus217] ~ % dozerReader.py -p CoolDozer --css fubar       
Usage: dozerReader.py [opts]

dozerReader.py: error: option --css: invalid choice: 'fubar' (choose from 'Dozer', 'Dolphin', 'Orca', 'Seagull', 'Seal')

When not specified standard CoolDozer style is used (Dozer). The preview of all available styles is here or here.

--plots, --plot-trend and --plot-histos

Those switches trigger generation of plots:

• --plot-trends for plotting time trends of "plottable" data for record set
• --plot-histos for plotting histograms of "plottable" data for record set
• --plots for plotting both trends and histograms

-fit, --fit-trends and --fit-histos

Those switches trigger fitting of trends (linear fit) and histograms (gaussian). Adding --fit option have ecatly the same meaning like putting --fit-trends and --fit-histos together.

Example run.

[lxplus217] ~/scratch2 % dozerReader.py --package CoolDozer --branch dev --dump-xml --dump-csv --lastndays 10 --plots --fit --output dozerReader/dozerReader.html --jobid rttSimpleCoolDozer_RTTRandomDozerApp --css Seagull

and its result is here.

-- KrzysztofCiba - 25-Jan-2010