AFP Data Quality Monitoring

Introduction

This page is used for organisation and as a logbook for adding AFP offline and online data quality monitoring to the central ATLAS offline data quality monitoring.

Goals

1. Prepare a list of distributions to be used in the offline monitoring

1. Add relevant ALFA distributions to offline data quality web display

1. Add automatic distribution checks were possible

1. Define defects

1. Prepare documentation and instructions for a person doing data quality monitoring

Tasks

Prepare a list of control distributions %TODO%

Prepare software environment for the new package development %TODO%

1. Set up environment for monitoring package developement

The monitoring package is responsible for filling the histograms. Only Athena is needed. No special other environment is needed. It is best to set up the same Athena release as is currently used at Tier0. The monitoring package is supposed to be a standard Athena package with classes deriving from ManagedMonitorToolBase. However it is useful to checkout the base package

Control/AthenaMonitoring

, because there is an example called

ManagedMonitorToolTest

.

It can be useful to have the code inspired by the classes based on the IHLTMonTool, that is why it is also useful to checkout TrigHLTMonitoring package. In order to get an example of the use of

TrigHLTMonitoring

a package

Trigger/TrigMonitoring/TrigMinBiasMonitoring

. It can be dangerous to develop classes based on the IHLTMonTool, because this is trigger class and we do not have any control other it, nor trigger should consider us when developing this class.

1.1. CMT instructions

The above can be set using the commands

setupATLAS
asetup 20.7.5.4,here # or other currently used at Tier0 release

1.2. CMake instructions

Create a working directory with 3 subdirectries: source, build, run. Set Athena environment in source directory.

mkdir WorkArea
cd WorkArea/
mkdir source build run
cd source/
setupATLAS
asetup AtlasProduction,rel_1,here # or other currently used at Tier0 release
svnco AthenaMonitoring #note this is not a type, an ATLAS command svnco is implemented
svnco TrigMinBiasMonitoring
svnco TrigHLTMonitoring
cd ..

Once all source packages are checked out go to build directory and generate make based on source directory file using CMake. Next compile.

cd build
cmake ../source
make # add -j4 to compile on 4 cores
cd ..

In order to setup runtime environemnt the setup.sh script from build directory should be executed.

. build/x86_64-slc6-gcc49-opt/setup.sh

Athena should be executed in run directory.

More information can be found in the page AthenaMonFramework. An example of the muon code can be found in the muon tutorial.

The following instructions are based on the twiki pages:

• DQOperationalRecipes#Modify_your_configuration_locall

• AthenaMonFramework

#testSetup

1.3. Testing the set-up

1. Create a directory for the test input data and download it

      mkdir TestData
      cd TestData 
     localSetupRucioClients
      voms-proxy-init -voms atlas
      rucio download data16_13TeV.00299055.physics_ZeroBias.merge.AOD.x423_m1588._lb0720-lb0722._0001.1
      cd ..
      

1. Set proper input file by setting correct path in the variable:

af.FilesInput

in the file

build/Control/AthenaMonitoring/share/ManagedMonitorToolTest.py

.

1. Initialise analysis environment (if not done before)

. build/x86_64-slc6-gcc49-opt/setup.sh

1. run test algorithm in the run directory example to test if everything works

      athena ../source/Control/AthenaMonitoring/share/ManagedMonitorToolTest.py
      cd ../
      

2. Set up DQWebDisplay and han.exe

It is enough to set up Athena environment to have access to

DQWebDisplay.py

and

han.exe

. It may be very useful to checkout

DataQuality/DataQualityInterfaces

, because there is an example of configuration file and data file.

      cmt co DataQuality/DataQualityInterfaces
      

2.1. Testing the set-up with DataQualityInterfaces example

1. Enter the directory with example files

   cd DataQuality/DataQualityInterfaces/share
   

1. Generate binary configuration file based on text configuration file

   han-config-gen.exe histogramAnalysis.config
   

2. Process the example data file using the generated configuration. This will produce result file Data_han.root. The output file can be partially browsed with root.

   han.exe histogramAnalysis.hcfg Data.root
   

3. Print summary of the results.

   han-results-print.exe Data_han.root
   

Instructions based on: HLTTrigMonHanConfigRecipesRun2

2.2. Testing display of histograms with web display

Web display is using the same configuration file as han.exe. In order to test the display an SVN access writes are needed. Detailed instructions are available in the paragraph "How to update DQ configurations or reference histograms" in page DQOperationalRecipes#How_to_update_DQ_configurations

2.3. SVN repository for package

The new package shall be placed in the central SVN repository. The ALFA monitoring will be store at atlasoff/ForwardDetectors/ALFA/ALFA_Monitring

Implement and test the package (IN PROGRESS)

1. Create directory in SVN repository (TODO)

Directory

atlasoff/ForwardDetectors/AFP/AFP_Monitoring

should be created with proper subdirectory structure (i.e. trunk, tags, etc). This allows checking out package using cmt command

cmt co ForwardDetectors/AFP/AFP_Monitoring

Currently a directory ForwardDetectors/AFP/ was created manually.

2. Create a new athena package

When this twiki is being written description how to automatically generate new Athena package using cmake (not cmt) was not found, that is why the directory structure was created manaully.

mkdir -p ForwardDetectors/AFP/AFP_Monitoring
mkdir -p AFP_Monitoring # create a directory for header files 
mkdir -p src/components # create a directory for source files
cp ../../../Control/AthenaMonitoring/CMakeLists.txt . #copy cmake file to be modified

The copied CMakeLists.txt file was modified to be approperiate for the AFP_Monitoring package. The name of the package had to be changed in several places.

It is important to remember that every new tool or algorithm needs additional entries in src/components/AFP\_Monitoring\_entries.cxx and src/components/AFP\_Monitoring\_load.cxx.

3. Create a new tool in the package

It is possible to copy the example tool (ManagedMonitorToolTest and delete the unnecessary code (i.e. most of the code) or simply create a new class that derives from ManagedMonitorToolBase.

4. Install ALFA xAOD

ALFA xAOD are not available in the production release, yet, so they have to be installed separately. They can be checked out and compiled with the following commands

cmt co Event/xAOD/xAODForward/
cd Event/xAOD/xAODForward/cmt
cmt make
cd ../../../../

5. Add call to the new class in an existing program

In order to test the code it has to be executed. It can be done by adding the call to the new algorithm to the example provided in

Control/AthenaMonitoring/run

(see Testing the set-up). In order to do it the file

Control/AthenaMonitoring/share/ManagedMonitorToolTest.py

should be modified by adding the following lines at the end of the file:

from ALFA_Monitoring.ALFA_MonitoringConf import ALFAMonTool
ALFATest = ALFAMonTool (
name           = "ALFATool",
PreScale       = 0,
EnableLumi = False,
ProcessNEvents = 40
## The next line is for cases when the AlgTool cannot be attached directly to AthenaMonManager
# ManagerName   = "AthMonTestMan"
)
ALFATest.OutputLevel = INFO

ToolSvc += ALFATest
monMan.AthenaMonTools += [ ALFATest ]

6. Implement distributions (TODO)

Prepare reference distributions (TODO)

Prepare defects (TODO)

Prepare documentation of the package and instructions for the person doing monitoring (TODO)

Prepare configuration that can be submitted to the central system (TODO)

Submit the new package to the central system (TODO)

Notes

Links

• Instructions at data quality page

• Documentation of Athena monitoring framework

• DQHistogramAnalyzer

• Muon tutorial and instructions

• HLT tutorial - Muon one is probably better

• Tier0 Update

• How to work with CMake

• Using CMake in ATLAS packages

Presentations connected with this project

• Analysis of run 267358

• Analysis of run 267358 (next presentation)

Trigger aware monitoring

Every monitoring tool is prepared to select events based on trigger chain. The same histograms can be filled with different trigger chains, this requires C++ tools to have a possibility to save results to customized output directory in order not to overwrite the histograms. Presentation about this feature of the monitoring tool was given in data quality meeting: https://indico.cern.ch/event/397895/contribution/4/material/slides/0.pdf

Meetings

WebNotify

Properties

-- GrzegorzPawelGach - 2015-06-18 -- GrzegorzPawelGach - 2016-04-13