Contents:

Meaning of this page

Listing and collecting items related to the Dashboard support/development/issues followed by me or interesting to me

Dashboard Monalisa Collectors for CMS jobs

CMS collectors troubleshooting.

CMS collector understanding reported by me here.

Other Dashbaord collectors example.

Oracle common schema for dashboard: source code here.

Oracle triggers for Ganga-Atlas: source code here.

Dashboard & WMAgent

Dashboard support tickets by WMAgent are:

• trac #89 - first ticket
• trac #1206 - performance info reporting
• trac #1232 - target CE - SE
• trac #1234 - GridJobId
• trac #1235 - WMAgent problem with hard-coded scheduler name
• trac #1236 - WMAgent problem with status order
• trac #1237 - WMAgent problem with status enter time accuracy
• trac #1264 - Use Actual JobExit Codes In Steps

Status reports:

• 11Th March at Weekly Computing Meeting: reported parameters check list
• 18Th March: few parameters being reported, not yet many of them. Steve problems was a problem with Steve link. All workflow are being registered.

I have refactored Monalisa client for CMS-DMWM to be used in WMAgent by the DashboardReporter component. Changes requested to satisfy DMWM requirements, seen the expressed impossibility to use the production one (ticket #89, private e-mails exchange and meetings).

Information that should be reported by WMAgent are being written here.

DMWM requirements for dashboard monitoring are listed here.

Attributes currently reported to Dashboard by CMS applications (PA, CRAB-2) are listed here.

Modification on Dashboard side

• Trigger + grid_status table to adapt the status + exit code of successful jobs
• CMS collectors on integration host: added CMSUser, WMAgent job monitor id.
• SQL schema changes ready to register performance information; needed also to modify the collectors to register the information on the propers tables.

Dashboard for data popularity support

Data popularity system being developed in IT-ES-VOS is relying on information being stored into dashboard database. Information is sent by CRAB jobs directly from the WN. Monalisa collectors process the information and store it in the database. First round of information is related to job-data information, this is the current list (21Th of March):

• list of files accessed by each job (this includes parent files);
• list of blocks accessed by each job;
• list of lumisection ranges by job.

More information will be stored as soon as CRAB will send it:

• timestamp of file access;
• success/failure of file access;
• used protocol for data access;
• ...

Modification on Dashboard side

Modification done on Dashboard side:

• New raw table to store information from collectors into integration Oracle db
• New tables to map job-data information into integration Oracle db
• Trigger and procedures to move data from raw table to proper job-data tables;
• Collectors have a new step to store the information in the raw table.

WLCG daily op. meeting

SAM reporting twiki is here.

WLCG daily operation meeting minute collector.

Using MSG

So to send or consume messages in STOMP format you can use: http://code.google.com/p/stomppy/

Ganga uses this with a wrapper called stomputil (which adds some connection management). This was originally written by a summer student but I added to it so I know it fairly well: https://svnweb.cern.ch/trac/ganga/browser/trunk/external/stomputil

A simple usage example in Ganga is here: https://svnweb.cern.ch/trac/ganga/browser/trunk/ganga/python/Ganga/Runtime/spyware.py

A more complex usage example in Ganga is here: https://svnweb.cern.ch/trac/ganga/browser/trunk/ganga/python/Ganga/Lib/MonitoringServices/Dashboard/DashboardMS.py

Config example: https://svnweb.cern.ch/trac/ganga/browser/trunk/ganga/templates/CONFIG_TEMPLATE.INI

Other staff

Useful commands

Remote desktop usage:

<verbatim>rdesktop  -a 16 -u USERNAME -d CERN -g 1650x1000 cernts.cern.ch</verbatim>

Oracle environment from lxplus:

source /afs/cern.ch/project/oracle/script/setoraenv.sh

Error reporting tool

Link of the error reporting tool for CMS.

-- MattiaCinquilli - 11-Feb-2011