RecExCommissionInDev < Main

Main Web>TWikiUsers>JamieBoyd>JamiesWiki>RecExCommissionInDev (2008-07-10, JamieBoyd) (raw view)
---+ !RecExCommission in 14.2.10 nightlies (rel_4, bugfix):

---+++ Current 'best' recipe
<verbatim>
source setup.sh -tag=14.2.10.Y,AtlasTier0,rel_4,32
source $AtlasArea/AtlasTier0RunTime/cmt/setup.sh
source RecExCommissionData_links.sh  

get_files -jo RecExCommission_ATN.py

edit the number of events and data you want to run over.

athena -c RTT=True RecExCommission_ATN.py >& log &
</verbatim>

---+++ Status on Monday
   * 3 muon problems remaining
<verbatim>
i) [bug #38338] Infinite loop in MuonRDO_Decoder?    
ii) data not the same in 14.2.10.Y vs 14.1.0.Y (presumably wrong in 14.2.10)
iii) cant have Raw data in ntuples
</verbatim>
   * More minor issues: verbosity like:
<verbatim>
ToolSvc.InDet::InDetDetailedTrackSelectorTool               WARNING Track Selector failed to extrapolate track to the vertex: (0,0,0)
ToolSvc.InDet::InDetDetailedTrackSelectorTool               WARNING Track preselection: Zero track pointer received. This track will not pass.
</verbatim>
   * Christian submitted this as bug  38558
<verbatim>
ToolSvc.InDetFKF                                               INFO lost PseudoMeasurement during extrapolation - PM-surface or sorting problem.
ToolSvc.InDetFKF                                               INFO lost PseudoMeasurement during extrapolation - PM-surface or sorting problem.
</verbatim>
   * Christian will ask Andi W
<verbatim>
ToolSvc.MuonHoughPatternTool                                   INFO Curvature too small, should not be possible: 1 1
ToolSvc.MuonHoughPatternTool                                   INFO Curvature too small, should not be possible: 1 1
</verbatim>

---+++ Status on Sunday
   * Can run RTT data and sim tests
   * see some DEBUG statements from ToolSvc.CSCRDOMonTool, ToolSvc.CSCPRDMonTool, ToolSvc.SCTHitsNoiseMonTool - needs fixing
   * No unchecked StatusCodes (good)
   * Still muon problems
   * Data doesnt seem to be correct
   * Infinite hang when running on more recent data (bug 38338)
   * to do:
<verbatim>
- fix both muon problems
- add in new tag writer (more useful quantities in the tag)
- add in filtered Tile digits into ESD
- add in option to use LAr data in physics mode (doDSP)
- update RTT test to use M7 data with all systems and 100% LAr coverage
- get HLT part fixed
</verbatim>
   * Current mem use from monitoring (RTT test on M6 data) looks like
<verbatim>
Vmem total (in MB) reported by perfmon (ini+1st event+event+fin)
            LArMonManager:    84
       InDetGlobalManager:     6
        IDAlignMonManager:     5
            SCTMonManager:    44
           CaloMonManager:    43
   ManagedAthenaGlobalMon:     4
           PrimaryManager:    22
          PixelMonManager:     2
            TRTMonManager:    18
     ManagedAthenaTileMon:    15
           MuonMonManager:   124
         L1CaloMonManager:    10
              MuonManager:     1
             L1MonManager:     1
    ManagedAthenaMonPilot:     0
                      sum:   380
</verbatim>
   * Muons and LAr need to reduce there mem use!!!


---+++ Status on Fri
   * Despite beging able to run the RTT test (as above) OK. if i try to run on more recent data i run into an infinite hang from
<verbatim>
(gdb) where
#0  0xf6314be6 in std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string () from /usr/lib/libstdc++.so.6
#1  0xe6826501 in CSMid::CSMid () from /afs/cern.ch/atlas/software/builds/AtlasEvent/14.2.10/InstallArea/i686-slc4-gcc34-opt/lib/libMDTcablingLib.so
#2  0xe683fba3 in MDTGeometry::getOfflineIDfromOnlineID ()
   from /afs/cern.ch/atlas/software/builds/AtlasEvent/14.2.10/InstallArea/i686-slc4-gcc34-opt/lib/libMDTcablingLib.so
#3  0xe682ca84 in MDTcablingSvc::getOfflineIDfromOnlineID ()
   from /afs/cern.ch/atlas/software/builds/AtlasEvent/14.2.10/InstallArea/i686-slc4-gcc34-opt/lib/libMDTcablingLib.so
#4  0xe68f84b4 in MdtROD_Decoder::fillCollections ()
</verbatim>
   * This looks exactly the same as the bug report: https://savannah.cern.ch/bugs/?func=detailitem&item_id=38338#options but i dont see howto fix it
   * It is odd to me that it appears for M7 but not M6 data (both have MDT hits in them)
   * to reproduce the problem follow the recipe below:
<verbatim>
source setup.sh -tag=14.2.10,oneTest,32
source $AtlasArea/AtlasOfflineRunTime/cmt/setup.sh
source RecExCommissionData_links.sh
cmt co -r RecExCommission-00-03-04 Reconstruction/RecExample/RecExCommission
cd Reconstruction/RecExample/RecExCommission/cmt
cmt bro cmt config; cmt bro gmake -j4
cd -
get_files -jo RecExCommission_Tier0.py
edit this file to have:
ATLASCosmicFlags.doHLT = False
ATLASCosmicFlags.doMuonHitMon = False
athenaCommonFlags.BSRDOInput=['/castor/cern.ch/grid/atlas/DAQ/M7/daq.p5_combined.0070237.physics.HLT_Cosmic_EM_L1Calo.LB0000.SFO-1._0002.data']
(this data is on atlcal castor diskpool so you must have STAGE_SVCCLASS=atlcal which should be done automatically by  RecExCommissionData_links.sh)

run like:
athena  RecExCommission_Tier0.py

it will hang at:
HistorySvc                                                     INFO Registered 169 Algorithms
HistorySvc                                                     INFO Registered 380 AlgTools
HistorySvc                                                     INFO Registered 68 Services

doing a:
gdb attach <PID>
will show you its in MDTGeometry::getOfflineIDfromOnlineID ()
</verbatim>
   * This needs to be fixed ASAP!

---+++ Status on Thurs 3nd July (rel_4)
   * Muon monitoring STILL DOES NOT COMPILE (http://atlas-computing.web.cern.ch/atlas-computing/links/distDirectory/nightlies/VALWebArea/nicos_web_areaVAL142X32BS4AnlOpt/nicos_buildsummary_4.html) this is really begining to annoy me. i have been complaining about this for 1 week. we are trying to build this release today. we cannot have things not compiling! - this is claimed to be fixed with: MuonRawDataMonitoring-00-00-54, MuonValUtils-00-00-19 but i havent tested this yet.
   *  I have made a pretty ugly hack to make the Muon BS stuff work by introducing a file:  RecExCommission_BSRead_config_hack.py which sets up the BS for us. this should be made common with RecExCommon for the REC merger...
   * I could then run the RTT test and see Moore tracks as well as ID tracks (to check ID hadnt been screwed up from this change)
<verbatim>
root [2] CollectionTree->Scan("MooreTracks:ConvertedMBoyMuonSpectroOnlyTracks:ConvertedMBoyTracks:TRT_Cosmic_Tracks:SC$
************************************************************************
*    Row   * MooreTrac * Converted * Converted * TRT_Cosmi * SCT_Cosmi *
************************************************************************
*        0 *         0 *         0 *         0 *         0 *         0 *
*        1 *         1 *         0 *         0 *         1 *         0 *
*        2 *         0 *         0 *         0 *         1 *         0 *
*        3 *         2 *         0 *         0 *         0 *         0 *
*        4 *         0 *         0 *         0 *         1 *         1 *
*        5 *         0 *         0 *         0 *         1 *         0 *
*        6 *         1 *         0 *         0 *         2 *         0 *
*        7 *         0 *         0 *         0 *         0 *         2 *
*        8 *         0 *         0 *         0 *         0 *         0 *
*        9 *         1 *         0 *         0 *         1 *         1 *
************************************************************************
</verbatim>
   * Comparing this to the 14.1.0.Y rel_4 RTT test tag file (running on the same events):
<verbatim>
root -l /afs/cern.ch/atlas/project/RTT/Results/rel_4/14.1.0.Y/build/i686-slc4-gcc34-opt/point1/RecExCommission/AthenaTestingRecExCommission/RecExCommission_ATN/1/TAG.root
root [1] CollectionTree->Scan("MooreTracks:ConvertedMBoyMuonSpectroOnlyTracks:ConvertedMBoyTracks:TRT_Cosmic_Tracks:SC$
************************************************************************
*    Row   * MooreTrac * Converted * Converted * TRT_Cosmi * SCT_Cosmi *
************************************************************************
*        0 *         1 *         1 *         1 *         1 *         0 *
*        1 *         1 *         0 *         0 *         1 *         0 *
*        2 *         1 *         0 *         0 *         1 *         0 *
*        3 *         2 *         1 *         1 *         1 *         0 *
*        4 *         1 *         1 *         1 *         1 *         1 *
*        5 *         0 *         0 *         0 *         1 *         0 *
*        6 *         1 *         0 *         0 *         2 *         0 *
*        7 *         0 *         0 *         0 *         1 *         2 *
*        8 *         1 *         0 *         0 *         1 *         0 *
*        9 *         1 *         0 *         0 *         1 *         1 *
</verbatim>
   * So this shows significant differences! - WHY???
   * When running like this i see on every event:
<verbatim>
AthenaEventLoopMgr                                             INFO   ===>>>  start processing event #13141, run #43719 1 events processed so far  <<<=== ----------------------------------------------------------------------------------------
 | WARNING the standard way of running Muonboy on cosmics is with UseRoaFromMdt = false |
 ----------------------------------------------------------------------------------------
StoreGateSvc                                                WARNING retrieve(non-const): No valid proxy for object CaloMuonIdContainer  of type Analysis::MuonContainer(CLID 1073853171)
 Try to use a const DataHandle
ToolSvc.MuTrkMon                                              ERROR Can't find MuonContainer object with name CaloMuonIdContainer in TDS
StoreGateSvc                                                WARNING retrieve(const): No valid proxy for object CSC_Measurements  of type Muon::CscStripPrepDataContainer(CLID 1240951609)
</verbatim>
   * which dont look so bad (compared to what i saw before...
</verbatim>

---+++ Status on Wed 2nd July (rel_3)
   * LAr monitoring problem is fixed (thanks)
   * ESD writing is fixed by adding the line: ATLASCosmicFlags.doCaloTrkMuId=False proper fix is on the way too (this is savanagh bug report 38385
   * Still need ATLASCosmicFlags.doMuonHitMon = False becausemuon monitoring STILL doesnt compile in val or bugfix. [[https://twiki.cern.ch/twiki/bin/view/Main/NectarB][Nektarios]] is trying to fix it. 
   * by adding to 
<verbatim>
    MuDetCosmicFlags.doMdtRawNt = False
    MuDetCosmicFlags.doRpcRawNt = False
    MuDetCosmicFlags.doCscRawNt = False
    MuDetCosmicFlags.doTgcRawNt = False
</verbatim>
   * To RecExCommissionSetFlags_jobOptions.py able to run with muons on. But still see many warnings like:
<verbatim>
ToolSvc.MdtPrepDataProviderTool                               ERROR In ProcessCSM - Couldn't record in the Container MDT Drift Circle Collection with hashID = 259 ext. id = [7.2.2.4.0.1.1.1]
ToolSvc.MdtPrepDataProviderTool                               ERROR In ProcessCSM - Couldn't record in the Container MDT Drift Circle Collection with hashID = 332 ext. id = [7.3.1.1.0.1.1.1]
ToolSvc.MdtPrepDataProviderTool                               ERROR In ProcessCSM - Couldn't record in the Container MDT Drift Circle Collection with hashID = 335 ext. id = [7.3.1.4.0.1.1.1]
ToolSvc.MdtPrepDataProviderTool                               ERROR In ProcessCSM - Couldn't record in the Container MDT Drift Circle Collection with hashID = 519 ext. id = [7.5.4.2.0.1.1.1]
ToolSvc.MdtPrepDataProviderTool                               ERROR In ProcessCSM - Couldn't record in the Container MDT Drift Circle Collection with hashID = 527 ext. id = [7.5.5.4.0.1.1.1]
ToolSvc.MdtPrepDataProviderTool                               ERROR In ProcessCSM - Couldn't record in the Container MDT Drift Circle Collection with hashID = 531 ext. id = [7.5.6.2.0.1.1.1]
ToolSvc.MdtPrepDataProviderTool                               ERROR In ProcessCSM - Couldn't record in the Container MDT Drift Circle Collection with hashID = 612 ext. id = [7.9.1.6.0.1.1.1]
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1253454202
DataProxy                                                   WARNING accessData: conversion failed for data object 1253454202/RPC_Measurements
 Returning NULL DataObject pointer
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1153454201
DataProxy                                                   WARNING accessData: conversion failed for data object 1153454201/MDT_DriftCircles
 Returning NULL DataObject pointer
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1253454202
DataProxy                                                   WARNING accessData: conversion failed for data object 1253454202/RPC_Measurements
 Returning NULL DataObject pointer
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1353454203
DataProxy                                                   WARNING accessData: conversion failed for data object 1353454203/TGC_Measurements
 Returning NULL DataObject pointer
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1153454201
DataProxy                                                   WARNING accessData: conversion failed for data object 1153454201/MDT_DriftCircles
 Returning NULL DataObject pointer
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1253454202
DataProxy                                                   WARNING accessData: conversion failed for data object 1253454202/RPC_Measurements
 Returning NULL DataObject pointer
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1353454203
DataProxy                                                   WARNING accessData: conversion failed for data object 1353454203/TGC_Measurements
 Returning NULL DataObject pointer
 ----------------------------------------------------------------------------------------
 | WARNING the standard way of running Muonboy on cosmics is with UseRoaFromMdt = false |
 ----------------------------------------------------------------------------------------
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1153454201
DataProxy                                                   WARNING accessData: conversion failed for data object 1153454201/MDT_DriftCircles
 Returning NULL DataObject pointer
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1353454203
DataProxy                                                   WARNING accessData: conversion failed for data object 1353454203/TGC_Measurements
 Returning NULL DataObject pointer
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1253454202
DataProxy                                                   WARNING accessData: conversion failed for data object 1253454202/RPC_Measurements
 Returning NULL DataObject pointer
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1253454202
DataProxy                                                   WARNING accessData: conversion failed for data object 1253454202/RPC_Measurements
 Returning NULL DataObject pointer
MooSegmentMaker.MooSegmentFinder.MuonPatternSegmentMaker    WARNING  Failed to retrieve RpcPrepDataContainer, will not recover rpc trigger hits
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1353454203
DataProxy                                                   WARNING accessData: conversion failed for data object 1353454203/TGC_Measurements
 Returning NULL DataObject pointer
MooSegmentMaker.MooSegmentFinder.MuonPatternSegmentMaker    WARNING  Failed to retrieve TgcPrepDataContainer, will not recover tgc trigger hits
</verbatim>
   * on every event. is themuon data OK??? seems not as i see no muon tracks whereas i do running on the same data in 14.1.0.Y


---+++ Status on Tue 1st July (rel_2)
   * Muon monitoring packages STILL not compiling in either val or bugfix <-- PLEASE FIX
   * LArRawChannelMonTool::fillHistograms () problem (see rel_0 bit below...) traced to LArOnlineID::isHecOnlineFebId by Michel Lefebvre and Frank Olaf BERGHAUS (thanks). this could be a problem as LArIdentifier package has many clients so may not be able to be fixed in a cache. Fix is in LArIdentifier-01-01-15 - but this requires a lot of recompiling - should make it into rel_3 bugfix i hope
   * Then running is similar to yesterday (ie. need rec.doHists=False to fix LArRawChannelMonTool::fillHistograms ()  problem, need to turn off muons to avoid crash MdtPrepRawDataNtuple crash)
   * ESD writing problem can be fixed by the line: ATLASCosmicFlags.doCaloTrkMuId=False then i can run...
---+++ Status on Mon 30th June (rel_1)
   * If using the HEAD of RecExCommission there is a problem due to LArCosmicFlags.doDSP - i hope this can be fixed soon...
   *  all probelms as below (rel_0) plus
<verbatim>
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1153454201
DataProxy                                                   WARNING accessData: conversion failed for data object 1153454201/MDT_DriftCircles
 Returning NULL DataObject pointer
MdtPrepRawDataNtuple                                          ERROR  Cannot retrieve MdtPrepDataContainer
CBNT_AthenaAware                                            WARNING  failure to  execute sub-algorithm:MdtPrepRawDataNtuple/MdtPrepRawDataNtuple

 *** Break *** segmentation violation
Using host libthread_db library "/lib64/tls/libthread_db.so.1".
Attaching to program: /proc/22118/exe, process 22118
[Thread debugging using libthread_db enabled]
[New Thread 4159502016 (LWP 22118)]
0xffffe410 in __kernel_vsyscall ()
#1  0x001fad23 in __waitpid_nocancel () from /lib/tls/libc.so.6
#2  0x001a47a9 in do_system () from /lib/tls/libc.so.6
#3  0x002d498d in system () from /lib/tls/libpthread.so.0
#4  0xf768aadf in TUnixSystem::Exec () from /afs/cern.ch/atlas/offline/external/LCGCMT/LCGCMT_54g/InstallArea/i686-slc4-gcc34-opt/lib/libCore.so
#5  0xf76904ed in TUnixSystem::StackTrace () from /afs/cern.ch/atlas/offline/external/LCGCMT/LCGCMT_54g/InstallArea/i686-slc4-gcc34-opt/lib/libCore.so
#6  0xf768d1aa in TUnixSystem::DispatchSignals () from /afs/cern.ch/atlas/offline/external/LCGCMT/LCGCMT_54g/InstallArea/i686-slc4-gcc34-opt/lib/libCore.so
#7  0xf768d238 in SigHandler () from /afs/cern.ch/atlas/offline/external/LCGCMT/LCGCMT_54g/InstallArea/i686-slc4-gcc34-opt/lib/libCore.so
#8  0xf768c485 in sighandler () from /afs/cern.ch/atlas/offline/external/LCGCMT/LCGCMT_54g/InstallArea/i686-slc4-gcc34-opt/lib/libCore.so
#9  <signal handler called>
#10 0xe71c79ce in IdentifiableContainer<MdtCsm>::const_iterator::operator* ()
   from /afs/cern.ch/atlas/software/builds/nightlies/bugfix/AtlasEvent/rel_1/InstallArea/i686-slc4-gcc34-opt/lib/libMuonMDT_CnvToolsLib.so
#11 0xe2c5fc15 in MdtRawDataNtuple::execute ()
   from /afs/cern.ch/atlas/software/builds/nightlies/bugfix/AtlasReconstruction/rel_1/InstallArea/i686-slc4-gcc34-opt/lib/libMuonCommAlgsLib.so
#12 0xf5abe721 in Algorithm::sysExecute ()
   from /afs/cern.ch/atlas/offline/external/GAUDI/v19r9-LCG54g/InstallArea/i686-slc4-gcc34-opt/lib/libGaudiKernel.so
#13 0xe779508e in CBNT_AthenaAware::execute ()
</verbatim>

---+++ Status on Sun 29th June (rel_0)
   * Not looking good!
   * Firstly muon monitoring packages still not compiling! - see 
   * Running OOTB with the line: ATLASCosmicFlags.doMuonHitMon = False gives a nasty crash ending like
<verbatim>
#11 0xe19c65d6 in LArRawChannelMonTool::fillHistograms ()
   from /afs/cern.ch/atlas/software/builds/nightlies/bugfix/AtlasAnalysis/rel_0/InstallArea/i686-slc4-gcc34-opt/lib/libLArMonToolsLib.so
#12 0xe759eba6 in ManagedMonitorToolBase::fillHists ()
   from /afs/cern.ch/atlas/software/builds/nightlies/bugfix/AtlasEvent/rel_0/InstallArea/i686-slc4-gcc34-opt/lib/libAthenaMonitoringLib.so
Previous frame inner to this frame (corrupt stack?)
</verbatim>
   * turning off monitoring: rec.doHist=False i can run over 7 events but then get:
<verbatim>
TRT_RDO_ContainerCnv                                           INFO Found container with TRT_LoLumRawData concrete type objects
ElementLink                                                   ERROR toPersistent: the internal state of link @0x51eca34c to the collection [] of type [CaloShowerContainer] is INVALID. Looks like you forgot to invoke setStorableObject. This link can not be written
T_AthenaPoolCustomCnv                                         FATAL Failed to convert object to persistent type: dereferencing invalid DataLink
AthenaPoolConverter                                           ERROR CreateRep failed, key = MuonClusterCollection
ToolSvc.StreamESDTool                                         ERROR Could not create Rep for DataObject (clid/key):2806 MuonClusterCollection
StreamESD                                                     ERROR streamObjects failed.
AthenaEventLoopMgr                                             INFO Execution of algorithm StreamESD failed with StatusCode::FAILURE
Py:PerfMonSvc        INFO [evt:  6] => VMem = 1998.922 MB     RSS = 1530.973 MB     malloc =  108.903 MB     cpu  = 1320.800 ms     real = 1336.574 ms
AthenaEventLoopMgr                                             INFO   ===>>>  done processing event #39007, run #43719 7 events processed so far  <<<===
AthenaEventLoopMgr                                            ERROR Terminating event processing loop due to errors
</verbatim>
   * turning off ESD writing as well: rec.doWriteESD =True , ATLASCosmicFlags.doFilteredESD=False runs but with our favourite muon errors on every event
<verbatim>
DataProxy                                                   WARNING accessData: conversion failed for data object 4187/MDTCSM
 Returning NULL DataObject pointer
ToolSvc.MdtPrepDataProviderTool                             WARNING No Mdt RDO container in StoreGate!
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 4190
DataProxy                                                   WARNING accessData: conversion failed for data object 4190/RPCPAD
 Returning NULL DataObject pointer
ToolSvc.RpcPrepDataProviderTool                             WARNING No RpC RDO container in StoreGate!
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1253454202
DataProxy                                                   WARNING accessData: conversion failed for data object 1253454202/RPC_Measurements
 Returning NULL DataObject pointer
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 4186
DataProxy                                                   WARNING accessData: conversion failed for data object 4186/TGCRDO
 Returning NULL DataObject pointer
ToolSvc.TgcPrepDataProviderTool                             WARNING No TGC RDO container in StoreGate!
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1153454201
DataProxy                                                   WARNING accessData: conversion failed for data object 1153454201/MDT_DriftCircles
 Returning NULL DataObject pointer
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1253454202
DataProxy                                                   WARNING accessData: conversion failed for data object 1253454202/RPC_Measurements
 Returning NULL DataObject pointer
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1353454203
DataProxy                                                   WARNING accessData: conversion failed for data object 1353454203/TGC_Measurements
 Returning NULL DataObject pointer
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1153454201
DataProxy                                                   WARNING accessData: conversion failed for data object 1153454201/MDT_DriftCircles
 Returning NULL DataObject pointer
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1253454202
DataProxy                                                   WARNING accessData: conversion failed for data object 1253454202/RPC_Measurements
 Returning NULL DataObject pointer
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1353454203
DataProxy                                                   WARNING accessData: conversion failed for data object 1353454203/TGC_Measurements
 Returning NULL DataObject pointer
 ----------------------------------------------------------------------------------------
 | WARNING the standard way of running Muonboy on cosmics is with UseRoaFromMdt = false |
 ----------------------------------------------------------------------------------------
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1153454201
DataProxy                                                   WARNING accessData: conversion failed for data object 1153454201/MDT_DriftCircles
 Returning NULL DataObject pointer
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1353454203
DataProxy                                                   WARNING accessData: conversion failed for data object 1353454203/TGC_Measurements
 Returning NULL DataObject pointer
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1253454202
DataProxy                                                   WARNING accessData: conversion failed for data object 1253454202/RPC_Measurements
 Returning NULL DataObject pointer
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1253454202
DataProxy                                                   WARNING accessData: conversion failed for data object 1253454202/RPC_Measurements
 Returning NULL DataObject pointer
MooSegmentMaker.MooSegmentFinder.MuonPatternSegmentMaker    WARNING  Failed to retrieve RpcPrepDataContainer, will not recover rpc trigger hits
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1353454203
DataProxy                                                   WARNING accessData: conversion failed for data object 1353454203/TGC_Measurements
 Returning NULL DataObject pointer
MooSegmentMaker.MooSegmentFinder.MuonPatternSegmentMaker    WARNING  Failed to retrieve TgcPrepDataContainer, will not recover tgc trigger hits
</verbatim>


---+++ Status on Thurs 26th June
   * LArCellRec-02-10-06 has made it into BUGFIX but MuonCommRecExample-00-03-36 has not yet
   * Seem to be able to run the RTT test OK adding the flags: 
<verbatim>
ATLASCosmicFlags.doMuonHitMon = False
</verbatim>
   * To stop a crash from Muon monitoring which seems to be because MuonRawDataMonitoring-00-00-53 doesnt compile - Nectarios is looking into this
   * Need to add the tags: MuonCommRecExample-00-03-36 and InDetTrackSelectorTool-00-01-21
   * SCT ESD writing issues seems to be fixed (not sure how??)
   * Muon data corruption seems to still be a problem
   * Suggested problem with TRT data in compressed format (ie. new data) RTT test is on M6 data which is old and so looks OK.
   * Seems to be a memory leak in bugfix - see comparison perfmon plots (especially page 6 of the pdfs) at: http://jboyd.web.cern.ch/jboyd/MemComp/
   * The main leaks seem to be in TRT_RodDecoder and SCT_RodDecoder - see: http://jboyd.web.cern.ch/jboyd/MemComp/leaks

---+++ Status on Wed 25th June
   * Same as Tuesday (see below)
   * I checked that the tags LArCellRec-02-10-06  and MuonCommRecExample-00-03-36 are in VAL so i hope they will be in BUGFIX tomorrow
   * even with InDetAlignmentMonitoring-00-02-14 which is in BUGFIX i see
<verbatim>
NameAuditor INFO About to Enter IDAlignMonManager Execute Method
ToolSvc.InDet::InDetDetailedTrackSelectorTool WARNING Track Selector failed to extrapolate track to the vertex: (0,0,0)
ToolSvc.InDet::InDetDetailedTrackSelectorTool WARNING Track preselection: Zero track pointer received. This track will not pass.
FATAL 2008-Jun-25 15:15:53 [static void ers::ErrorHandler::SignalHandler::action(...) at ers/src/ErrorHandler.cxx:88] Got signal 11 Segmentation fault (invalid memory reference)
</verbatim>
   * This is contuinuation of savanagh bug #38098
   * Also still see crash from SCT ESD writing and Muon monitoring
   * Muon monitoring should be fixed with: MuonTrigCoinData (00-00-03) (in val and bugfix we have: MuonTrigCoinData-00-02-00 )
   * SCT ESD writing is hopefully under investigation
   * Muon data is (probably still corrupt too!)
   * With above changes i can run the 500 event RTT test OK. not sure the data is OK.
   * See some plots comparing rel_2, 14.1.0.Y with bugfix rel_3 with above fixes etc...
   * Comparing TRT (looks similar but not exactly the same)
   * NtrtTracks_comp.gif: <br />
     <img src="%ATTACHURLPATH%/NtrtTracks_comp.gif" alt="NtrtTracks_comp.gif" width='696' height='472' />

   * TrtTrackPars_comp.gif: <br />
     <img src="%ATTACHURLPATH%/TrtTrackPars_comp.gif" alt="TrtTrackPars_comp.gif" width='696' height='472' />
   * Comparing SCT (looks exactly the same)

   * NsctTracks_comp.gif: <br />
     <img src="%ATTACHURLPATH%/NsctTracks_comp.gif" alt="NsctTracks_comp.gif" width='696' height='472' />

   * SctTrackPars_comp.gif: <br />
     <img src="%ATTACHURLPATH%/SctTrackPars_comp.gif" alt="SctTrackPars_comp.gif" width='696' height='472' />

   * Comparing LAr (looks exactly the same)

   * LArCellE_comp.gif: <br />
     <img src="%ATTACHURLPATH%/LArCellE_comp.gif" alt="LArCellE_comp.gif" width='696' height='472' />

   * Comparing Tile (looks similar(ish) but not exactly the same)
   * TileCellE_comp.gif: <br />
     <img src="%ATTACHURLPATH%/TileCellE_comp.gif" alt="TileCellE_comp.gif" width='696' height='472' />


   * Because of Muon issues from IDC migration dont try to compare that yet...

   * For more info please look at NTP, HIST file and logfile from this job at:
/afs/cern.ch/user/g/gencomm/w0/jboyd/bugfix_3
   * NTP can be compared to the one from 14.1.0.Y RTT which is called rtt.root in same dir

---+++Status on Tuesday 24th June
   * General recipe
<verbatim>
> > > > > > > > source setup.sh -tag=bugfix,AtlasOffline,rel_2,oneTest
> > > > > > > > source $AtlasArea/AtlasOfflineRunTime/cmt/setup.sh
> > > > > > > > cmt co MuonSpectrometer/MuonCommissioning/MuonCommRecExample
> > > > > > > > cd  MuonSpectrometer/MuonCommissioning/MuonCommRecExample/cmt
> > > > > > > > cmt config; gmake -j4
> > > > > > > > cd -
> > > > > > cmt co -r LArCellRec-02-10-06 LArCalorimeter/LArCellRec      
> > > > > > cd LArCalorimeter/LArCellRec/cmt
> > > > > > cmt config; gmake -j4
> > > > > > cd -
> > > > > > > > get_files -jo RecExCommission_ATN.py
> > > > > > > > edit this to have:
> > > > > > > > rec.doHist=False
> > > > > > > > source RecExCommissionData_links.sh
> > > > > > > > athena -c RTT=True RecExCommission_ATN.py >& log &
</verbatim>
   * Fix for IDAlignement bug in tag: InDetAlignmentMonitoring-00-02-14   
   * Remaining problems
   * Muon monitoring bug
<verbatim>
Shortened traceback (most recent user call last):
  File 
"/afs/cern.ch/atlas/software/builds/nightlies/bugfix/AtlasCore/rel_2/InstallArea/share/bin/athena.py", 
line 367, in <module>
    include( script )
  File "./RecExCommission_ATN.py", line 102, in <module>
    include( "RecExCommission/RecExCommission_topOptions.py" )
  File 
"/afs/cern.ch/atlas/software/builds/nightlies/bugfix/AtlasAnalysis/rel_2/InstallArea/jobOptions/RecExCommission/RecExCommission_topOptions.py", 
line 270, in <module>
    include( "RecExCommission/RecExCommissionMonitoring_jobOptions.py" )
  File 
"/afs/cern.ch/atlas/software/builds/nightlies/bugfix/AtlasAnalysis/rel_2/InstallArea/jobOptions/RecExCommission/RecExCommissionMonitoring_jobOpt
ions.py", 
line 23, in <module>
    include("MuonCommRecExample/MuDetCosmicMonitoring_jobOptions.py")
  File 
"/afs/cern.ch/user/g/gencomm/w0/jboyd/InstallArea/jobOptions/MuonCommRecExample/MuDetCosmicMonitoring_jobOptions.py", 
line 6, in <module>
    include ( "MuonDQAMonitoring/MuonDQAMonitoring_jobOptions.py")
  File 
"/afs/cern.ch/atlas/software/builds/nightlies/bugfix/AtlasAnalysis/rel_2/InstallArea/jobOptions/MuonDQAMonitoring/MuonDQAMonitoring_jobOptions.p
y", 
line 11, in <module>
    include("MuonRawDataMonitoring/MuonRawDataMonitoring_options.py")
  File 
"/afs/cern.ch/atlas/software/builds/nightlies/bugfix/AtlasAnalysis/rel_2/InstallArea/jobOptions/MuonRawDataMonitoring/MuonRawDataMonitoring_opti
ons.py", 
line 54, in <module>
    from MuonRawDataMonitoring.MuonRawDataMonitoringConf import 
MDTRawDataValAlg
ImportError: No module named 
MuonRawDataMonitoring.MuonRawDataMonitoringConf
Py:Athena            INFO leaving with code 3: "an error occurred when 
importing a file"
</verbatim>
   * Apparently this is because we need a later tag of:  MuonTrigCoinData (00-00-03)
   * SCT ESD writing problem
<verbatim>
NameAuditor                                                    INFO About 
to Enter StreamESD Execute Method
   DbSession     Info     Open     DbSession    
Domain[ROOT_All]     Info >   Access   DbDomain     UPDATE    [ROOT_All] 
ClassIDSvc                                                     INFO  
getRegistryEntries: read 12904 CLIDRegistry entries for module ALL
ByteStreamCnvSvc                                               INFO No 
converter for object   CLID= 1153454201
DataProxy                                                   WARNING 
accessData: conversion failed for data object 1153454201/MDT_DriftCircles
 Returning NULL DataObject pointer  
ByteStreamCnvSvc                                               INFO No 
converter for object   CLID= 1253454202
DataProxy                                                   WARNING 
accessData: conversion failed for data object 1253454202/RPC_Measurements
 Returning NULL DataObject pointer  
ByteStreamCnvSvc                                               INFO No 
converter for object   CLID= 1353454203
DataProxy                                                   WARNING 
accessData: conversion failed for data object 1353454203/TGC_Measurements
 Returning NULL DataObject pointer  
PoolXMLFileCatalog     Info filename cosmicsESD.pool.root does not exist
XMLFileCatalog     Info lookupFilebyPFN: Empty File ID found
PoolXMLFileCatalog     Info filename cosmicsESD.pool.root does not exist
XMLFileCatalog     Info lookupFilebyPFN: Empty File ID found
PoolXMLFileCatalog     Info filename cosmicsESD.pool.root does not exist
XMLFileCatalog     Info lookupFilebyPFN: Empty File ID found
PoolXMLFileCatalog     Info filename cosmicsESD.pool.root does not exist
XMLFileCatalog     Info lookupFilebyPFN: Empty File ID found
PoolXMLFileCatalog     Info filename cosmicsESD.pool.root does not exist
XMLFileCatalog     Info lookupFilebyPFN: Empty File ID found
Domain[ROOT_All]     Info ->  Access   DbDatabase   CREATE    [ROOT_All] 
B8979CBE-ED41-DD11-A46B-000423D5ADDA
Domain[ROOT_All]     Info                           cosmicsESD.pool.root
cosmicsESD.pool.root   Always Root file version:51800
  StorageSvc     Info Building shape according to reflection information 
using shape ID for:
  StorageSvc     Info EventInfo_p2 [22006E19-F0DA-4EFB-AF55-6FBDA421BF06]
PixelRDO_Contai...   INFO PixelRDO_ContainerCnv::initialize()
  StorageSvc     Info Building shape according to reflection information 
using shape ID for:
  StorageSvc     Info InDetRawDataContainer_p1 
[DA76970C-E019-43D2-B2F9-25660DCECD9D]
SCT_RDO_Contain...   INFO SCT_RDO_ContainerCnv::initialize()
SCT_RDO_ContainerCnv                                           INFO Found 
container with SCT3_RawData concrete type objects
FATAL 2008-Jun-24 15:02:08 [static void 
ers::ErrorHandler::SignalHandler::action(...) at 
ers/src/ErrorHandler.cxx:88] Got signal 11 Segmentation fault (invalid 
memory reference)
</verbatim>
   * emailed Christian/Thijs about this
   * Also get ERRORs like:
<verbatim>
NameAuditor                                                    INFO About to Enter MdtPrepRawDataNtuple Execute Method
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1153454201
DataProxy                                                   WARNING accessData: conversion failed for data object 1153454201/MDT_DriftCircles
 Returning NULL DataObject pointer  
MdtPrepRawDataNtuple                                          ERROR  Cannot retrieve MdtPrepDataContainer 
NameAuditor                                                    INFO  About to Enter ExceptionSvc Initialization Method
NameAuditor                                                    INFO Just Exited ExceptionSvc Initialization Method
MemStatAuditor                                                 INFO Memory usage has changed after  ExceptionSvc        Initialization Method   virtual size = 1347.09 MB    
        resident set size = 1145.43 MB deltaVsize = 0.421875  MB 
NameAuditor                                                    INFO Just Exited MdtPrepRawDataNtuple Execute Method
MemStatAuditor                                                 INFO Memory usage has changed after  MdtPrepRawDataNtuple        Execute Method  virtual size = 1347.09 MB    
        resident set size = 1145.44 MB deltaVsize = 0  MB 
CBNT_AthenaAware                                            WARNING  failure to  execute sub-algorithm:MdtPrepRawDataNtuple/MdtPrepRawDataNtuple
NameAuditor                                                    INFO About to Enter MdtRawDataNtuple Execute Method
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 4187
DataProxy                                                   WARNING accessData: conversion failed for data object 4187/MDTCSM
 Returning NULL DataObject pointer  
MdtRawDataNtuple                                              ERROR Could not find Mdt CSM container
NameAuditor                                                    INFO Just Exited MdtRawDataNtuple Execute Method
MemStatAuditor                                                 INFO Memory usage has changed after  MdtRawDataNtuple    Execute Method  virtual size = 1347.38 MB       resid
ent set size = 1145.55 MB deltaVsize = 0.285156  MB 
CBNT_AthenaAware                                            WARNING  failure to  execute sub-algorithm:MdtRawDataNtuple/MdtRawDataNtuple
NameAuditor                                                    INFO About to Enter RpcPrepRawDataNtuple Execute Method
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1253454202
DataProxy                                                   WARNING accessData: conversion failed for data object 1253454202/RPC_Measurements
 Returning NULL DataObject pointer  
RpcPrepRawDataNtuple                                          ERROR  Cannot retrieve RpcPrepDataContainer 
NameAuditor                                                    INFO Just Exited RpcPrepRawDataNtuple Execute Method
MemStatAuditor                                                 INFO Memory usage has changed after  RpcPrepRawDataNtuple        Execute Method  virtual size = 1348.09 MB    
        resident set size = 1145.71 MB deltaVsize = 0.710938  MB 
CBNT_AthenaAware                                            WARNING  failure to  execute sub-algorithm:RpcPrepRawDataNtuple/RpcPrepRawDataNtuple
NameAuditor                                                    INFO About to Enter RpcRawDataNtuple Execute Method
StoreGateSvc                                                WARNING retrieve(default): No valid proxy for default object 
 of type RpcSectorLogicContainer(CLID 1298668563)
RpcRawDataNtuple                                              ERROR Cannot retrieve the RpcSectorLogicContainer
NameAuditor                                                    INFO Just Exited RpcRawDataNtuple Execute Method
MemStatAuditor                                                 INFO Memory usage has changed after  RpcRawDataNtuple    Execute Method  virtual size = 1349.37 MB       resid
ent set size = 1146.09 MB deltaVsize = 1.28125  MB 
NameAuditor                                                    INFO About to Enter TgcPrepRawDataNtuple Execute Method
ByteStreamCnvSvc                                               INFO No converter for object   CLID= 1353454203
DataProxy                                                   WARNING accessData: conversion failed for data object 1353454203/TGC_Measurements
 Returning NULL DataObject pointer  
TgcPrepRawDataNtuple                                          ERROR  Cannot retrieve TgcPrepDataContainer 
NameAuditor                                                    INFO Just Exited TgcPrepRawDataNtuple Execute Method
MemStatAuditor                                                 INFO Memory usage has changed after  TgcPrepRawDataNtuple        Execute Method  virtual size = 1351.37 MB    
        resident set size = 1146.63 MB deltaVsize = 2  MB 
CBNT_AthenaAware                                            WARNING  failure to  execute sub-algorithm:TgcPrepRawDataNtuple/TgcPrepRawDataNtuple
</verbatim>
   * on this Rosy says:
<verbatim>
Due to the migration on the new Identifiable Containers in the ByteStream, there are some parts
of the algorithms in the filling of the raw data ntuple side that are not correct.
(MdtPrepRawDataNtuple for example, as you see)
We are working on this
</verbatim>

---+++old stuff (out dated)...

   * InDet, CTP OK
   * Cannot run with doLAr = False
   * Crash in LAr:
<verbatim>
Exited CBNT_LArCell Execute Method
NameAuditor                                                    INFO About
to Enter CBNTAA_LArDigits Execute Method
FATAL 2008-Jun-19 12:03:41 [static void
ers::ErrorHandler::SignalHandler::action(...) at
ers/src/ErrorHandler.cxx:88] Got signal 11 Segmentation fault (invalid
memory reference)
</verbatim>

Guillaume: Solved in LArROD-02-11-16. The version in 14.2.0 has potentially the same problem, so we would probably need a branch tag there.

   * Muons crash:
<verbatim>
    ServiceMgr += CfgMgr.MdtCalibrationSvc(CorrectTof = False)
  File
"/afs/cern.ch/atlas/software/builds/nightlies/bugfix/AtlasEvent/rel_4/InstallArea/python/MdtCalibSvc/MdtCalibSvcConf.py",
line 28, in __init__
    setattr(self, n, v)
AttributeError: 'MdtCalibrationSvc' object has no attribute 'CorrectTof'
Py:Athena            INFO leaving with code 8: "an unknown exception
occurred"
</verbatim>

Soved in MuonCommRecExample-00-03-34.

   * HLT crash:
<verbatim>
  File "/afs/cern.ch/atlas/software/builds/nightlies/bugfix/AtlasTrigger/rel_4/InstallArea/jobOptions/TriggerRelease/RecExCommission_deserialize.py", line 168, in <module>
    extr.Navigation.Serializer = ser
AttributeError: 'HLTNavigationOffline' object has no attribute 'Serializer'
Py:Athena            INFO leaving with code 8: "an unknown exception occurred"
</verbatim>
   * Answer from Phillip on this:
<verbatim>
There was a problem related to the serializer in the rel_4 nightly that should
be fixed for rel_5.
</verbatim>
   * Conclusion - work to do!

-- Main.JamieBoyd - 03 Jun 2008
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who
gif	LArCellE_comp.gif	r1	manage	8.1 K	2008-06-25 - 16:29	JamieBoyd
gif	NsctTracks_comp.gif	r1	manage	7.7 K	2008-06-25 - 16:28	JamieBoyd
gif	NtrtTracks_comp.gif	r1	manage	8.1 K	2008-06-25 - 16:27	JamieBoyd
gif	SctTrackPars_comp.gif	r1	manage	13.5 K	2008-06-25 - 16:28	JamieBoyd
gif	TileCellE_comp.gif	r1	manage	7.1 K	2008-06-25 - 16:29	JamieBoyd
gif	TrtTrackPars_comp.gif	r1	manage	13.3 K	2008-06-25 - 16:27	JamieBoyd
Topic revision: r30 - 2008-07-10 - JamieBoyd
Main
Webs
Public webs
Welcome Guest
- Cern Search
- TWiki Search
- Google Search
Main All webs
Copyright &© 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback