TWiki> Main Web>TWikiUsers>LeonardoSala>LSIOforAnalysisOperations>LSIOforAnalysisOperations-20091203_autodet (2009-12-04, unknown) EditAttachPDF

Standard test performed on 2009-12-03 with auto TFileAdaptor settings

Dataset: /QCD_Pt80/Summer09-MC_31X_V3-v1/GEN-SIM-RECO

Cfgs: CSCSkim, PAT, JRA

TFileAdaptor:

  • CacheHint=auto-detect
  • ReadHint=auto-detect (lazy-download for dCache-RFIO, storage-only for others)
  • Cache=20MB

Involved sites (info from BDII):

  • CIEMAT - dcache
  • DESY - dcache 1.9.5-9
  • KNU - dcache 1.8.0
  • Pisa - dcache
  • Rome - dcache 1.8.0
  • UCSD - hadoop+bestman 2.2.1.2.e1

Results - report

PAT seems to masters better IO than JRA, generally speaking. Can this be due to different UserTimes?

CIEMAT

Big improvements wrt standard settings, CPU eff~90% (~30% improvement).

DESY

Very big improvement, from ~30% to ~90% of CPU Efficiency? How much is the slightly lighter load and how much the auto-detect settings?

http://dashb-cms-job.cern.ch/dashboard/request.py/jobsummary#user=&site=T2_DE_DESY&ce=&submissiontool=&dataset&application=&rb=&activity=&grid=&date1=2009-12-03%2023%3A05%3A44&date2=2009-12-04%2015%3A05%3A44&sortby=activity&nbars=&jobtype=&tier=&status=running&status=done&check=submitted

KNU

Even if not big as the CIEMAT, DESY, it shows a ~10-15% improvement.

Pisa

Very bad CPU Efficiency... Writing the file to some shared area (nfs, afs)? SW afs location problem? Pisa is the only site using afs sw area

Lot of time for opening files, for example wrt CIEMAT:

CIEMAT 

03-Dec-2009 20:12:31 CET  Initiating request to open file dcap://dchadm.ciemat.es:22125/pnfs/ciemat.es/data/cms/prod/store/mc/Summer09/QCD_Pt80/GEN-SIM-RECO/MC_31X_V3-v1/0010/8051F6A9-DC80-DE11-A06C-001F29C7BDEE.root
03-Dec-2009 20:12:35 CET  Successfully opened file dcap://dchadm.ciemat.es:22125/pnfs/ciemat.es/data/cms/prod/store/mc/Summer09/QCD_Pt80/GEN-SIM-RECO/MC_31X_V3-v1/0010/8051F6A9-DC80-DE11-A06C-001F29C7BDEE.root

Pisa 

03-Dec-2009 20:36:23 CET  Initiating request to open file dcap://cmsdcache.pi.infn.it/pnfs/pi.infn.it/data/cms/store/mc/Summer09/QCD_Pt80/GEN-SIM-RECO/MC_31X_V3-v1/0010/8051F6A9-DC80-DE11-A06C-001F29C7BDEE.root
03-Dec-2009 20:50:44 CET  Successfully opened file dcap://cmsdcache.pi.infn.it/pnfs/pi.infn.it/data/cms/store/mc/Summer09/QCD_Pt80/GEN-SIM-RECO/MC_31X_V3-v1/0010/8051F6A9-DC80-DE11-A06C-001F29C7BDEE.root

4 secs vs 14 mins!!!!!!

Also, mani 8009 errors (configuration):

http://dashb-cms-job.cern.ch/dashboard/request.py/jobsummary#user=&site=T2_IT_Pisa&ce=&submissiontool=&dataset&application=&rb=&activity=&grid=&date1=2009-12-03%2023%3A05%3A44&date2=2009-12-04%2015%3A05%3A44&sortby=activity&nbars=&jobtype=&tier=&status=running&status=done&check=submitted

Rome

Small improvement wrt standard settings... This is strange.

http://dashb-cms-job.cern.ch/dashboard/request.py/jobsummary#user=&site=T2_IT_Rome&ce=&submissiontool=&dataset&application=&rb=&activity=&grid=&date1=2009-12-03%2023%3A05%3A44&date2=2009-12-04%2015%3A05%3A44&sortby=activity&nbars=&jobtype=&tier=&status=running&status=done&check=submitted

UCSD

It does not improve the already good performances using the auto-detect features (which means storage-only in this case). Probably using a lazy-download setting it will perform as the other sites.

Results - Images

PAT

PAT1_RHAUTO_CHAUTO_CACHE20-20091203-CrabCpuPercentage.pngPAT1_RHAUTO_CHAUTO_CACHE20-20091203-CrabUserCpuTime.png PAT1_RHAUTO_CHAUTO_CACHE20-20091203-CrabWrapperTime.png PAT1_RHAUTO_CHAUTO_CACHE20-20091203-read-total-msecs.png PAT1_RHAUTO_CHAUTO_CACHE20-20091203-tstoragefile-read-total-msecs.png

JRA

JRA1_RHAUTO_CHAUTO_CACHE20-20091203-CrabCpuPercentage.png JRA1_RHAUTO_CHAUTO_CACHE20-20091203-CrabUserCpuTime.png JRA1_RHAUTO_CHAUTO_CACHE20-20091203-CrabWrapperTime.png JRA1_RHAUTO_CHAUTO_CACHE20-20091203-read-total-msecs.png JRA1_RHAUTO_CHAUTO_CACHE20-20091203-tstoragefile-read-total-msecs.png

CSCSkim

Results - Tables

PAT

  CIEMAT DESY KNU Pisa Rome UCSD
Success 100.0% (20 / 20) 100.0% (20 / 20) 100.0% (20 / 20) 100.0% (20 / 20) 100.0% (20 / 20) 100.0% (20 / 20)
CrabCpuPercentage 90.85 +- 9.36 93.35 +- 2.26 84.90 +- 9.17 9.10 +- 1.34 85.05 +- 2.36 81.50 +- 2.97
CrabStageoutTime 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00
CrabSysCpuTime 20.78 +- 4.51 32.72 +- 6.79 64.23 +- 14.39 49.13 +- 10.88 35.71 +- 7.44 23.11 +- 30.30
CrabUserCpuTime 1243.73 +- 253.42 1512.51 +- 319.54 2701.12 +- 561.88 2303.36 +- 502.42 1610.82 +- 363.83 1833.81 +- 514.84
CrabWrapperTime 1377.70 +- 267.46 1651.70 +- 338.02 3191.25 +- 586.33 24300.10 +- 4534.76 1933.05 +- 428.70 2290.05 +- 762.59
OPEN
dcap-open-num-successful-operations 5.30 +- 1.10 // 5.30 +- 1.10 5.30 +- 1.10 5.30 +- 1.10 //
file-open-num-successful-operations 1.00 +- 0.00 1.00 +- 0.00 1.00 +- 0.00 1.00 +- 0.00 1.00 +- 0.00 6.30 +- 1.10
gsidcap-open-num-successful-operations // 5.30 +- 1.10 // // // //
dcap-open-num-operations 5.30 +- 1.10 // 5.30 +- 1.10 5.30 +- 1.10 5.30 +- 1.10 //
file-open-num-operations 1.00 +- 0.00 1.00 +- 0.00 1.00 +- 0.00 1.00 +- 0.00 1.00 +- 0.00 6.30 +- 1.10
gsidcap-open-num-operations // 5.30 +- 1.10 // // // //
dcap-open-total-msecs 2508.69 +- 3351.33 // 4878.75 +- 1623.79 8770.37 +- 5415.34 3456.45 +- 1187.21 //
file-open-total-msecs 0.11 +- 0.01 0.16 +- 0.02 0.28 +- 0.06 0.24 +- 0.05 0.17 +- 0.01 10346.66 +- 6566.78
gsidcap-open-total-msecs // 713.77 +- 999.81 // // // //
dcap-open-total-megabytes 0.00 +- 0.00 // 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 //
file-open-total-megabytes 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00
gsidcap-open-total-megabytes // 0.00 +- 0.00 // // // //
READ
dcap-read-num-successful-operations 43.55 +- 8.56 // 43.55 +- 8.56 43.55 +- 8.56 43.55 +- 8.56 //
file-read-num-successful-operations 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 127473.15 +- 26728.12
gsidcap-read-num-successful-operations // 43.55 +- 8.56 // // // //
tstoragefile-read-async-num-successful-operations 12729.20 +- 2684.38 12729.20 +- 2684.38 12729.20 +- 2684.38 12729.20 +- 2684.38 12729.20 +- 2684.38 12729.20 +- 2684.38
tstoragefile-read-num-successful-operations 127473.15 +- 26728.12 127473.15 +- 26728.12 127473.15 +- 26728.12 127473.15 +- 26728.12 127473.15 +- 26728.12 127473.15 +- 26728.12
tstoragefile-read-prefetch-to-cache-num-successful-operations 117618.85 +- 24664.45 117618.85 +- 24664.45 117618.85 +- 24664.45 117618.85 +- 24664.45 117618.85 +- 24664.45 117618.85 +- 24664.45
dcap-read-num-operations 43.55 +- 8.56 // 43.55 +- 8.56 43.55 +- 8.56 43.55 +- 8.56 //
file-read-num-operations 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 127473.15 +- 26728.12
gsidcap-read-num-operations // 43.55 +- 8.56 // // // //
tstoragefile-read-async-num-operations 12729.20 +- 2684.38 12729.20 +- 2684.38 12729.20 +- 2684.38 12729.20 +- 2684.38 12729.20 +- 2684.38 12729.20 +- 2684.38
tstoragefile-read-num-operations 127473.15 +- 26728.12 127473.15 +- 26728.12 127473.15 +- 26728.12 127473.15 +- 26728.12 127473.15 +- 26728.12 127473.15 +- 26728.12
tstoragefile-read-prefetch-to-cache-num-operations 117721.75 +- 24685.83 117721.75 +- 24685.83 117721.75 +- 24685.83 117721.75 +- 24685.83 117721.75 +- 24685.83 117721.75 +- 24685.83
dcap-read-total-msecs 105018.49 +- 76208.73 // 110480.64 +- 31690.55 21103776.50 +- 4086017.71 270656.64 +- 76751.42 //
file-read-total-msecs 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 191377.70 +- 72382.90
gsidcap-read-total-msecs // 79738.49 +- 20888.56 // // // //
tstoragefile-read-async-total-msecs 30519.44 +- 21753.88 28948.76 +- 8772.23 38362.26 +- 10188.27 7536972.00 +- 1499534.01 98919.87 +- 29651.40 120443.22 +- 55814.56
tstoragefile-read-prefetch-to-cache-total-msecs 20924.70 +- 18966.11 18110.74 +- 5763.08 19610.34 +- 7179.40 4593529.55 +- 969786.45 64997.41 +- 19754.54 112860.54 +- 55103.67
tstoragefile-read-total-msecs 96500.00 +- 72656.73 72000.00 +- 17670.60 101500.00 +- 29899.83 1000000.00 +- 2500.00 239250.00 +- 67699.24 305500.00 +- 127548.03
dcap-read-total-megabytes 5013.24 +- 987.28 // 5013.24 +- 987.28 5013.24 +- 987.28 5013.24 +- 987.28 //
file-read-total-megabytes 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 848.64 +- 177.67
gsidcap-read-total-megabytes // 5013.24 +- 987.28 // // // //
tfile-read-total-megabytes 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00
tstoragefile-read-async-total-megabytes 4788.07 +- 999.53 4788.07 +- 999.53 4788.07 +- 999.53 4788.07 +- 999.53 4788.07 +- 999.53 4788.07 +- 999.53
tstoragefile-read-prefetch-to-cache-total-megabytes 786.48 +- 164.93 786.48 +- 164.93 786.48 +- 164.93 786.48 +- 164.93 786.48 +- 164.93 786.48 +- 164.93
tstoragefile-read-total-megabytes 848.64 +- 177.67 848.64 +- 177.67 848.64 +- 177.67 848.64 +- 177.67 848.64 +- 177.67 848.64 +- 177.67
SEEK
tstoragefile-seek-num-successful-operations 363076.90 +- 76126.71 363076.90 +- 76126.71 363076.90 +- 76126.71 363076.90 +- 76126.71 363076.90 +- 76126.71 363076.90 +- 76126.71
tstoragefile-seek-num-operations 363076.90 +- 76126.71 363076.90 +- 76126.71 363076.90 +- 76126.71 363076.90 +- 76126.71 363076.90 +- 76126.71 363076.90 +- 76126.71
tstoragefile-seek-total-msecs 698.53 +- 146.00 1290.56 +- 313.25 3128.47 +- 664.80 2316.85 +- 591.24 1411.68 +- 309.62 1402.82 +- 1147.59
tstoragefile-seek-total-megabytes 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00

JRA

  CIEMAT DESY KNU Pisa Rome UCSD
Success 100.0% (20 / 20) 100.0% (20 / 20) 100.0% (20 / 20) 100.0% (20 / 20) 100.0% (20 / 20) 100.0% (20 / 20)
CrabCpuPercentage 80.00 +- 13.58 79.50 +- 4.82 75.05 +- 5.67 1.95 +- 0.22 52.20 +- 4.24 74.55 +- 2.36
CrabStageoutTime 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00
CrabSysCpuTime 25.57 +- 6.11 35.51 +- 9.81 89.60 +- 25.05 61.79 +- 17.07 46.07 +- 12.13 10.48 +- 2.30
CrabUserCpuTime 368.75 +- 89.25 477.58 +- 120.20 805.90 +- 193.85 661.56 +- 167.23 496.70 +- 128.22 502.79 +- 120.38
CrabWrapperTime 490.35 +- 103.09 646.20 +- 153.63 1215.90 +- 301.17 29057.90 +- 6267.35 1041.60 +- 254.16 699.55 +- 165.37
OPEN
dcap-open-num-successful-operations 7.20 +- 1.72 // 7.20 +- 1.72 7.20 +- 1.72 7.20 +- 1.72 //
file-open-num-successful-operations // // // // // 7.20 +- 1.72
gsidcap-open-num-successful-operations // 7.20 +- 1.72 // // // //
dcap-open-num-operations 7.20 +- 1.72 // 7.20 +- 1.72 7.20 +- 1.72 7.20 +- 1.72 //
file-open-num-operations // // // // // 7.20 +- 1.72
gsidcap-open-num-operations // 7.20 +- 1.72 // // // //
dcap-open-total-msecs 1285.23 +- 1810.78 // 5065.38 +- 1480.11 11316.27 +- 6484.58 4608.63 +- 2719.21 //
file-open-total-msecs // // // // // 669.39 +- 332.56
gsidcap-open-total-msecs // 1415.28 +- 2064.05 // // // //
dcap-open-total-megabytes 0.00 +- 0.00 // 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 //
file-open-total-megabytes // // // // // 0.00 +- 0.00
gsidcap-open-total-megabytes // 0.00 +- 0.00 // // // //
READ
dcap-read-num-successful-operations 61.30 +- 14.45 // 61.30 +- 14.45 61.30 +- 14.45 61.30 +- 14.45 //
file-read-num-successful-operations // // // // // 167939.90 +- 41688.94
gsidcap-read-num-successful-operations // 61.30 +- 14.45 // // // //
tstoragefile-read-async-num-successful-operations 10432.45 +- 2590.18 10432.45 +- 2590.18 10432.45 +- 2590.18 10432.45 +- 2590.18 10432.45 +- 2590.18 10432.45 +- 2590.18
tstoragefile-read-num-successful-operations 167939.90 +- 41688.94 167939.90 +- 41688.94 167939.90 +- 41688.94 167939.90 +- 41688.94 167939.90 +- 41688.94 167939.90 +- 41688.94
tstoragefile-read-prefetch-to-cache-num-successful-operations 167452.30 +- 41570.29 167452.30 +- 41570.29 167452.30 +- 41570.29 167452.30 +- 41570.29 167452.30 +- 41570.29 167452.30 +- 41570.29
dcap-read-num-operations 61.30 +- 14.45 // 61.30 +- 14.45 61.30 +- 14.45 61.30 +- 14.45 //
file-read-num-operations // // // // // 167939.90 +- 41688.94
gsidcap-read-num-operations // 61.30 +- 14.45 // // // //
tstoragefile-read-async-num-operations 10432.45 +- 2590.18 10432.45 +- 2590.18 10432.45 +- 2590.18 10432.45 +- 2590.18 10432.45 +- 2590.18 10432.45 +- 2590.18
tstoragefile-read-num-operations 167939.90 +- 41688.94 167939.90 +- 41688.94 167939.90 +- 41688.94 167939.90 +- 41688.94 167939.90 +- 41688.94 167939.90 +- 41688.94
tstoragefile-read-prefetch-to-cache-num-operations 167599.90 +- 41606.79 167599.90 +- 41606.79 167599.90 +- 41606.79 167599.90 +- 41606.79 167599.90 +- 41606.79 167599.90 +- 41606.79
dcap-read-total-msecs 94420.19 +- 36813.80 // 324275.19 +- 103798.44 27857634.50 +- 6112416.15 494767.90 +- 134266.31 //
file-read-total-msecs // // // // // 65879.77 +- 17750.80
gsidcap-read-total-msecs // 131328.42 +- 35330.24 // // // //
tstoragefile-read-async-total-msecs 30953.80 +- 8995.62 46109.83 +- 14292.40 119013.29 +- 40299.95 10260659.50 +- 2308586.81 181043.73 +- 50054.89 103431.42 +- 26151.39
tstoragefile-read-prefetch-to-cache-total-msecs 21147.65 +- 5796.83 29894.95 +- 9508.76 76174.54 +- 24249.97 6427615.60 +- 1540583.79 115818.83 +- 33632.30 96456.81 +- 24703.35
tstoragefile-read-total-msecs 86500.00 +- 33226.50 117750.00 +- 31483.13 291750.00 +- 90902.08 1000000.00 +- 2500.00 432750.00 +- 117510.37 163500.00 +- 42766.81
dcap-read-total-megabytes 7082.96 +- 1672.26 // 7082.96 +- 1672.26 7082.96 +- 1672.26 7082.96 +- 1672.26 //
file-read-total-megabytes // // // // // 381.61 +- 94.39
gsidcap-read-total-megabytes // 7082.96 +- 1672.26 // // // //
tfile-read-total-megabytes 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00
tstoragefile-read-async-total-megabytes 6143.54 +- 1521.35 6143.54 +- 1521.35 6143.54 +- 1521.35 6143.54 +- 1521.35 6143.54 +- 1521.35 6143.54 +- 1521.35
tstoragefile-read-prefetch-to-cache-total-megabytes 364.05 +- 90.37 364.05 +- 90.37 364.05 +- 90.37 364.05 +- 90.37 364.05 +- 90.37 364.05 +- 90.37
tstoragefile-read-total-megabytes 381.61 +- 94.39 381.61 +- 94.39 381.61 +- 94.39 381.61 +- 94.39 381.61 +- 94.39 381.61 +- 94.39
SEEK
tstoragefile-seek-num-successful-operations 502992.10 +- 124866.02 502992.10 +- 124866.02 502992.10 +- 124866.02 502992.10 +- 124866.02 502992.10 +- 124866.02 502992.10 +- 124866.02
tstoragefile-seek-num-operations 502992.10 +- 124866.02 502992.10 +- 124866.02 502992.10 +- 124866.02 502992.10 +- 124866.02 502992.10 +- 124866.02 502992.10 +- 124866.02
tstoragefile-seek-total-msecs 948.45 +- 235.25 1461.00 +- 462.24 4254.52 +- 1073.54 2718.26 +- 968.07 1912.09 +- 483.08 1226.60 +- 304.33
tstoragefile-seek-total-megabytes 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00 0.00 +- 0.00

CSCSkim

-- LeonardoSala - 04-Dec-2009

Attachments Attachments
Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r2 - 2009-12-04 - unknown
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback