T2TWDecX

December09 Exercise

The purposes of December '09 exercise are:

• Challenge the readiness of T2_TW_Taiwan site.
• Teach the twcms users to make good use of T2_TW_Taiwan resource.
• Optimize the resource sharing with tw-atlas users.

• Contact people: K.F Chen (NTU), C.M. Kuo (NCU), S.J. Zhou (AS), Yuan Chao(NTU), S.W. Li (NCU)
• ASGC T2 contact: H.L. Shih, G. Qing.

The Exercise Details

• STEP0: Tutorial and grid role test
• STEP1: MC production and data subscription
• STEP2: Analysis ntuple production
• STEP3: Test with tw-atlas users
• STEP4: Review & summary

Pre-exercise preparation

• Documentation web page: (Jason & Gang)
• Monitoring web page (Jason & Gang)
  • CPU / Disk storage Ganglia page:
  • Job status
  • Total number of queues available. (Jason & Gang)

• Service mailing list. (Jason & Gang)
• Tutorial materials
  • Twiki workbook: https://twiki.cern.ch/twiki/bin/view/CMS/WorkBookStartingGrid
  • Twiki CRAB howtos: https://twiki.cern.ch/twiki/bin/view/CMS/SWGuideCrabHowTo
  • YuanChaoT2TWUsage
  • T2 TW usage tutorial Please choose a slot from doodle --> Hold on Dec. 2.
  • Indico page of the tutorial.
  • ychao_20091201.pdf: Yuan CHAO's slides for T2 usage tutorial

Two-hour tutorial one week before the exercise (STEP0) with one-hour lecture plus one-hour hand-on session.

• Datasets: Official MC productions. (to be updated)

• Useful links:

The "starting page" Twiki: http://wiki.twgrid.org/apwiki/test?action=show

FTT Batch System Running Job Monitoring (running): https://osadm.grid.sinica.edu.tw/rrd/rrd.cgi/ftt/ftt_batch_running.html

FTT: https://osadm.grid.sinica.edu.tw/acctmon/?bat=f-ce01&key=month&num=14

ASGC Batch System Running Job Monitoring (pending): https://osadm.grid.sinica.edu.tw/rrd/rrd.cgi/batch/asgc_batch_pending.html

ASGC Batch System Running Job Monitoring (running): https://osadm.grid.sinica.edu.tw/rrd/rrd.cgi/batch/asgc_batch.html

CASTOR2 LSF Job Submission Monitoring https://osadm.grid.sinica.edu.tw/rrd/rrd.cgi/castor2/platform_lsf.html

ASGC CASTOR Atlas Job Scheduling Monitoring https://osadm.grid.sinica.edu.tw/rrd/rrd.cgi/castor2/c2_lsf_atlas.html

CASTOR2 Network Throughput - CMS https://osadm.grid.sinica.edu.tw/rrd/rrd.cgi/castor2/castor2_cms_throughput.html

CASTOR2 Network Throughput - Atlas https://osadm.grid.sinica.edu.tw/rrd/rrd.cgi/castor2/castor2_atlas_throughput.html

ASGC RRD Gateway (castor2): https://osadm.grid.sinica.edu.tw/rrd/rrd.cgi/castor2/

ASGC RRD Gateway (ftt): https://osadm.grid.sinica.edu.tw/rrd/rrd.cgi/ftt/

Mailing list for user meeting: http://lists.grid.sinica.edu.tw/mailman/listinfo/user-meeting

MC Production & Data subscription (Dec. week I)

• We can produce b' signal pythia/madgraph MC at this moment.
• Background processes?
• Output to FTT SE and publish in local DB

• Samples to be subscribed to T2_TW
  • 7 TeV official MC?

• Results:

• Problems & solutions:

Analysis ntuple production

run analysis code to make ntuples from the MC production of the first week or the subscribed samples. We will try output the results to either CERN castor, T2 SE and T3 SE. (STEP2) We need some people using non-twcms role during this test and submit the jobs first. As we have very very limited slots for non-twcms, the jobs would just stay in queue, we can give the pressure on the batch system in this way. These jobs should last for at least two days and the tw-cms users start from the 2nd day.

1) fill all the queues from a generic cms user (non tw-cms) member; cut in test from tw-cms / cut in test from tw-atlas user. 2) fill all the queues from a generic atlas user (non tw-atlas) member; cut in test from tw-cms / cut in test from tw-atlas user. 3) fill all the queues from a tw-cms user; cut in test from a generic atlas user / cut in test from tw-atlas user. 4) fill all the queues from a tw-atlas user; cut in test from a generic cms user / cut in test from tw-cms user.

Co-test with tw-atlas users

people repeat the same test with swapped stage-out SE. Those who use non-twcms role should swap the role with others. And we have tw-atlas people submit the jobs at the same time. (STEP3)

• Results:
• Problems we met:

Review and Summary

• How to improve the usage of T2_TW? (attached slides)
  • ychao_20091105.pdf: Test items and thoughts on how to improve the T2_TW usage experience

- One user meeting for summary and discussions - Few further tests for performance fine-tuning?

-- YuanChao - 12-Nov-2009