Script and Tools to copy, untar, move and rename large amounts of files.

Under the castor area:

/castor/cern.ch/cms/generation/tools

There are several helper scripts:

get_files.pl
mcopy_from_CASTOR.ksh
mcopy_to_CASTOR.ksh
multimv_lis.csh
multimv_ntp.csh
multitar.ksh

chmod u+x name.extention

Copy multiple files from CASTOR to any other machine specified.

Define the initial directory "incas", the final directory "outcas" (including the computer name) and a match string "match" in order to copy the files, identified by the match, in an efficient way. These parameters can be set in the script mcopy_from_CASTOR.ksh . The files not copied properly, due to a connection timed out or any other problem, are collected in the script "error.sh". The latter can be executed immediately to copy the files that failed to be copied before. If one has to copy hundreds of files from CASTOR, the script get_files.pl is better suited for the job. This script will stage the files first (if needed) in a packetsize the user can set her/him self (default 50). The other settings are identical to the previous script.

Untar multiple files.

Define the initial directory "incas" and a match string "match". Then execute the script multitar.ksh in the directory where you want the output.

Move and rename multiple files.

Submitting data for production, one typically ends up with ntuples (.ntpl) and data information (.lis) files distributed under different cmkin directories. In order to submit them, one moves the ntuples to one directory using the script multimv_ntp.csh and moves the data information (.lis) files to the same directory using the script multimv_lis.csh. The latter renames the .lis files to avoid identical names and hence information loss. Both scripts require one sets the number of files "NFILE" to be moved starting from file 1.

Copy multiple files to CASTOR from any other machine specified.

The initial directory "incas" and the final directory "outcas" should be defined in mcopy_to_CASTOR.ksh . The computer name has to be defined in the initial directory, but in order to login on that machine, one has to change my login 'polbrech' to your own login. All other settings are similar to those explained above.

-- Main.segoni - 08 Nov 2006