Files not correctly registered

Several possible cases:

The normal case (as a reference)

A file should be registered in LFC and BKK (except user data). How to check: Let's start from a file on the storage:

$ lcg-ls -l lfn:/grid/lhcb/MC/MC10/ALLSTREAMS.DST/00009117/0000/00009117_00000686_1.allstreams.dst
-rw-rw-r--   1 19496  2695 2118842480 lfn:/grid/lhcb/MC/MC10/ALLSTREAMS.DST/00009117/0000/00009117_00000686_1.allstreams.dst

Check if it's in the LFC:

$ lfc-ls /grid/lhcb/MC/MC10/ALLSTREAMS.DST/00009117/0000/00009117_00000686_1.allstreams.dst
/grid/lhcb/MC/MC10/ALLSTREAMS.DST/00009117/0000/00009117_00000686_1.allstreams.dst
$ lfc-ls -l /grid/lhcb/MC/MC10/ALLSTREAMS.DST/00009117/0000/00009117_00000686_1.allstreams.dst
-rw-rw-r--   1 19496    2695             2118842480 Jan 21 09:29 /grid/lhcb/MC/MC10/ALLSTREAMS.DST/00009117/0000/00009117_00000686_1.allstreams.dst

then, check its replicas:

[lxplus409] /afs/cern.ch/user/l/lanciott/my-dirac-scripts > lcg-lr lfn:/grid/lhcb/MC/MC10/ALLSTREAMS.DST/00009117/0000/00009117_00000686_1.allstreams.dst
srm://gridka-dCache.fzk.de/pnfs/gridka.de/lhcb/MC/MC10/ALLSTREAMS.DST/00009117/0000/00009117_00000686_1.allstreams.dst
srm://srm-lhcb.cern.ch/castor/cern.ch/grid/lhcb/MC/MC10/ALLSTREAMS.DST/00009117/0000/00009117_00000686_1.allstreams.dst
srm://srm-lhcb.gridpp.rl.ac.uk/castor/ads.rl.ac.uk/prod/lhcb/MC/MC10/ALLSTREAMS.DST/00009117/0000/00009117_00000686_1.allstreams.dst
srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/lhcb/MC/MC10/ALLSTREAMS.DST/00009117/0000/00009117_00000686_1.allstreams.dst

issue a lcg-ls -l of a replica's SURL to see if the file is ONLINE:

[lxplus409] /afs/cern.ch/user/l/lanciott/my-dirac-scripts > lcg-ls -l srm://gridka-dCache.fzk.de/pnfs/gridka.de/lhcb/MC/MC10/ALLSTREAMS.DST/00009117/0000/00009117_00000686_1.allstreams.dst
-rw-r--r--   1     2     2 2118842480  ONLINE_AND_NEARLINE /pnfs/gridka.de/lhcb/MC/MC10/ALLSTREAMS.DST/00009117/0000/00009117_00000686_1.allstreams.dst
        * Checksum: 0d293078 (adler32)
        * Space tokens: 7933987
[lxplus409] /afs/cern.ch/user/l/lanciott/my-dirac-scripts > lcg-ls -l srm://srm-lhcb.gridpp.rl.ac.uk/castor/ads.rl.ac.uk/prod/lhcb/MC/MC10/ALLSTREAMS.DST/00009117/0000/00009117_00000686_1.allstreams.dst
-rw-r--r--   1     2     2 2118842480  ONLINE_AND_NEARLINE /castor/ads.rl.ac.uk/prod/lhcb/MC/MC10/ALLSTREAMS.DST/00009117/0000/00009117_00000686_1.allstreams.dst
        * Checksum: 0d293078 (ADLER32)

as you see, the space token is optional (not always returned).

File not registered in LFC

File registered in LFC, but no replica exists (very rare case!)

Files registered in the LFC but no replica is in the storage

Spotted on Jan 2011

The file is on the LFC:

[lxplus416] /afs/cern.ch/user/l/lanciott > lfc-ls -l /grid/lhcb/data/2010/EW.DST/00008380/0000/00008380_00000023_1.ew.dst
-rw-rw-r--   1 18950    2695              778892282 Dec 03 12:57 /grid/lhcb/data/2010/EW.DST/00008380/0000/00008380_00000023_1.ew.dst

but it has NO replica!

lcg-lr does return a list of two replicas, but lcg-ls cannot give the details for the problematic file:

[lxplus416] $ > lcg-lr srm://ccsrm.in2p3.fr/pnfs/in2p3.fr/data/lhcb/data/2010/EW.DST/00008380/0000/00008380_00000023_1.ew.dst 
srm://ccsrm.in2p3.fr/pnfs/in2p3.fr/data/lhcb/data/2010/EW.DST/00008380/0000/00008380_00000023_1.ew.dst
srm://srm-lhcb.cern.ch/castor/cern.ch/grid/lhcb/data/2010/EW.DST/00008380/0000/00008380_00000023_1.ew.dst
[lxplus416] $ > lcg-ls -l srm://ccsrm.in2p3.fr/pnfs/in2p3.fr/data/lhcb/data/2010/EW.DST/00008380/0000/00008380_00000023_1.ew.dst 
/pnfs/in2p3.fr/data/lhcb/data/2010/EW.DST/00008380/0000/00008380_00000023_1.ew.dst: [SE][Ls][SRM_INVALID_PATH] could not get storage info by path: No such file or directory ///pnfs/in2p3.fr/data/lhcb/data/2010/EW.DST/00008380/0000/00008380_00000023_1.ew.dst

dirac-dms-lfn-replicas lists the file as successful, but can't get any replicas on the SE:

[lxplus416] $ > dirac-dms-lfn-replicas /lhcb/data/2010/EW.DST/00008380/0000/00008380_00000023_1.ew.dst 
2011-02-01 10:35:30 UTC dirac-dms-lfn-replicas/DiracAPI  INFO: Replica Lookup Time: 0.31 seconds
{'Failed': {},
 'Successful': {'/lhcb/data/2010/EW.DST/00008380/0000/00008380_00000023_1.ew.dst': {}}}

for a normal file it should return the SE and the corresponding SURL:

[lxplus416] $ > dirac-dms-lfn-replicas /lhcb/data/2010/EW.DST/00008380/0000/00008380_00000024_1.ew.dst 
2011-02-01 10:35:43 UTC dirac-dms-lfn-replicas/DiracAPI  INFO: Replica Lookup Time: 0.25 seconds
{'Failed': {},
 'Successful': {'/lhcb/data/2010/EW.DST/00008380/0000/00008380_00000024_1.ew.dst': {'CERN_M-DST': 'srm://srm-lhcb.cern.ch/castor/cern.ch/grid/lhcb/data/2010/EW.DST/00008380/0000/00008380_00000024_1.ew.dst',
                                                                                    'CNAF-DST': 'srm://storm-fe-lhcb.cr.cnaf.infn.it/t0d1/lhcb/data/2010/EW.DST/00008380/0000/00008380_00000024_1.ew.dst',
                                                                                    'IN2P3_M-DST': 'srm://ccsrm.in2p3.fr/pnfs/in2p3.fr/data/lhcb/data/2010/EW.DST/00008380/0000/00008380_00000024_1.ew.dst',
                                                                                    'RAL-DST': 'srm://srm-lhcb.gridpp.rl.ac.uk/castor/ads.rl.ac.uk/prod/lhcb/data/2010/EW.DST/00008380/0000/00008380_00000024_1.ew.dst'}}}

Marked with P

They are marked with P (maybe problematic) in the LFC. How??? to be understood.

The DataIntegrityDB

They are stored into the DataIntegrityDB, on volhcb23:

mysql> select * from Problematics where LFN like '%/lhcb/data/2010/EW.DST/00008380/0000/00008380_00000023_1.ew.dst';
+--------+------------+-----------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------+------+-------------+------+--------+---------+---------------------+---------------------+---------------------+
| FileID | Prognosis  | LFN                                                             | PFN                                                                                                       | Size | SE          | GUID | Status | Retries | InsertDate          | LastUpdate          | Source              |
+--------+------------+-----------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------+------+-------------+------+--------+---------+---------------------+---------------------+---------------------+
| 483232 | PFNMissing | /lhcb/data/2010/EW.DST/00008380/0000/00008380_00000023_1.ew.dst | srm://srm-lhcb.cern.ch/castor/cern.ch/grid/lhcb/data/2010/EW.DST/00008380/0000/00008380_00000023_1.ew.dst | NULL | CERN_M-DST  | NULL | New    |       0 | 2011-01-31 15:00:10 | 2011-01-31 15:00:10 | DataIntegrityClient |
| 483233 | PFNMissing | /lhcb/data/2010/EW.DST/00008380/0000/00008380_00000023_1.ew.dst | srm://ccsrm.in2p3.fr/pnfs/in2p3.fr/data/lhcb/data/2010/EW.DST/00008380/0000/00008380_00000023_1.ew.dst    | NULL | IN2P3_M-DST | NULL | New    |       0 | 2011-01-31 15:00:11 | 2011-01-31 15:00:11 | DataIntegrityClient |
+--------+------------+-----------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------+------+-------------+------+--------+---------+---------------------+---------------------+---------------------+
2 rows in set (0.00 sec)

Etc..

FAQ

Is it possible that a file does not register in the BKK?

No. Unless it is a user file. All files coming from production/reconstruction jobs are automatically registered and they are never removed from the Bkk DB. They might have the replica flag = No. How to check it:

-- ElisaLanciotti - 11-Feb-2011