A proposal for a strategy to manage software installations in the LCG Applications Area afs space

Involved parties

The information channel in this matter from the Software Process and Infrastructure (SPI) side will be setup through the Architects Forum (AF) by means of the AF mailing list (project-lcg-af@cernNOSPAMPLEASE.ch) and AF meetings. So involved parties include

• LHC experiments software architects
• LCG Application Area project leaders
• Grid representatives
• LCG/AA work package managers

If information needs to be forwarded to a broader audience this will happen through the hubs above.

The current situation

The SPI project is responsible for installation of software components necessary for the development, maintenance, debugging, documentation, testing, building and running of LHC experiment framework software and applications. For this reason two afs trees were established, where users (developers, experiments, ...) can use all necessary components without restrictions. These two afs trees are

• /afs/cern.ch/sw/lcg/app/releases for the LCG/AA developed software projects (e.g. COOL, POOL, ROOT, etc.)
• /afs/cern.ch/sw/lcg/external for all other software components which are not developed in house (e.g. Python, Qt, doxygen, CMT, ...)

Over time the space used for these installations has grown, because of

• new software components being added
• new versions of a given component have been installed
• new platforms for an already installed version of a component have been added

A proposal on how to move ahead

The goal of this proposal is to decrease the space used for these afs installations to the only needed components

In the proposal below both the Application Area projects and "external" packages will be treated the same way.

Different stages of actions

The procedure for software cleanup consists of 2 consecutive steps:

1. Call for current usage
   1. SPI will contact all stake holders to retrieve the LCG configurations being currently used. Note this is very important, SPI will only ask for LCG Configuration numbers currently used not for individual packages or package versions
   2. From the union of all currently used LCG configurations SPI can compute the union of all software packages and their versions currently in use. With this information we can also compute the "packages/versions" to be removed.
   3. SPI will check with the help of afs access numbers if one of the candidates for removal has been accessed a "reasonable" amount of time. If this is the case this will be brought to the attention of the AF.
   4. Finally the package/versions which are candidates for removal will be appended a string "_to_be_removed_[month]_[year]" to the version string inside the package.
2. Clean and Backup
   1. One month before the actual removal SPI will contact again all stake holders with the list of packages/versions that will be removed
   2. Starting from the actual removal date the following actions will be executed per package/version
      1. A backup of the package - including all currently installed platforms - into CASTOR into a user space of a "generic" SPI user
      2. Remove the package/version from afs and eventually also remove the volume if this version is the only one or last one in the volume
      3. In case the package-version was the last one of a package, remove the package top level directory

How often this will be executed

The procedure proposed shall be executed twice a year, starting every

• December 1
• June 1

The procedure shall be implemented with a "pipelining" mechanism, i.e. at any execution day the "clean and backup" for all packages from the previous call shall be executed while a new call for "current usage" will be issued.

-- StefanRoiser - 03 Oct 2008