Grid Service Discovery Meeting - Copenhagen

Attendance:

• Laurence Field (EGEE)
• Steve Fisher (EGEE)
• Antony Wilson (EGEE)
• Arumugam Paventhan (EGEE)
• Markus Schulz (EGEE)
• Steven Timm (OSG)
• Shaowen Wang (OSG)
• Sergio Andreozzi (OMII)
• Weijian Fang (OMII)
• Oxana Smirnova (Nordugrid)
• Balazs Koyna (Nordugrid)
• Aleksei Nazarov (NorduGrid)
• Laura Pearlman (Teragrid)
• Ahda Iamhitchi (USF)
• Lydia Priefo (USF)

Overview:

The main purpose of this meeting was to discuss a common way to do Service Discovery across multiple grid infrastructures. It is of critical importance that the solution will scale to the future requirements of the expanding grid infrastructures. The meeting was split into three main sections; the past, the present and the future. Firstly, the existing systems used to discover services in the different grid infrastructures were described in order to understand the similarity's and differences between the exiting systems and to understand the pros and cons of each. Next a common method of service discovery was discussed and finally a road map of how to achieve the end result was devised. An agenda was provided as a guide for discussion.

Existing systems and use cases:

The follow existing systems were discuses and the pros and cons of each system was highlighted.

• MDS2
• Nordugrid use of MDS2
• BDII
• MDS4
• R-GMA
• Grimoires
• Service Discovery in glite,
• Naregi Cell Domain

One of the main outcomes of this discussion was the surprising similarities in each system. Most used an index which contained information of the site level interface. The system would then pull the information from the site level. On a conceptual point of view the site could be represented by a database and an interface. One of the main differences between the different systems is how grid level caching is handled. Some common use cases of the systems were walked through to demonstrate the kind of information that needed to be found and the frequency.

Service Discovery:

Before any details could be discussed, the concept of Service Discovery need to be agreed. This is essentially the question asked and the answer which is returned. It was agreed that the questions which could be asked are anything that is generic for all services and the answer is a handle. In fact there are two handles, the Service Access Point and the Information End Point.

A resource is seen as a properly as a service. A there is a many to many relationship between services and resources, therefore resources also need a unique identifier. Hence resource discovery can be seen as a reverse lookup.

The need for a common Service Discovery interface was discussed and it was agreed that this would be neede. There is on going work to define the API in the SAGA activity within OGF. A plugin specification is also defined to enable the API to be used with multiple systems. Similar plugins were developed as part of the OGF gin-info activity.

It was agreed that an information provider interface would be useful so that the developers of the underlaying systems, eg batch systems and storage systems can as maintain the provider. The interface should be simple. A command that produces XML on standard out would suffice. This interface can be used both at the provider and plugin level. It was suggested that we many want to set up a community repository for plugins. What information needs to be produced is defined by the schema and we must not forget about caching to protect against overloading the underlying resource.

The use case of finding information which is specific to a service type was discussed. The name given to this use case was "service query". In order for this to be achieved a schema needs to be defined for each service type. Static and Dynamic attributes were discussed. The conclusion is that all attributes can change. The main different is the frequency on what we expect them to change. If something is static we don't expect it to change more frequently than 6 hours, dynamic attributes would change more frequently. Schema design needs to take into consideration dynamic values so that systems can be made more efficient by only moving the dynamic values around with a high frequency.

Query Interface:

A discussion followed on the need for a common query interface. Three options were presented; extend the Service Discovery API, have a common generic query interface, or service specific APIs. It was a greed that extending the Service Discovery would not be an option as key value pairs can not be used to express the complex structure of some service types. Service specific API would be very sensitive to schema changes. The conclusion was that a common query interface would be required however, it was not clear what this should be or how we can agree on one.

Bootstrapping:

Investigating the existing systems showed that each grid has a top level aggregator. The end points of these agregators needs to be passed to the configuration of the Service API so that it can find these aggregators. One possibility is that the VO should know these as they have already negotiated with the infrastructure to gain access. Grid infrastructures should look at the VOs they support to understand which Grids they need to interoperte with and they can get the endpoints from the VO. An agreed format may need to be worked out.

Conclusion:

The meeting was very fruitful with many areas being covered. The main out come of the meetings were:

1. We agree that there is a need for a common way to do Service Discovery.
2. A Service Discovery API is needed and this work will be done in the SAGA working group within OGF.
3. This will need a generic description of the Service and this will be defined in the GLUE working group within OGF.
4. The gin-info group could help to provide the required plugins.
5. There is a need for Service specific schema. This will also be defined in the GLUE working group within OGF.
6. A common information provider interface would be helpful. It was decided that this would be just a command which returned XML. There is no need to make this an official standard but it would help if this recommendation was adopted so that information providers could be shared and possibly developed by the developers of the underlaying system which is being queried.
7. It would be a good idea to set up a community repository which can be used to share plugins etc.
8. In principle it was agree that a common query API would be required however it wasn't clear how this would be decided.

-- LaurenceField - 11 Jan 2007