TStore sync message

The sync message is part of the TStore Administrative SOAP Service Protocol. It makes sure that the table definitions in a view configuration match the tables in the database. There are three modes: Sync from database, sync to database, and both ways.

The abbreviation 'sync' was chosen to avoid conflict between British and American spellings of synchronise/synchronize. Sync was chosen over synch because it's shorter and snazzier.

Parameters

For all modes the synchronisation only affects tables matching a POSIX regular expression supplied in the pattern parameter. This way any unrelated tables will not be touched. The pattern parameter is matched against the database table names (which are uppercase... using Oracle's quoted identifiers to create case-sensitive object names is not recommended with TStore) and/or the uppercased version of the table names in the configuration. Therefore the pattern should match uppercase table names.

Perhaps the easiest way to make sure that only the appropriate tables are affected by a sync is to name your tables with a prefix depending on the table name. For example, to sync all tables whose names begin with MYVIEW_ you would use the regular expression ^MYVIEW_.* .

The Sync message also supports the test parameter. These parameters should both be in the TStore namespace (like the connection ID) since there is no view involved.

Sync to database

In this mode the tables in the configuration always take priority over what is in the database.

Table in configuration but not in database

The table will be created in the database, using the same column types used by addTable

Table in database but not in configuration

The table will be removed from the database.

Table both in configuration and in database

If the columns have different names or the datatypes are incompatible, TStore will back up the current table with a different name and create a new one matching the configuration.

Primary key in database different from that in the configuration

TStore will back up the current table with a different name and create a new one with primary key as in the configuration.

Foreign key constraints in database different from those in the configuration

TStore will back up the current table with a different name and create a new one with foreign key constraints as in the configuration.

Sync from database

In this mode the tables in the database always take priority over what is in the configuration. This is a bit trickier because there is less information, we don't know for sure which XData types to put in the XML. To find the right XData types, TStore will first check whether the columns have types and check constraints exactly (to the letter) matching those used by when TStore creates a table in the database. If so then TStore can determine with certainty the correct types. In cases where the same check constraint could apply to multiple types (e.g. either an UnsignedInteger on a 32 bit machine or an UnsignedInteger32) then the more explicit type is used.

If a column has no check constraint, or it has one which does not exactly match one of the known check constraints, TStore will use the default type mappings. Technically the default mappings do not need to be in the XML since they are the defaults anyway, but storing them is necessary if we want to later sync in the other direction.

Table in configuration but not in database

The table will be removed from the configuration

Table in database but not in configuration

The table will be added to the configuration using the types mentioned above.

Table both in configuration and in database

The configuration will be changed to have the same columns as the database.

Primary key in database different from those in the configuration

TStore will update the configuration with the primary key as read from the database.

Foreign key constraints in database different from that in the configuration

TStore will update the configuration with the foreign key constraints as read from the database.

Both ways

In this mode tables and columns in either the configuration or in the database are added in the other place. No data is removed.

Table in configuration but not in database

The table will be created in the database, using the same column types used by addTable

Table in database but not in configuration

The table will be added to the configuration using the same rules as in Sync from database mode.

Table both in configuration and in database

If the columns have different names or the datatypes are incompatible, TStore will back up the current table with a different name and create a new one. The new table will have all of the columns which were in either the configuration or in the database. If the columns exist in both places, but the column types are incompatible, an error will be returned.

Primary key in database different from that in the configuration

If the primary key in the configuration is blank, TStore will assume that the data is simply missing and will copy the primary key from the database to the configuration. Otherwise, an error will be returned. While it would be possible to devise a primary key based on the columns in the two conflicting keys, it would not be clear which order to put them in, and a primary key should be decided on by a human based on the requirements.

Foreign key constraints in database different from those in the configuration

Any foreign key constraints which exist in the database but not in the configuration will be added to the configuration. Any foreign key constraints which exist in the configuration but not in the database will be added to the database. (Note that unlike with tables and columns, there is no concept of a foreign key which exists in both places but is different. There could be many constraints referencing different columns of the same table, and these do not necessarily conflict.)

Type checking

There could be a parameter to determine what is meant by types in the configuration being compatible with types in the database. TStore could either return an error only when there is a definite conflict that will affect all values (a date column being treated as a number for example) or whenever there is a potential conflict for certain values (a number(30) for a short int for example, or any number column for an unsigned type.) Since Oracle types do not map exactly to XData types, in the second case the only way that TStore can be 100% sure a column is completely compatible with an XData type is if it has the exact check constraints that are on columns created by addTable. There may be other constraints which would lead to the same result but TStore is not a constraint-parsing-engine so these will have to be checked manually.

The parameter has not yet been implemented. For now the type checking is based on whether TStore's best guess at the XData data type for a column (based on check constraints, or if they don't exist, the default type for that datatype) is the same as, or has the same range of values as, the type in the configuration.

Table versioning

When a table is backed up and replaced, the backup version is given a name ending in an underscore followed by a three-digit version number. This number is one higher than the current highest version of that table found in the database. So if the sync mode dictates that a table named MYTABLE needs to be replaced, then it will be renamed to MYTABLE_001 and a new table will be created with the name MYTABLE . If the table needs to be changed again, then MYTABLE will be renamed to MYTABLE_002 and a new table will be created with the name MYTABLE .

Limitations

Not all characteristics of a table are stored in the view configuration file. Therefore, some information can be lost if you create a table in the database, then sync from the database, and then remove the table from the database and sync to the database to recreate the table. If the table was created by TStore, and you have not made any modifications to the table by hand, then the recreated table should be the same as the original. However, if you created or modified the table yourself, you might not get a table identical to what you had before.

The table will have the same column names, the same primary key columns and the same foreign key constraints. The columns will have the default column types and constraints for the xdata types deduced from the original table, even if the original table had slightly different data types. Any other constraints that were on the original table, or other properties of the table, will be lost. This includes triggers and indexes. If you add anything like this to a table which you intend to modify with TStore, then you should make sure you have a backup of the commands needed to add the required properties to a newly created table.

SOAP Message

<soap-env:Envelope soap-env:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:soap-env="http://schemas.xmlsoap.org/soap/envelope/">
   <soap-env:Header/>
   <soap-env:Body>
      <tstoresoap:sync tstoresoap:connectionID="connectionID" tstoresoap:mode="to database|from database|both ways" tstoresoap:pattern="pattern" 
      xmlns:tstoresoap="http://xdaq.web.cern.ch/xdaq/xsd/2006/tstore-10.xsd"/>
   </soap-env:Body>
</soap-env:Envelope>

SOAP Response

<soap-env:Envelope soap-env:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:soap-env="http://schemas.xmlsoap.org/soap/envelope/">
   <soap-env:Header/>
      <soap-env:Body>
         <tstoresoap:syncResponse xmlns:tstoresoap="http://xdaq.web.cern.ch/xdaq/xsd/2006/tstore-10.xsd">
            ...details of operations performed or simulated...
         <tstoresoap:syncResponse/>
      </soap-env:Body>
</soap-env:Envelope>

The details of operations performed are the same as for other administrative commands.

Code example

try {
	xoap::MessageReference msg = xoap::createMessage();
	xoap::SOAPEnvelope envelope = msg->getSOAPPart().getEnvelope();
	xoap::SOAPName msgName = envelope.createName( "sync", "tstoresoap", TSTORE_NS_URI);
	xoap::SOAPElement element = envelope.getBody().addBodyElement ( msgName );

	//Add connection ID which you obtained via a previous connect message on a view
	xoap::SOAPName connectionName = envelope.createName("connectionID", "tstoresoap", TSTORE_NS_URI);
	element.addAttribute(connectionName, connectionID); 

	//Add attribute to specify the sync mode. mode should be either "to database", "from database" or "both ways"
	xoap::SOAPName modeName = envelope.createName("mode", "tstoresoap", TSTORE_NS_URI);
	element.addAttribute(modeName, mode); 
	
	//Add attribute to specify the regular expression which the table names should match
	xoap::SOAPName patternName = envelope.createName("pattern", "tstoresoap", TSTORE_NS_URI);
	element.addAttribute(patternName, pattern_);
	
	xoap::MessageReference reply=getApplicationContext()->postSOAP(msg, tstoreDescriptor);
	xoap::SOAPBody body = reply->getSOAPPart().getEnvelope().getBody();
	if (body.hasFault()) {
		//do something about the fault
	}
} catch (xcept::Exception &e) {
	//handle the exception
}

-- VadimKhotilovich - 11-Aug-2011