dbCCP4i: Communication Protocols

Introduction

This document outlines the general principles for the communications between the handler and the client applications.

Basics

The communications between the handler and the clients are via sockets and are marked up in a form of pseudo-XML.

Communications from the client to the handler are requests; communications from the handler to the client are either responses to specific requests, or unsolicited broadcasts which do not correspond to any specific request.

The following sections describe the markup for each type of communication.

General principles

Each communication is wrapped in a tag that expresses the overall type of communication (request, response or broadcast). The following implementation details are generally applicable to all types of communication:

Communications are newline delimited This means that none of the tags or data can cannot unescaped newline characters. Note that although the examples below are shown with newlines between many of the tags, this is for clarity of presentation only.
Data items may contain non-newline whitespace characters
The special characters "<" and ">" within data should be escaped "<" should be translated to "<" and ">" should be translated to ">".

Requests

Requests are wrapped in the following markup:

<db_request>
<command>DbRequest</command>
<argument>arg1</argument>
<argument>arg2</argument>
...
<request_id>id</request_id>
</db_request>

DbRequest is the name of a command recognised by the handler (see separate documentation on the handler API). There should only be one command per request.
arg1, arg2 etc are the values of the arguments that are appropriate to the specific command. There is no limit to the number of arguments that can be specified. The argument values can either be single values, or lists of values (see Markup of the argument and result content).
id is an arbitrary string that can optionally be specified by the client; if it is then it will be included in the response from the handler.

Requests fall into one of two broad categories: either a request for data or a request for action. An example of a request for data is asking for the value of a particular database item; an example of a request for action is asking to modify the value of a database item.

Responses

Responses to requests are wrapped in the following general markup:

<db_response>
<status>Status</status>
<result>result</result>
<response_id></response_id>
</db_response>

Status indicates whether the handler was able to fufill the request from the client; it can take one of two possible values:
- OK: indicates that the handler was able to fufill the request
- ERROR: indicates that the handler was unable to fufill the request.
result is the result of the request returned by the handler. The result can either be a single value, or a list of values (see Markup of the argument and result content).
id is the identifying string supplied by the calling application with the original request.

The content of the result depends on the type of request and on the status returned by the handler:

Request type	Example request	Response status	Result content
Action	DbOpen	OK	Message describing the outcome
Action	DbOpen	ERROR	Message describing the error, with a suggestion for how to recover if appropriate
Data	DbGetData	OK	The requested data
Data	DbGetData	ERROR	Message describing the error, with a suggestion for how to recover if appropriate

Markup of the argument and result content

Within the arg and result tags, the structure of the content may be enforced by additional markup. The following structures are available:

Single value
List of values

Single value is appropriate when one item is being sent or returned, for example:

<db_request>
<command>DbRequest</command>
<argument>arg1</argument>
<request_id>id</request_id>
</db_request>

<db_response>
<status>Status</status>
<result>result string</result>
<response_id></response_id>
</db_response>

Lists of values are appropriate when a single argument of a request consists of several discrete items, or when multiple items are returned from a single request. Lists are used to provide a programming language-independent description of the data being passed:

<db_request>
<command>DbRequest</command>
<argument>
<list>
<item>item1</item>
<item>item2</item>
...
</list>
</argument>
<request_id>id</request_id>
</db_request>

<db_response>
<status>Status</status>
<result>
<list>
<item>item1</item>
<item>item2</item>
...
</list>
</result>
<response_id></response_id>
</db_response>

For responses, lists can also be nested within other lists:

<list>
<item>
<list><item>item1</item><item>item2</item> ...</list>
</item>
...
</list>

There is no limit on the level of nesting within responses, so it is possible to have lists inside lists inside lists and so on.

Note however that no nesting is currently permitted in the arguments of requests.

Broadcasts

Unsolicited broadcast messages from the handler to the client are wrapped in the following general markup:

<db_broadcast>
<message>Message text</message>
<operation>operation</operation>
<project>projectname</project>
<jobid>jobid</jobid>
<agent>agent</agent>
</db_broadcast>

Note that none of these items are considered compulsory and ordering is arbitrary. The general meanings of each element are as follows:

Element	Content
message	A free-text message describing the broadcast, which is intended to be human-readable e.g. for debugging purposes. The message text can change arbitrarily so it should not be used as the basis to trigger client responses to the broadcast.
operation	The name of the operation that triggered the broadcast (see below for the list of possible operation names that can appear here).
project	The name of the project that the operation relates to. This can be blank.
jobid	The job number in the project that the operation relates to. This can be blank.
agent	The name of the user agent that triggered the broadcast.

The possible operations are:

Operation	Description
Shutdown	Issued to all clients when the Shutdown request is received and executed by the handler.
DirectoriesReadonly	Issued to all clients when the project reference data in directories.def has been reloaded into the handler as readonly.
NewProject	Issued to all clients when a new project is created and a reference is added to directories.def. The project element contains the name of the new project.
DeleteProject	Issued to all clients when a project reference is removed from directories.def. The project element contains the name of the deleted project.
ImportProject	Issued to all clients when an existing project directory is (re)added to directories.def. The project element contains the name of the imported project.
AddDataDirRef	Issued to all clients when a default (data) directory reference is added to directories.def. The project element contains the name of the new data directory.
DeleteDataDirRef	Issued to all clients when a default (data) directory reference is removed from directories.def. The project element contains the name of the data directory that was removed.
DbReadonly	Issued to clients connected to a specific project, indicating that the project is now only accessible in read-only mode. The project element indicates which project is now read-only.
DbNewJob	Issued to clients connected to a specific project, indicating that a new job has been addded to the project. The project and jobid elements indicate the project and new job number.
DbDeleteJob	Issued to clients connected to a specific project, indicating that a job has been deleted from the project. The project and jobid elements indicate the project and the job number which was removed.
DbSetData	Issued to clients connected to a specific project, indicating that data associated with a job has been updated. The project and jobid elements indicate the project and job number that have been modified.
AddInputFile	Issued to clients connected to a specific project, indicating that a new input file has been associated with a job. The project and jobid elements indicate the project and job number that have been modified.
AddOuputFile	Issued to clients connected to a specific project, indicating that a new input file has been associated with a job. The project and jobid elements indicate the project and job number that have been modified.
AddSubJob	Issued to clients connected to a specific project, indicating that a new subjob has been created. The project and jobid elements indicate the project and new subjob number.

Note that some requests which should trigger broadcasts currently do not (specifically, many of those associated with modifying the SQLite database components).

Examples of requests, responses and broadcasts

Example requests and matching responses:

Request for status of the handler:

<db_request><command>DbRequestStatus</command><request_id>0</request_id></db_request>

Response:

<db_response><status>OK</status><result>Active</result><response_id>0</response_id></db_response>

Request for user to register with the handler:

<db_request><command>DbRegister</command><argument>wy45</argument><argument>15376</argument><argument>False</argument><request_id>request#1</request_id></db_request>

Response:

<db_response><status>OK</status><result>True</result><response_id>request#1</response_id></db_response>

Example of a broadcast message:

<db_broadcast><message>Update TEST 40</message><project>TEST</project><jobid>40</jobid><operation>data
changed</operation><agent>MrBump</agent></db_broadcast>

Limitations and issues

The following comments suggest future possibilities that are not currently implemented:

The possible status values for responses should probably be expanded to OK, FAILED and ERROR. FAILED would indicate where the handler rejected the request (for example not opening a project because it is locked); ERROR would indicate where the handler was unable to fufil the request (for example in the instance of runtime error in the database code). In the case of errors or failures the handler should return an error code, which can be defined in the handler API documentation.

Authors

Peter Briggs and Wanjuan Yang

Last commit: $Date: 2008/08/12 10:48:16 $