Description of CCP4i DB handler

Wanjuan Yang

January 5, 2007

Overview

CCP4i DB handler is a server application. It serves clients which need to access ccp4i database. Currently DB hanlder supports ccp4i database.def file and SQLite database.

Usage of DB handler

Components of DB handler:

DB handler includes modules dbccp4i.py, manager.py, DBcommand.py, ccp4i.py, dbapi_sqlite.py, schema.sql.

Running DB handler

First, you need to set environment variables.

set CCP4I_TOP point to CCP4i directory.
To do this, source ccp4.setup in the ccp4 installation. e.g. source /home/wy45/ccp4-6.0.2/include/ccp4.setup
Set DBCCP4I_TOP point to the dbccp4i directory. Also set GRAPHVIZ_BIN to point to the directory containing the Graphviz dot program (see http://www.graphviz.org) if you want to use the dbviewer application.
To do this, source dbccp4i.setup in $DBCCP4I_TOP/etc. e.g source /home/wy45/Bioxhit_db/dbccp4i/etc/dbccp4i.setup

There is a command line script 'dbccp4i' in $DBCCP4I_TOP/bin directory:

dbccp4i --- run the handler and the output is written in the file dbhandler.log in the directory .CCP4.
dbccp4i debug --- run the handler and the output is on the screen.

Alternatively, go to the directory $DBCCP4I_TOP/dbccp4i, type:

python dbccp4i.py

Once the DB handler is running, it is waiting for clients to connect to it. If there is no any client to connect it, it will stop automatically after 30 seconds.

Communication between db handler and client

Client sends requests to the db handler. DB handler receives requests, processes them and sends back a response to client. Apart from the response for specific request, db handler also sends broadcast message to clients if there is any update in the database. All the requests, responses and broadcasts must be in a pre-defined XML form. Detailed communication protocols are described in dbCCP4i: Communication Protocols

Requests and corresponding responses

This section lists all the requests which are accepted by db handler. A request consists of command and arguments. As mentioned above, requests are sent to the handler in the form of XML, but the requests listed here are without the XML, in the form:

command arg1 arg2...

Commands for interacting with the handler

DbRegister

DbRequestStatus

DbSupported

DbDisconnect

ShutDown

DbRegister username useragent [ broadcast_flag ]
Summary: Client registers with the handler. This is the first request that a client should send to db handler before sending any other requests.
Arguments	username	user name of the system
	useragent	The application name that connects to the handler
	broadcast_flag	True if client wishes handler to send broadcast message to it. False otherwise.

Response		Description
status	result	Description
OK	True or 'data locked, force to override, data loaded'	If everything ok, handler return True. If there is a lock in directories.def, handler will override the lock and send back message.
ERROR	error message	If something goes wrong, handler sends back error messages.

DbRequestStatus

Summary: Get the status of the handler

Arguments

None

Response		Description
status	result	Description
OK	Active	This should return 'Active'.
ERROR	error message	If something goes wrong, handler sends back error messages.

DbSupported dbtype

Summary: Check if handler supports the given db backend.

Arguments

dbtype

database backend, e.g def, SQLite

Response		Description
status	result	Description
OK	True or False	If the handler supports the specified database, it returns True. If the handler doesn't support the specified database, it returns 'False'.
ERROR	error message	If anything goes wrong, handler returns error message.

DbDisconnect
Summary: Disconnect to the handler. Handler sends 'True' to client for this request.
Arguments	None

ShutDown
Summary: Shutdown the handler. Handler doesn't send any response to client for this request.
Arguments	None

NewDb project dir
Summary: Create a new project in def backend.
Arguments	project	the name of the project
	dir	directory of the project

Response		Description
status	result	Description
OK	True	Handler returns True if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

DbOpen project [ -grablock/-readonly ]
Summary: Create a new project in def backend.
Arguments	project	the name of the project
	-grablock	Optional. If the database is locked, with this option, the lock will be overwritten.
	-readonly	Optional. If the database is locked, with this option, the database is readonly.

Response		Description
status	result	Description
OK	True	Handler returns True if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

DbNewJob project
Summary: Create a job in database.
Arguments	project	the name of the project

Response		Description
status	result	Description
OK	Job id	Handler returns job id if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

DbDeleteJob project jobid
Summary: Delete a job in database.
Arguments	project	the name of the project
	jobid	the id of the job to be deleted.

Response		Description
status	result	Description
OK	True	Handler returns True if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

DbSetData project jobid item value [ item2 value2 [...] ]
Summary: Set the value(s) of one or more data item(s) for a job in project.
Arguments	project	the name of the project
	jobid	the id of the job to be set data.
	item	item name of the job.
	value	value of the item.

Response		Description
status	result	Description
OK	True	Handler returns True if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

DbGetData project jobid item
Summary: Retrieve the values of one or more data item(s) from a job in a project.
Arguments	project	the name of the project
	jobid	the id of the job.
	item	item name of the job.

Response		Description
status	result	Description
OK	value(s) of the specified item(s)	Handler returns value(s) if this operation is successful. If retrieve mutiple items, a list of values is returned.
ERROR	error message	If this operation fails, handler sends back error messages.

Updatetime project jobid
Summary: Update the time of a job.
Arguments	project	the name of the project
	jobid	the id of the job.

Response		Description
status	result	Description
OK	True	Handler return True if this operation is successful.
ERROR	False or error message	If this operation fails, handler sends back 'False' or error messages.

DbItemExists project jobid item
Summary: Check that item exists for a job.
Arguments	project	the name of the project
	jobid	the id of the job.
	item	the item name of the job.

Response		Description
status	result	Description
OK	True	Handler return True if this operation is successful.
ERROR	False or error message	If this operation fails, handler sends back 'False' or error messages.

DbSelectJobs project item pattern
Summary: Retrieve a list of jobs based on some selection criterion.
Arguments	project	the name of the project
	item	the item name of the job.
	pattern	regular expression pattern.

Response		Description
status	result	Description
OK	a list of job ids.	Handler return a list of job ids where the values of the given item match the supplied regular expression.
ERROR	error message	If this operation fails, handler sends back error messages.

ListJobs project
Summary: List all the job ids of a project.
Arguments	project	the name of the project.

Response		Description
status	result	Description
OK	a list of job ids.	Handler returns a list of job ids if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

DbReturnJob project joblist itemlist format
Summary: Given a list of job ids, return a list of descriptions which are generated from the suppied lists of data items in itemlist and format.
Arguments	project	the name of the project
	joblist	a list of job ids to be retrived description.
	itemlist	a list of items to be retrived.
	format	the format in which to display the result - this is a list of integers (one integer per item specified in the itemlist) that specifies the field width (number of characters) to allocate to the corresponding datat item.

Response		Description
status	result	Description
OK	a list of formatted string populated with job data.	Handler returns a list of formatted string populated with job data.
ERROR	error message	If this operation fails, handler sends back error messages.

ListProjects
Summary: List all the project names which are recorded in directories.def.
Arguments	None

Response		Description
status	result	Description
OK	a list of project names.	Handler returns a list of project names.
ERROR	error message	If this operation fails, handler sends back error messages.

GetNProject
Summary: Get the total number of projects.
Arguments	None

Response		Description
status	result	Description
OK	the total number of projects	Handler returns the total number of projects.
ERROR	error message	If this operation fails, handler sends back error messages.

ImportProject alias path
Summary: Add a reference of a project (alias and path) in the directories.def file.
Arguments
	alias	The alias of the project.
	path	The directory of the project.

Response		Description
status	result	Description
OK	True	Handler returns True if this operation is successful.
ERROR	False or error message	If this operation fails, handler returns False or error messages.

DeleteProject project
Summary: Delete the reference of a project in directories.def.
Arguments
	project	The name of the project.

Response		Description
status	result	Description
OK	True	Handler returns True if this operation is successful.
ERROR	False or error message	If this operation fails, handler returns False or error messages.

AddDataDirRef dir_def path
Summary: Add a reference of a default directory in the directories.def file.
Arguments
	dir_def	The default directory name.
	path	The path of the default directory.

Response		Description
status	result	Description
OK	True	Handler returns True if this operation is successful.
ERROR	False or error message	If this operation fails, handler returns False or error messages.

DeleteDataDirRef dir_def
Summary: Delete the reference of a default directory in directories.def.
Arguments
	def_dir	The name of the default directory.

Response		Description
status	result	Description
OK	True	Handler returns True if this operation is successful.
ERROR	False or error message	If this operation fails, handler returns False or error messages.

GetProjectDir project
Summary: Get the directory name of the project.
Arguments
	project	The name of the project.

Response		Description
status	result	Description
OK	The directory name	Handler returns the directory name if this operation is successful.
ERROR	Empty string or error message	If this operation fails, handler returns empty string or error messages.

ListDataDirs
Summary: Return a list of the user's default directories.
Arguments
	None

Response		Description
status	result	Description
OK	a list of the user's default directories	Handler returns a list of the user's default directories if this operation is successful.
ERROR	error message	If this operation fails, handler returns error messages.

GetDataDir def_dir
Summary: Return the directory corresponding to a def dir name.
Arguments
	def_dir	default directory name.

Response		Description
status	result	Description
OK	directory	Handler returns directory name if this operation is successful.
ERROR	error message	If this operation fails, handler returns error messages.

GetDirectoriesData keyword alias [keyword alias ...]
Summary: Return multiple data items for many different aliases
Arguments
	One or more keyword-alias pairs.	A keyword can be 'ProjectDir' (fetches the corresponding project directory for the subsequent alias), 'ProjectDBDir' (the corresponding database directory) or 'DataDir' (the corresponding data directory).

Response		Description
status	result	Description
OK	directory	Handler returns a list with one value corresponding to each keyword-alias pair that was supplied. If a particular keyword-alias pair was invalid (e.g. alias doesn't exist, or keyword is not recognised) then the value will be an empty string.
ERROR	error message	This should only happen in exception circumstances e.g. handler internal error.

GetNJOB project
Summary: Return the last number of job in project.
Arguments
	project	The name of the project.

Response		Description
status	result	Description
OK	The last number of the job.	Handler returns the last number of the job if this operation is successful.
ERROR	error message	If this operation fails, handler returns error messages.

GetNextJobList project jobid
Summary: Return a list of job ids which are related to the given job.
Arguments
	project	The name of the project.
	jobid	The id of the job.

Response		Description
status	result	Description
OK	a list of job ids	Handler returns a list of job ids which are related to the given job if this operation is successful.
ERROR	error message	If this operation fails, handler returns error messages.

GetAllFileLinks project
Summary: Returns a list of job pairs. Each pair of jobs indicates these two jobs are related to each other.
Arguments
	project	The name of the project.

Response		Description
status	result	Description
OK	a list of job pairs	Handler returns a list of job pairs if this operation is successful.
ERROR	error message	If this operation fails, handler returns error messages.

GetFileLinks project joblist
Summary: Given the job list, for each job id in the job list, return a list of job pairs. Each pair of jobs indicates these two jobs are related to each other.
Arguments
	project	The name of the project.
	joblist	A list of job ids.

Response		Description
status	result	Description
OK	a list of job pairs	Handler returns a list of job pairs if this operation is successful.
ERROR	error message	If this operation fails, handler returns error messages.

DbGetListofRecords project joblist itemlist
Summary: Get a list of jobs' item value.
Arguments
	project	The name of the project.
	joblist	A list of job ids.
	itemlist	A list of item names.

Response		Description
status	result	Description
OK	a list of jobs' item value.	Handler returns a list of jobs' item value if this operation is successful.
ERROR	error message	If this operation fails, handler returns error messages.

GetAllChildren project jobid
Summary: For a particular project, given a job id, get all the jobs which are children(all levels) of this job.
Arguments
	project	The name of the project.
	jobid	job id

Response		Description
status	result	Description
OK	a list of job ids.	Handler returns a list of job ids if this operation is successful.
ERROR	error message	If this operation fails, handler returns error messages.

GetAllParents project jobid
Summary: For a particular project, given a job id, get all the jobs which are parent (all levels) of this job.
Arguments
	project	The name of the project.
	jobid	job id

Response		Description
status	result	Description
OK	a list of job ids.	Handler returns a list of job ids if this operation is successful.
ERROR	error message	If this operation fails, handler returns error messages.

GetChildren project jobid
Summary: For a particular project, given a job id, get all the jobs which are children of this job.
Arguments
	project	The name of the project.
	jobid	job id

Response		Description
status	result	Description
OK	a list of job ids.	Handler returns a list of job ids if this operation is successful.
ERROR	error message	If this operation fails, handler returns error messages.

GetParents project jobid
Summary: For a particular project, given a job id, get all the jobs which are parent of this job.
Arguments
	project	The name of the project.
	jobid	job id

Response		Description
status	result	Description
OK	a list of job ids.	Handler returns a list of job ids if this operation is successful.
ERROR	error message	If this operation fails, handler returns error messages.

GetAllParentsChildren project jobid
Summary: For a particular project, given a job id, get all the jobs which are related to the job, including all parents and children.
Arguments
	project	The name of the project.
	jobid	job id

Response		Description
status	result	Description
OK	a list of job ids.	Handler returns a list of job ids if this operation is successful.
ERROR	error message	If this operation fails, handler returns error messages.

GetNotebook project jobid
Summary: Return notebook name with full path for a given job.
Arguments
	project	The name of the project.
	jobid	job id

Response		Description
status	result	Description
OK	notebook name with full path	Handler returns notebook name with full path if this operation is successful.
ERROR	error message	If this operation fails, handler returns error messages.

GetFiles project jobid
Summary: Return input file or output file list for a given job.
Arguments
	project	The name of the project.
	jobid	job id
	tyoe	file type

Response		Description
status	result	Description
OK	a list of files	Handler returns a list of files if this operation is successful.
ERROR	error message	If this operation fails, handler returns error messages.

AddFile project jobid type
Summary: .
Arguments
	project	The name of the project.
	jobid	job id
	type	file type

Response		Description
status	result	Description
OK	True	Handler returns True if this operation is successful.
ERROR	False or error message	If this operation fails, handler return False or error messages.

DbClose project
Summary: Close a project.
Arguments	project	the name of the project

Response		Description
status	result	Description
OK	True	Handler returns True if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

SetJobQuality project jobid quality
Summary: Set the value of 'JobQuality' in 'Jobs' table
Arguments	project	the name of the project
	jobid	Job id
	quality	The quality of the job

Response		Description
status	result	Description
OK	True	Handler returns True if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

SetSQLdbData project jobid item value
Summary: Set data in 'Jobs' table.
Arguments
	project	the name of the project
	jobid	Job id
	item	The attribute of the table.
	value	The value of the attribute.

Response		Description
status	result	Description
OK	True	Handler returns True if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

GetSQLdbData project jobid item [item1 item2...]
Summary: Get data in 'Jobs' table.
Arguments
	project	the name of the project
	jobid	Job id
	item	The attribute of the table.

Response		Description
status	result	Description
OK	a list of value(s)	Handler returns a list of value(s) if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

GetAllSQLdbData project item [item1 item2...]
Summary: Get all data in 'Jobs' table.
Arguments
	project	the name of the project
	item	The attribute of the table.

Response		Description
status	result	Description
OK	a list of value(s)	Handler returns a list of value(s) if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

NewTableRecord table item value [item2 value2 ...]
Summary: Create a new record in given table.
Arguments
	table	the table name
	item	the attribute of the table
	value	the value of the attribute

Response		Description
status	result	Description
OK	record id	Handler returns a record id if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

DeleteTableRecord table recordId
Summary: Remove a record in given table.
Arguments
	table	the table name
	recordId	the id of the record

Response		Description
status	result	Description
OK	True	Handler returns True if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

DeleteTableRecords table condition
Summary: Remove records in given table based on a certain conditions.
Arguments
	table	the table name
	condition	a SQL 'where' statement

Response		Description
status	result	Description
OK	True	Handler returns True if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

SetTableData table recordId attribute value
Summary: Set value for a table record.
Arguments
	table	the table name
	recordId	record id.
	attribute	the attribute of the item.
	value	the value of the attribute

Response		Description
status	result	Description
OK	True	Handler returns True if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

GetTableData table recordId attribute
Summary: Get value for a table record.
Arguments
	table	the table name
	recordId	record id.
	attribute	the attribute of the item.

Response		Description
status	result	Description
OK	value	Handler returns the value if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

GetTablePrimaryKey table condition
Summary: Retrieve primary key(s) in a given table based on a certain conditions.
Arguments
	table	the table name
	condition	a SQL 'where' statement

Response		Description
status	result	Description
OK	record id(s)	Handler returns record id(s)/primary key(s) if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

GetAllTableRecords projectname table itemlist
Summary: Retrieve all the records in a given table
Arguments
	projectname	the project name.
	table	the table name
	itemlist	The list of items.

Response		Description
status	result	Description
OK	a list of records	Handler returns a list of records if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

GetTableRecords projectname table itemlist condition
Summary: Retrieve records in a given table based on a certain conditions.
Arguments
	projectname	the project name.
	table	the table name
	itemlist	The list of items.
	condition	a SQL 'where' statement.

Response		Description
status	result	Description
OK	a list of records	Handler returns a list of records if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

NewDB dbname dir
Summary: Create a new database in SQLite.
Arguments	dbname	the name of the database
	dir	the directory of the database

Response		Description
status	result	Description
OK	True	Handler returns True if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

OpenDB dbname dir
Summary: Open an existing database in SQLite.
Arguments	dbname	the name of the database
	dir	the directory of the database

Response		Description
status	result	Description
OK	True	Handler returns True if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

GetJobData jobid item [ item2 ...]
Summary: Return the value(s) of one or more items in a job.
Arguments	jobid	the id of job.
	item	item name of the job. Could specify one or more item names.

Response		Description
status	result	Description
OK	a list of item value(s)	Handler returns a list of item value(s) if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

GetProjectId projectname
Summary: Given a project name, return project id.
Arguments	projectname	the name of the project.

Response		Description
status	result	Description
OK	project id	Handler returns project id if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

GetData table id attribute [ attribute1 ...]
Summary: Given a table name and id of the record and attribute(s),, return the value of the attribute(s).
Arguments	table	the table name.
	id	the id of the record
	attribute	attribute of the table, could specify one or more attributes.

Response		Description
status	result	Description
OK	a list of value(s)	Handler returns a list of value(s) if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

GetProjectList
Summary: Return a list of projects in the database.
Arguments	None

Response		Description
status	result	Description
OK	a list of project names.	Handler returns a list of project names if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

GetJobList project
Summary: Return a list of job records for a given proejct.
Arguments	project	the name of project.

Response		Description
status	result	Description
OK	a list of job records.	Handler returns a list of jobrecords if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

GetRowofTable table
Summary: Return a row of a table..
Arguments	id	the id of record.

Response		Description
status	result	Description
OK	record	Handler returns a record of a table if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

NewProject projectname
Summary: Create a new project.
Arguments	projectname	the project name

Response		Description
status	result	Description
OK	project id	Handler returns a project id if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

NewJob projectname
Summary: Create a new job in a project.
Arguments	projectname	the project name

Response		Description
status	result	Description
OK	job id	Handler returns a job id if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

NewRecord table
Summary: Create a new record in table.
Arguments	table	the table name

Response		Description
status	result	Description
OK	record id	Handler returns a record id if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

SetJobData jobid item value [item1 value1 ... ]
Summary: Set item/attribute value for a given job.
Arguments
	jobid	the job id
	item	the item/attribute of job. Could set one or more item/value at a time.
	value	the value of the item

Response		Description
status	result	Description
OK	True	Handler returns True if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

SetData table recordid attribute value [ attribute1 value1 ...]
Summary: Set one or more item/attribute value for a table record.
Arguments
	table	the table name
	recordid	record id.
	attribute	the attribute of the item, could set one or more pairs of attribute/value.
	value	the value of the attribute

Response		Description
status	result	Description
OK	True	Handler returns True if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

CloseDB
Summary: Close database in SQLite.
Arguments	None

Response		Description
status	result	Description
OK	True	Handler returns True if this operation is successful.
ERROR	error message	If this operation fails, handler sends back error messages.

Last Update: $Date: 2008/08/12 10:48:16 $

Description of CCP4i DB handler

Overview

Usage of DB handler

Components of DB handler:

Running DB handler

Communication between db handler and client

Requests and corresponding responses

Commands for interacting with the handler

DbRegister DbRequestStatus DbSupported DbDisconnect ShutDown

Commands for accessing to database.def backend

Commands for accessing to Parallel SQLite backend

Commands for accessing to Central SQLite backend

DbRegister

DbRequestStatus

DbSupported

DbDisconnect

ShutDown