These utilities are mostly used in the run scripts which run the programs and are in the directory $CCP4I_TOP/scripts
Argument list: <deffilein> <args> To run a job a ccp4ish process is started. This is tclsh (with no Tk so no graphics) and some of the basic utilities from src/system.tcl and src/utils.tcl. The bin/ccp4ish.tcl script calls ExecuteScript which reads the def file for the job and then sources the appropriate script script/taskname.script. Note that the script is sourced and is not called as a separate procedure. Because the run script is effectively part of the ExecuteScript procedure it can see the same variables. There is a potential danger in this: that the variables in the procedure and those in the run script might get mixed up To avoid this some variables are unset before sourcing the run script.
ExecuteScript reads the def file and sets the parameters listed in the file to the values given in the file. The parameters from the header section of the def file are loaded into a global array job_params which can be accessed by other utility procedures to get information such as the project name or job id.
deffilein Name of the def file
Argument list: None Argument list: <args> The usual form of the returned file name is:
project_dir/project_jobid
-tmp
Give file name for file in the temporary directory
-map
Give file name for file in the user's choice of output directory for maps this is either the temporary directory or the project directory
Argument list: <command> <script> <statusout> <reportout> <args>
-success
Give exit code of command script that corresponds to success. This defaults to 0. There is a special value "999" which means don't trap for failure.
Argument list: None This function should be invoked by a running job. It determines whether it is necessary to reset the database connection, and then attempts to do so if required. This should only be necessary for jobs that are being run on a different machine to that running the parent CCP4i process.
Argument list: <command> <script> command The command line to run the program
script The name of the script file
Argument list: <status> <args>
Argument list: None Argument list: <status> <report> <> status Termination status: 1=finished, 0=failed
report (Optional) Termination report - usually output from last run program
Argument list: <textline> <args> textline The text to write to the log file
-noheader
Do not write the header of the wrapper
-nofooter
Do not write the footer of the wrapper
Argument list: <mode> <name> <args> mode beg/end Create the opening or closing tag
name Name of tag
Argument list: <arrayname> <data> <ncol> <nrow> <title> <clabelin> <\> This is used for the scaleit summary file which is more of a 'crossword' table and not suitable for the usual loggraph format.
arrayname Name of data array
data Root name of array elements containing the data e.g. array(data,i,j)
ncol Number of columns
nrow Number of rows
title Title for table
clabelin List of column labels
rlabelin List of row labels
wrlabel Maximum allowed length of row label (as number of characters)
wcol Width of each column (as number of characters)
dformat Data format (suitable for Tcl format command). This can either be one value which is applied to all columns or a list with one value per column.
textVar Output. text string containing the table.
The command scripts are written to a temporary file with a name $CCP4_SCR/project_jobid_n_com.tmp where n is incremented for each temporary file create for the job. This file can be written using WriteComFile or created from a template using CreateComScript.
CreateComScript has several support procedures which handle stacks and which act as mediators between the different levels in the procedure stack.
Argument list: <template> <fileVar> <args> Beware - this is probably the most complex bit of code in ccp4i! It is not always robust when the input com file is misformatted. To find the error in the com file use the -diag option to list the lines from the com file as they are processed.
The main problem here is to access the parameters which are defined in the context of the run script (which is the calling procedure for this procedure). The parameters should not be visible in the context of this procedure unless they are explicitly passed to the procedure. The trick here is to use the Tcl uplevel command which evaluates an expression at a different level in the procedure stack. The next problem is then to return a status flag to this procedure so that it knows the outcome of any given function. his is achieved by using the the set_control_status procedure to set a global parameter and then using get_control_status to read that global parameter.
The syntax of the com file (program command template) is defined The procedure interprets the com file on a line by line basis. If there is an error in interpreting any feature of a line then nothing from the line will be written to the output command script. This interpreter will handle references to undefined variables but the entire line containing the undefined variable will be ignored.
There are two types of syntax in the com file - the interpretable lines and the control structures. On reading a new line the interpreter first tests to see if contains any recognised control statement - and it not then it attempts to interpret the line.
Note that there is a problem with the AT command in certain circumstances, which is recorded in the Programmers Docs.
-diag
Diagnostic. Print out the lines of the com file as they are processed
Argument list: <level> <var> <value> level Level in procedure calling stack
var Variable name
value Value to set variable
Argument list: <in> in The name of the file
Argument list: None Argument list: <command> <labellist> <args> command 'LABIN' or 'LABOUT' or continuation character ('-' or '&')
labellist Program label list
Argument list: <input> <nlmax> <nlVar> <\> The returned level is 0 for the control level and 1 for command line level and >1 for every nesting of continuation lines
input the text of the com file
nlmax Maximum number of lines in com file
nlVar in/output The current line number
levelVar output The nesting level of the current line
nelementsVar output The number of words in the line
commandVar output The control keyword if level=0
In this mode ccp4ish will repeat a range of jobs in one project and put newly created files, and a new project database, in a sub-directory of the original project directory.
Argument list: <work_subdir> <project> work_subdir The name of the sub-directory of project directory to contain test output
project The name of the project
Argument list: <project> <work_subdir> <deffile> <taskname> <job_id> project The name of the original master project
work_subdir The name of the working test directory
deffile The original def file
taskname The name of the task
job_id The job id
Argument list: <command> <args> Send a communication to the process controlling access to the job database, to update the job status.
Available commands are UpdateStatus and AddOutput file:
UpdateStatus
AddOutputFile
The communication is sent via a socket that is closed immediately afterwards i.e. the connection is not persistent.
command The request to be executed (UpdateStatus or AddOutputFile)
args Arguments specific to the command
Argument list: <command> <args> Send a communication to the process controlling access to the job database, to update the job status.
Available commands are UpdateStatus and AddOutput file:
UpdateStatus
AddOutputFile
The communication is sent via a socket that is closed immediately afterwards i.e. the connection is not persistent.
command The request to be executed (UpdateStatus or AddOutputFile)
args Arguments specific to the command
Argument list: <command> <script> <edit_returnVar> This opens a socket connection to the main CCP4i process and passes the command script to be inspected and edited by the user via CCP4i's EditComFile procedure.
The procedure returns 1 if the request successfully returned a result from the main CCP4i process, and 0 otherwise. On success the variable in the calling procedure that is referenced by edit_returnVar will contain the result.
command The command line to be executed
script The script to be executed
edit_returnVar Name of a variable to return the result in
Argument list: None This opens a socket connection to the main CCP4i process to trigger the RunAborted command, which performs clean up of the job.
Argument list: <status> This opens a socket connection to the main CCP4i process to trigger the RunCompleted command, which performs post-run operations such as running any review scripts.
status The final status of the job on completion e.g. FINISHED
Argument list: None
Argument list: None
Argument list: <args> Silently register an output directory for a job - the directory will not appear in the list of output files seen by the user, but will be removed by any subsequent clean-up
This is just a wrapper for AddOutputFile
Note that PROJECT is not a valid argument to AddOutputDir, and will be ignored.
args list of directories to register
Argument list: <args>
Most CCP4i scripts to run programs are written in Tcl - this information is for use with those scripts which are not Tcl. Normally the non-Tcl script will have been started from a standard Tcl script called taskname.script. It would be possible to leave this process running and return the name of output files to this script when the main non-Tcl script completes. Alternatively the Tcl process can be terminated and the necessary communication with the main CCP4i process done from the non-Tcl script. Ideally the communication functionality would be provided by a library in the appropriate scripting language but, failing that, the non-Tcl script starts a Tcl/CCP4i process and passes it the appropriate parameters to do the communication.
The command from the non-Tcl script must be a Unix shell type command:
$CCP4I_TOP/bin/ccp4ish -socket $CCP4I_DEFFILE ccp4i_command
where $CCP4I_TOP is the standard CCP4i environment variable
$CCP4I_DEFFILE is the name of the CCP4i def file for this job. This file is read by the ccp4ish process and contains the necessary information about sockets and job ids for communication with the main CCP4i process. This file is in the user's project database directory i.e.
project_directory/CCP4_DATABASE/jobid_taskname.def
it is easiest if the full path name for this file is passed into the non-Tcl script as an argument. The path name can be defined in the Tcl script as
set deffile [FileJoin [GetDbDirPath] $job_params(JOB_ID)_$job_params(TASKNAME).def ]
ccp4i_command is the command which the ccp4ish process is required to execute. The currently supported commands are AddOutputFile and TerminateScript which are described in the CCP4i Programmers Documentation.
Argument list: <args> This procedure is called from ccp4ish.tcl and is used when the main work is been done by a non-Tcl script (eg arp/warp in csh) and this script needs to communicate status & output file info.
-deffile deffile
The name of the def file for the job - the header contains info on the server socket
-project project_dir
The name of the current project
-command command
Text string with the command to send to the main CCP4i - must begin with the name of an allowed command: RunNotification, FailNotification, AddOutputFile, AddOutputDir or TerminateScript
-silent
Suppress echoing of the commands and warnings to stdout
Argument list: <project> <command> <deffile> Writes the command to a command buffer file for the appropriate project. BufferSocketCommand should be called if a socket cannot be opened to the process controlling access to the database to execute the command immediately. The contents of the buffer can be flushed by DbFlushCommandBuffer, which will execute the commands via ExecuteSocket at that point.
project Project alias for the database to be communicated with
command Socket command to be buffered
deffile (optional) Def file name associated with the job sending the socket command
Argument list: <project> Clone of DbGetBufferFile in database.tcl
project Project alias for the database to be communicated with
Argument list: <args> Return 1 if socket was successfully opened, 0 if not.
-silent
Do not report failure to open socket
Argument list: None The server port number is stored in the database lock file for the current project. If the port number cannot be extracted then the number stored in the job_params array will be returned. If this value cannot be extracted then an empty string is returned.
These commands are utilities which are associated with running jobs but which need to be shared between different parts of CCP4i that otherwise do not have much in common.ExecuteScript Read the def file and source the run script.
GetJobid Return the job id
SetOutputFileRoot Return a root file name which identifies the job that created the file
Execute Execute a command and handle editting of command script and program failure.
ResetDatabaseAccess Reset the database access mode for running jobs
AppendCommand Append the command line as a comment to input script
TerminateScript Cleanly terminate a run script process and notify the main CCP4i process
InitialiseLog Write ccp4i header to the log file for the job
WriteTerminationToLog Write ccp4i footer to the log file
WriteToLog Write comments to log file with wrappers to indicate that comments are from ccp4i script.
WriteHtml Write an html tag to log file
WriteTable Write a neatly formatted table to log file or other summary file
Creating the Program Command Script
CreateComScript Create the program command script using a template.
UpsetVar Set a variable at a higher level in the procedure calling stack
set_at_filename Handle the AT command in a com file - save the name of a file to be sourced
get_at_filename Return the name of the 'AT' file
set_label_stack Handling the LABELLINE keyword for MTZ LABIN/LABOUT
getcomline Read the next line from the 'com' template file
Automatic Test Mode
ExecuteAutoTest Execute a script in autotest mode
UpdateAutoTestDef Copy def file to working test directory and update the def file header and file names
Procedures for Communications to the Database
NotifyDatabase Communicate with the job database
dbccp4i_notify_database Communicate with the job database via the database handler
Procedures for Communications with the Main CCP4i Process
NotifyEditComFile Send command script to main CCP4i process for user to inspect/edit
NotifyRunAborted Notify main CCP4i process that the run has been aborted
NotifyRunCompleted Notify main CCP4i process that the run has completed
RunNotification Send message to main CCP4i process to say this process is running
FailNotification Send message to main graphical CCP4i process to say this process has failed
AddOutputDir Notify the main CCP4i process of additional output dirs
AddOutputFile Notify the main CCP4i process of additional output files
Communication between Script not written in Tcl and CCP4i
ExecuteSocket Use ccp4ish to communicate to CCP4i main process from a non-Tcl script
BufferSocketCommand Add a command to the buffer for execution later
GetBufferFile Return the name of the database command buffer file
OpenSocket Open a client side socket to connect to the server in the main CCP4i process
GetServerPort Return server port for CCP4i process associated with current project
Helper commands for CCP4i jobs (utils/job_utils.tcl)
Helper commands for CCP4i jobs