This section discusses programming communicating jobs using the generic scheduler interface. This interface lets you execute jobs on your cluster with any scheduler you might have.
The principles of using the generic scheduler interface for communicating jobs are the same as those for independent jobs. The overview of the concepts and details of submit and decode functions for independent jobs are discussed fully in Program Independent Jobs for a Generic Scheduler. For this purpose, you need to have Parallel Computing Toolbox™ as well as MATLAB® Distributed Computing Server™ running on your cluster.
The basic steps follow.
Coding a communicating job for a generic scheduler involves the same procedure as coding an independent job.
Create an object representing your cluster with parcluster.
Set the appropriate properties on the cluster object if they are not defined in the profile. Because the scheduler itself is often used by many users and applications, it is probably best to use a profile for programming these properties. See Discover Clusters and Use Cluster Profiles.
Among the properties required for a communicating job is IntegrationScriptsLocation.
This property points to the location of the communicatingSubmitFcn.m function
that executes when a communicating job is submitted. You can write
your own communicating submit and decode functions, or use those that
come with the product for various schedulers and platforms; see the
following section, Troubleshooting and Debugging.
Use createCommunicatingJob to
create a communicating job object for your cluster.
Create a task, run the job, and retrieve the results as usual.
To support usage of the generic scheduler interface, templates and scripts can be installed from the following locations:
Each installer provides templates and scripts for the supported
submission modes for shared file system, nonshared file system, or
remote submission. Each submission mode has its own subfolder within
the installation folder, which contains a file named README that
provides specific instructions on how to use the scripts. To use these
support scripts with your cluster object, set its IntegrationScriptsLocation property
to the location of this subfolder.
For more information on programming jobs for generic schedulers, see:
For each scheduler type, the folder (or configuration subfolder) contains wrappers, submit functions, and other job management scripts for independent and communicating jobs. For example, you can install the following files for use with a PBS scheduler:
| Filename | Description |
|---|---|
independentSubmitFcn.m | Submit function for an independent job |
communicatingSubmitFcn.m | Submit function for a communicating job |
independentJobWrapper.sh | Script that is submitted to PBS to start workers that evaluate the tasks of an independent job |
communicatingJobWrapper.sh | Script that is submitted to PBS to start workers that evaluate the tasks of a communicating job |
deleteJobFcn.m | Script to delete a job from the scheduler |
extractJobId.m | Script to get the job’s ID from the scheduler |
getJobStateFcn.m | Script to get the job’s state from the scheduler |
getSubmitString.m | Script to get the submission character vector for the scheduler |
postConstructFcn.m | Script that is run following cluster object creation |
These files are all programmed to use the standard decode functions
provided with the product, so they do not have specialized decode
functions. For communicating jobs, the standard decode function provided
with the product is parallel.cluster.generic.communicatingDecodeFcn.
You can view the required variables in this file by typing
edit parallel.cluster.generic.communicatingDecodeFcn
The folders for other scheduler types contain similar files.
As more files or solutions for more schedulers might become
available at any time, visit the product page at http://www.mathworks.com/products/distriben/.
This Web page provides links to updates, supported schedulers, requirements,
and contact information in case you have any questions.