Configure Environment for Interactive Debugging

Supported Platform: Linux® only.

Before you can interactively debug your applications using the MATLAB® API for Spark™, you must configure your MATLAB environment by adding the location of the Spark assembly jar to the front of MATLAB's static Java class path.

A Spark assembly JAR file includes all the Spark dependencies, including some internal Hadoop® dependencies. This JAR file is created during Spark installation and is specific to a particular combination of Spark and Hadoop versions. For example, using Spark version 1.4.0 and Hadoop version 2.4.0 creates the file spark-assembly-1.4.0-hadoop2.4.0.jar.

To add the location of the Spark assembly jar to the front of MATLAB’s static Java® class path, use the following procedure:

  1. Open a Linux terminal.

  2. Navigate to the directory <matlabroot>/bin, where <matlabroot> is the location where MATLAB is installed.

  3. Set the environment variable SPARK_HOME to point to the Spark installation directory. For example:

    ShellCommand
    csh / tcsh 

    % setenv SPARK_HOME /share/spark

    bash 

    $ export SPARK_HOME=/share/spark

  4. From the directory <matlabroot>/bin execute the shell script generatesparkjcpath. For example:

    $ ./generatesparkjcpath

    This will create a file called javaclasspath.txt in your <matlabroot>/bin directory. The contents of a sample javaclasspath.txt file look like this:

    <before>
/share/spark/conf/
/share/spark/lib/spark-assembly-1.4.0-hadoop2.4.0.jar
/share/spark/lib/datanucleus-rdbms-3.2.9.jar
/share/spark/lib/datanucleus-api-jdo-3.2.6.jar
/share/spark/lib/datanucleus-core-3.2.10.jar

    The <before> tag ensures that spark-assembly-1.4.0-hadoop2.4.0.jar is added to the front of the Java class path.

  5. Start MATLAB from the <matlabroot>/bin directory.

Note

You can create the javaclasspath.txt file in a directory other than <matlabroot>/bin by specifying the -o option. The file has to be named javaclasspath.txt. For example:

$ ./generatesparkjcpath -o /home/user/javaclasspath.txt
Start MATLAB from the directory /home/user. It is important that you start MATLAB from the directory containing the javaclasspath.txt file.

You can verify whether spark-assembly-1.4.0-hadoop2.4.0.jar was added to the front of the Java class path by typing the following at the MATLAB command prompt:

>> spath = javaclasspath('-static') ;
>> spath(1:5)
You should see /share/spark/lib/spark-assembly-1.4.0-hadoop2.4.0.jar on the top of the list.

You need to complete this configuration just once. Subsequent usage only requires that you start MATLAB from the folder containing the javaclasspath.txt file.

Related Topics

Was this topic helpful?
MATLAB Compiler Documentation
Other Documentation
Support