matlab.mapreduce.DeployHadoopMapReducer class

Package: matlab.mapreduce

Configure a MapReduce application for deployment against Hadoop

expand all in page

Description

A DeployHadoopMapReducer object represents executing MapReduce on a Hadoop® cluster with MATLAB® Runtime.

Construction

config = matlab.mapreduce.DeployHadoopMapReducer creates a matlab.mapreduce.DeployHadoopMapReducer object that specifies the default properties for Hadoop execution.

Use the resulting object as input to the mapreducer function to specify the configuration properties for Hadoop execution. For deploying a standalone application, pass the matlab.mapreduce.DeployHadoopMapReducer object as input to mapreduce.

config = matlab.mapreduce.DeployHadoopMapReducer(Name,Value) creates a matlab.mapreduce.DeployHadoopMapReducer object with properties specified by one or more name-value pair arguments.

Input Arguments

expand all

Name-Value Pair Arguments

Specify optional comma-separated pairs of Name,Value arguments. Name is the argument name and Value is the corresponding value. Name must appear inside single quotes (' '). You can specify several name and value pair arguments in any order as Name1,Value1,...,NameN,ValueN.

expand all

Path to Hadoop installation, specified as the comma-separated pair consisting of the HadoopInstallFolder and a character vector.

The default value of Hadoop install folder is specified by the environment variables in the order of precedence of MATLAB_HADOOP_INSTALL, HADOOP_PREFIX, and HADOOP_HOME.

Path to Hadoop application configuration files, specified as the comma-separated pair consisting of the HadoopConfigurationFile and a character vector.

MATLAB Runtime installation folder for Hadoop cluster, specified as the comma-separated pair consisting of the MCRRoot and a character vector.

MCRRoot specifies the MATLAB Runtime installation folder used by Hadoop when executing mapreduce tasks in Hadoop.

Example: 'MCRRoot','/hd-shared/hadoop-2.2.0/MCR/v84'

A containers.Map object of name-value pairs that specify Hadoop configuration properties for a specific job or application. Name-value pairs must be specified as character vectors.

The properties specified in the containers.Map object are passed as a [GENERIC_OPTION] consisting of name-value pairs signaled by a -D flag to the hadoop shell command.

Example:

setenv('HADOOP_PREFIX', '/usr/lib/hadoop') % replace with your Hadoop install location
name = {'mapreduce.map.maxattempts','mapreduce.job.reduces'};
value = {'2','1'};
prop = containers.Map(name,value);
obj = matlab.mapreduce.DeployHadoopMapReducer('HadoopProperties', prop)

Examples

expand all

Create and use a matlab.mapreduce.DeployHadoopMapReducer object to deploy into a standalone application, and deploy against Hadoop.

config = matlab.mapreduce.DeployHadoopMapReducer('MCRRoot',...
    '/hd-shared/hadoop-2.2.0/MCR/v84');
mr = mapreducer(config);

See Also

|

Topics

Was this topic helpful?
MATLAB Compiler Documentation
Other Documentation
Support