mapreducer

Define parallel execution environment for mapreduce

mapreducer is the execution configuration function for mapreduce. This function specifies where mapreduce execution takes place. With Parallel Computing Toolbox™, you can expand the execution environment to include various compute clusters.

Syntax

  • mapreducer
  • mapreducer(0)
  • mapreducer(poolobj)
  • mapreducer(hcluster)
  • mapreducer(mr)
  • mr = mapreducer(___)
  • mr = mapreducer(___,'ObjectVisibility','Off')

Description

mapreducer specifies the default global execution environment for mapreduce.

If you have Parallel Computing Toolbox installed, and your default cluster profile specifies a local cluster, then mapreducer also opens a parallel pool so that mapreduce can distribute mapper and reducer tasks to the pool workers.

You can set your parallel preferences so that a pool does not automatically open. In this case, you must explicitly start a pool if you want to use parallel resources. See Parallel Preferences.

mapreducer(0) specifies that mapreduce run in the MATLAB client session without using any parallel resources.

mapreducer(poolobj) specifies a cluster for parallel execution of mapreduce. poolobj is a parallel.Pool object. The default pool is the current pool that is returned or opened by gcp.

mapreducer(hcluster) specifies a Hadoop® cluster for parallel execution of mapreduce. hcluster is a parallel.cluster.Hadoop object.

mapreducer(mr) sets the global execution environment for mapreduce using a previously created MapReducer object, mr, if its ObjectVisibility property is 'On'.

mr = mapreducer(___) returns a MapReducer object to specify the execution environment. You can define several MapReducer objects, allowing you to swap execution environments by passing one as an input argument to mapreduce.

mr = mapreducer(___,'ObjectVisibility','Off') hides the visibility of the MapReducer object, mr, using any of the previous syntaxes. Use this syntax to create new MapReducer objects without affecting the global execution environment of mapreduce.

If this object's ObjectVisibility property is 'On' (the default), mr defines the default global execution environment for mapreduce. If the ObjectVisibility property is 'Off', you can pass mr as an input argument to mapreduce to explicitly specify the execution environment for that particular call.

Input Arguments

collapse all

poolobj — Pool for parallel executiongcp (default) | parallel.Pool object

Pool for parallel execution, specified as a parallel.Pool object.

Example: p = gcp

hcluster — Hadoop cluster for parallel executionparallel.cluster.Hadoop object

Hadoop cluster for parallel execution, specified as a parallel.cluster.Hadoop object.

Example: hcluster = parallel.cluster.Hadoop

Output Arguments

collapse all

mr — Execution environment for mapreduceMapReducer object

Execution environment for mapreduce, returned as a MapReducer object.

Introduced in R2014b

Was this topic helpful?