Documentation

Hadoop Settings File

In creating a deployable archive, you must create a Hadoop® settings file that contains configuration details. If you are using mcc, create a text file. If you are using deploytool, the Hadoop Compiler app automatically creates the file for you when you select the map function, the reduce function, the input type, and the output type. You can view the contents of your settings file in the Configuration file contents section of the Hadoop Compiler app.

Parameter TypeDescriptionDefault Value

mw.mapper

MATLAB® map function name

Hadoop identity map function

mw.reducer

MATLAB reduce function name

Hadoop identity reduce function

mw.ds.in.type

MATLAB input type

The input type is of two types, tabulartext and binary. The tabulartext input type is a formatted text file. The file is either a source file or result of the previous mapreduce job. The binary input type is a sequence file.

tabulartext

mw.ds.in.format

This parameter is valid with tabulartext input type. This parameter specifies a .mat file that contains a datastore.

None

mw.ds.in.readsize

This parameter is valid with binary input type. This parameter specifies a number that are number of rows for passing to the map function.

1

mw.ds.out.type

MATLAB output type

The output type is of two types, tabulartext and binary. The tabulartext output type writes to a text file. The binary output type writes to a sequence file.

binary

This example shows a settings file with tabulartext input type:

mw.mapper = maxArrivalDelayMapper
mw.reducer = maxArrivalDelayReducer
mw.ds.in.type = tabulartext
mw.ds.in.format = airlinesmall.mat
mw.ds.out.type = tabulartext

This example shows a settings file with binary input type:

mw.mapper = maxArrivalDelayMapper
mw.reducer = maxArrivalDelayReducer
mw.ds.in.type = binary
mw.ds.in.readsize = 1
mw.ds.out.type = tabulartext

Related Examples

Was this topic helpful?
MATLAB Compiler Documentation
Other Documentation
Support