getCheckpointFile

Class: matlab.compiler.mlspark.RDD
Package: matlab.compiler.mlspark

Get the name of the file to which an RDD is checkpointed

expand all in page

Syntax

file = getCheckpointFile(obj)

Description

file = getCheckpointFile(obj) gets the name of the file to which the RDD obj is checkpointed.

Input Arguments

expand all

An input RDD, specified as an RDD object.

Output Arguments

expand all

The name of the file to which an RDD was checkpointed, returned as a character vector.

Examples

expand all

Get the name of the file to which an RDD was checkpointed.

%% Connect to Spark
sparkProp = containers.Map({'spark.executor.cores'}, {'1'});
conf = matlab.compiler.mlspark.SparkConf('AppName','myApp', ...
                        'Master','local[1]','SparkProperties',sparkProp);
sc = matlab.compiler.mlspark.SparkContext(conf);

%% getCheckpointFile
sc.setCheckpointDir('myDir')  % set the checkpoint directory
x = sc.parallelize({1,2,3})
y = x.map(@(x)({x,1}));
y.checkpoint()  % tell spark to checkpoint the RDD
y.collect()
% need to call collect, so that spark actually materializes the RDD 
% and checkpoints to the myDir directory
y.getCheckpointFile()

See Also

| | |

Introduced in R2016b

Was this topic helpful?
MATLAB Compiler Documentation
Other Documentation
Support