Documentation

Different Workers

By default, each worker in a cluster working on the same job has a unique random number stream. This example uses two workers in a parallel pool to show they generate unique random number sequences.

p = parpool(2);
spmd
    R = rand(1,4); % Different on each worker
end
R{1},R{2}

    0.3246    0.6618    0.6349    0.6497

    0.2646    0.0968    0.5052    0.4866

delete(p)

If you need all workers to generate the same sequence of numbers, you can seed their generators all the same.

p = parpool(2);
spmd
    s = RandStream('twister'); % Default seed 0.
    RandStream.setGlobalStream(s);
    R = rand(1,4); % Same on all workers
end
R{1},R{2}

    0.8147    0.9058    0.1270    0.9134

    0.8147    0.9058    0.1270    0.9134

delete(p)

For instructions on how to generate a reproducible set of random numbers in a parfor-loop, see Repeat Random Numbers in parfor-Loops.

Client and Workers

By default, the MATLAB^® client and MATLAB workers use different random number generators, even if the workers are part of a local cluster on the same machine with the client. For the client, the default is the Mersenne Twister generator ('twister'), and for the workers the default is the Combined Multiple Recursive generator ('CombRecursive' or 'mrg32k3a'). If it is necessary to generate the same stream of numbers in the client and workers, you can set one to match the other.

For example, you might run a script as a batch job on a worker, and need the same generator or sequence as the client. Suppose you start with a script file named randScript1.m that contains the line:

R = rand(1,4);

You can run this script in the client, and then as a batch job on a worker. Notice that the default generated random number sequences in the results are different.

randScript1; % In client
R

R =
    0.8147    0.9058    0.1270    0.9134

parallel.defaultClusterProfile('local')
c = parcluster();
j = batch(c,'randScript1'); % On worker
wait(j);load(j);
R

R =
    0.3246    0.6618    0.6349    0.6497

For identical results, you can set the client and worker to use the same generator and seed. Here the file randScript2.m contains the following code:

s = RandStream('CombRecursive','Seed',1);
RandStream.setGlobalStream(s);
R = rand(1,4);

Now, run the new script in the client and on a worker:

randScript2; % In client
R

R =
    0.4957    0.2243    0.2073    0.6823

j = batch(c,'randScript2'); % On worker
wait(j); load(j);
R

R =
    0.4957    0.2243    0.2073    0.6823

Client and GPU

By default MATLAB clients use different random generators than code running on a GPU. GPUs are more like workers in this regard, and use the Combined Multiple Recursive generator ('CombRecursive' or 'mrg32k3a') unless otherwise specified.

This example shows a default generation of random numbers comparing CPU and GPU in a fresh session.

Rc = rand(1,4)

Rc =
    0.8147    0.9058    0.1270    0.9134

Rg = rand(1,4,'gpuArray')

Rg =
    0.7270    0.4522    0.9387    0.2360

Be aware that the GPU supports only three generators ('CombRecursive', 'Philox4x32-10', and 'Threefry4x64-20'). The following table lists the algorithms for these generators and their properties.

None of these is the default client generator for the CPU. To generate the same sequence on CPU and GPU, you must use the only generator supported by both: 'CombRecursive'.

sc = RandStream('CombRecursive','Seed',1);
RandStream.setGlobalStream(sc);
Rc = rand(1,4)

Rc =
    0.4957    0.2243    0.2073    0.6823

sg = parallel.gpu.RandStream('CombRecursive','Seed',1);
parallel.gpu.RandStream.setGlobalStream(sg);
Rg = rand(1,4,'gpuArray')

Rg =
    0.4957    0.2243    0.2073    0.6823

For normally distributed random numbers created by randn, CPU code by default uses a random stream with a NormalTransform setting of Ziggurat, while GPU code uses a setting of Inversion. You can set CPU and GPU generators the same to get the same randn sequence. The GPU supports only Inversion, so set the CPU to match:

sc = RandStream('CombRecursive','NormalTransform','Inversion','Seed',1);
RandStream.setGlobalStream(sc)

sg = parallel.gpu.RandStream('CombRecursive','NormalTransform','Inversion','Seed',1);
parallel.gpu.RandStream.setGlobalStream(sg);

Rc = randn(1,4)

Rc =
   -0.0108   -0.7577   -0.8159    0.4742

Rg = randn(1,4,'gpuArray')

Rg =
   -0.0108   -0.7577   -0.8159    0.4742

Worker CPU and Worker GPU

Code running on a worker’s CPU uses the same generator to create random numbers as code running on a worker’s GPU, but they do not share the same stream. You can use a common seed to generate the same sequence of numbers, as shown in this example, where each worker creates the same sequence on GPU and CPU, but different from the sequence on the other worker.

p = parpool(2);
spmd
    sc = RandStream('CombRecursive','Seed',labindex);
    RandStream.setGlobalStream(sc);
    Rc = rand(1,4)

    sg = parallel.gpu.RandStream('CombRecursive','Seed',labindex);
    parallel.gpu.RandStream.setGlobalStream(sg);
    Rg = rand(1,4,'gpuArray')
end
delete(p)

For normally distributed random numbers from randn, by default a worker CPU uses a NormalTransform setting of Ziggurat while a worker GPU uses a setting of Inversion. You can set them both to use Inversion if you need the same sequence from CPU and GPU.