Broadcasting
Broadcast an expression to all batch workers of (future) batch jobs and return a batch future. The batch future can be passed as a function argument instead of the variable.
batch_future = @bcast expr
The use of @bcast
is recommended to pass large arguments to functions (e.g. arrays). This avoids copying input arguments to each individual task separately. Instead, expressions tagged via @batchdef
are uploaded to blob storage once and their blob reference is passed to one or multiple tasks.
To access a broadcasted variable inside an executed function, use the fetch
or fetch!
(in-place) function:
# Create and broadcast array
A = randn(2, 2)
_A = @bcast A
# Define function
@batchdef function print_array(_A)
A = fetch(_A) # load A into memory
print(A)
end
# Remotely execute function
@batchexec print_array(_A) # pass batch future
Calling A = fetch(_A)
on the local machine (rather than on a batch worker) downloads the broadcasted variable from blob storage and returns it.