API

delayed Wraps a function or object to produce a Delayed.
dask.delayed.delayed()

Wraps a function or object to produce a Delayed.

Delayed objects act as proxies for the object they wrap, but all operations on them are done lazily by building up a dask graph internally.

Parameters:

obj : object

The function or object to wrap

name : string or hashable, optional

The key to use in the underlying graph for the wrapped object. Defaults to hashing content.

pure : bool, optional

Indicates whether calling the resulting Delayed object is a pure operation. If True, arguments to the call are hashed to produce deterministic keys. Default is False.

nout : int, optional

The number of outputs returned from calling the resulting Delayed object. If provided, the Delayed output of the call can be iterated into nout objects, allowing for unpacking of results. By default iteration over Delayed objects will error. Note, that nout=1 expects obj, to return a tuple of length 1, and consequently for nout=0`, obj should return an empty tuple.

traverse : bool, optional

By default dask traverses builtin python collections looking for dask objects passed to delayed. For large collections this can be expensive. If obj doesn’t contain any dask objects, set traverse=False to avoid doing this traversal.

Examples

Apply to functions to delay execution:

>>> def inc(x):
...     return x + 1
>>> inc(10)
11
>>> x = delayed(inc, pure=True)(10)
>>> type(x) == Delayed
True
>>> x.compute()
11

Can be used as a decorator:

>>> @delayed(pure=True)
... def add(a, b):
...     return a + b
>>> add(1, 2).compute()
3

delayed also accepts an optional keyword pure. If False (default), then subsequent calls will always produce a different Delayed. This is useful for non-pure functions (such as time or random).

>>> from random import random
>>> out1 = delayed(random, pure=False)()
>>> out2 = delayed(random, pure=False)()
>>> out1.key == out2.key
False

If you know a function is pure (output only depends on the input, with no global state), then you can set pure=True. This will attempt to apply a consistent name to the output, but will fallback on the same behavior of pure=False if this fails.

>>> @delayed(pure=True)
... def add(a, b):
...     return a + b
>>> out1 = add(1, 2)
>>> out2 = add(1, 2)
>>> out1.key == out2.key
True

The key name of the result of calling a delayed object is determined by hashing the arguments by default. To explicitly set the name, you can use the dask_key_name keyword when calling the function:

>>> add(1, 2)    
Delayed('add-3dce7c56edd1ac2614add714086e950f')
>>> add(1, 2, dask_key_name='three')
Delayed('three')

Note that objects with the same key name are assumed to have the same result. If you set the names explicitly you should make sure your key names are different for different results.

>>> add(1, 2, dask_key_name='three')
>>> add(2, 1, dask_key_name='three')
>>> add(2, 2, dask_key_name='four')

delayed can also be applied to objects to make operations on them lazy:

>>> a = delayed([1, 2, 3])
>>> isinstance(a, Delayed)
True
>>> a.compute()
[1, 2, 3]

The key name of a delayed object is hashed by default if pure=True or is generated randomly if pure=False (default). To explicitly set the name, you can use the name keyword:

>>> a = delayed([1, 2, 3], name='mylist')
>>> a
Delayed('mylist')

Delayed results act as a proxy to the underlying object. Many operators are supported:

>>> (a + [1, 2]).compute()
[1, 2, 3, 1, 2]
>>> a[1].compute()
2

Method and attribute access also works:

>>> a.count(2).compute()
1

Note that if a method doesn’t exist, no error will be thrown until runtime:

>>> res = a.not_a_real_method()
>>> res.compute()  
AttributeError("'list' object has no attribute 'not_a_real_method'")

Methods are assumed to be impure by default, meaning that subsequent calls may return different results. To assume purity, set pure=True. This allows sharing of any intermediate values.

>>> a.count(2, pure=True).key == a.count(2, pure=True).key
True

As with function calls, method calls also support the dask_key_name keyword:

>>> a.count(2, dask_key_name="count_2")
Delayed("count_2")