Eager functions in API appear in conflict with lazy implementation

[cbourjau]

We are looking at adapting this API for a lazy array library built on top of ONNX (and Spox). It seems to be an excellent fit for most parts. However, certain functions in the specification appear to be in conflict with a lazy implementation. One instance is __bool__ which is defined as:

array.__bool__() → bool

The output is documented as "a Python bool object representing the single element of the array". This is problematic for lazy implementations since the value is not available at this point. How should a standard compliant lazy implementation deal with this apparent conflict?

[asmeurer]

It's impossible for __bool__ to be lazy. bool(x) will call x.__bool__() and convert the result into a boolean.

How should a standard compliant lazy implementation deal with this apparent conflict?

I guess it's up to how you want to design your library, but I guess you really only have two choices: either make bool() implicitly perform a compute on the lazy graph, or make it raise an exception.

Note that the standard does already discuss this in the context of data-dependent output shapes https://data-apis.org/array-api/latest/design_topics/data_dependent_output_shapes.html, but perhaps we also need to identify other functions such as __bool__ that are problematic for such libraries.

One question I have is, for a lazy evaluation library, how should it work for array consuming libraries (e.g. scipy) that are written against a generic array API expecting both eager and non-eager array backends? Is it the case that a scipy function should just take in a lazy array object and return a computation graph object, and fail if it tries to call a function that cannot be performed lazily? Or do these libraries need to know how to call compute() (or whatever) at the end? Should the function to perform an actual compute be standardized?

There's also a question of how we can support libraries like this in the test suite. The test suite calls things like __bool__ all the time because it compares actual values in the tests. So it would need to support calling compute() or whatever to get actual array values. But this is orthogonal to the discussions here for the actual standard (feel free to open an issue in the test suite repo if you want to discuss this further).

[adityagoel4512]

One question I have is, for a lazy evaluation library, how should it work for array consuming libraries (e.g. scipy) that are written against a generic array API expecting both eager and non-eager array backends?

I think clearly advertising if the array backend in use currently is lazy or eager might make sense as part of the standard. If "lazy", any API return types are also lazy and must be explicitly materialised by the user with a standard top-level compute(array, *args) call. If a library is "eager" (as most are) then return values are already available.

This may make things easier to work with in the test suite as well?

I guess you really only have two choices: either make bool() implicitly perform a compute on the lazy graph, or make it raise an exception.

I think doing implicit computation is not possible in situations where you are looking to construct a computation graph but do not have your "inputs"/"arguments" available yet. It would also lead to a situation where some parts of the API yield concrete values and others are lazily constructing the computation graph which may be hard to navigate for a user.

I suppose this very similar to https://data-apis.org/array-api/latest/design_topics/data_dependent_output_shapes.html as you have mentioned. Just raising for __bool__ might be a solution but maybe it makes sense to update the specification to suggest that the function should throw if the eager value is not available.

[leofang]

Sorry but it makes zero sense to return anything other than the built-in bool. It's part of the Python language requirement
and we can't do anything about it. In particular, the return type cannot be anything other than bool, such as an array type or a deferred evaluation proxy type. You can go check that, for example, Dask forces evaluating the graph in its array and errors out in its proxy object, respectively, when __bool__ is called.

If there's a computation which could compute a Boolean that you could then use in the subsequent evaluation, lazily, what you need should be simply arr.astype(arr.__array_namespace__.bool), which should work in a lazy array implementation.

[adityagoel4512]

I don't disagree, the lazy/eager suggestion was for the specific question not for __bool__.

[leofang]

Not sure what question you're referring to, Aditya?

Regarding Arron's question about when to call compute(), I have a strong opinion that it should be the end user's responsibility, not any array-consuming libraries'. Suppose we have 3 libraries A, B, C, where A provides a lazy array, B calls A's APIs, C calls B's APIs, and the end user creates A arrays and calls C APIs. For simplicity we could assume A being Dask, B SciPy, and C scikit-learn. If B or C calls compute() on behave of the user (of C), the graph created by the user would be disconnected/materialized at the library boundary, not across the 3 libraries. This also saves all of the array commuting libraries' lives, by staying at lazy/eager agnostic.

[rgommers]

Regarding lazy and .compute(), I agree with @leofang's answer - array-consuming libraries should be agnostic to this, and return the same array type as that of the input array type. So no reason not to stay lazy. We had the same question on the dataframe side, where I wrote up a longer answer: data-apis/dataframe-api#120

Sorry but it makes zero sense to return anything other than the built-in bool. It's part of the Python language requirement
and we can't do anything about it.

I'm not sure I agree with that. We must indeed specify -> bool in the standard, however if there's an implementation that's purely lazy, I don't see why it would be a problem to return a proxy object that duck types with bool. It will technically deviate from what the Python language docs and the standard docs say, but everything will work just fine I believe. We have multiple similar cases where we specify returning a Python scalar, (e.g. finfo attributes) and we've always said it's fine to duck type. I don't yet see why __bool__ would be different.

Does anyone know why Dask forces evaluation? My guess would be that it's a pragmatic decision only - keeping conditionals as graph nodes can explore the graph size very quickly, since everything that follows bifurcates each time you call __bool__.

[cbourjau]

Thanks for the insightful discussion and links! A little more about our use case: We are building a lazy library that can be used to "trace" standard compliant code and then exports the resulting graph to ONNX. I.e. we are (primarily) interested in the computational graph. The values on which it will run are not necessarily available. That said, we happen to have some eager computations on top of that for debugging and testing.

I would be fine with raising an error when __bool__ is called.

Should the standard be updated to reflect the fact that lazy libraries may raise from __bool__ and similar functions?

[asmeurer]

I'm not sure I agree with that. We must indeed specify -> bool in the standard, however if there's an implementation that's purely lazy, I don't see why it would be a problem to return a proxy object that duck types with bool. It will technically deviate from what the Python language docs and the standard docs say, but everything will work just fine I believe. We have multiple similar cases where we specify returning a Python scalar, (e.g. finfo attributes) and we've always said it's fine to duck type. I don't yet see why bool would be different.

Just to be completely clear, it's physically impossible for __bool__ to return anything other than bool. Returning anything other than True or False from __bool__ results in an error:

>>> class Test:
...     def __bool__(self):
...         return 1
>>> bool(Test())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __bool__ should return bool, returned int

(the same is true for __int__, __float__, and __complex__).

And even if this limitation didn't exist, that wouldn't really help, because 90% of the time bool(x) is called implicitly as the result of something like if x or assert x.

I think the main takeaways from this discussion are:

• We should have a page discussing delayed evaluation, noting that it is supported and certain operations may not work with it, and that the function an actual compute is not standardized because it should typically be done by end-users on a per-library basis.

• We should add a note to __bool__, __int__, __float__, and __complex__ that they may not be supported by such libraries.

[ogrisel]

Regarding lazy and .compute(), I agree with @leofang's answer - array-consuming libraries should be agnostic to this, and return the same array type as that of the input array type. So no reason not to stay lazy.

I don't see how you can follow this policy in an Array API consuming library that implements an iterative solver (e.g. scikit-learn fitting a machine learning model). At each iteration you typically compute some summary scalar value and compare that to a scalar tolerance level (the result would be a scalar boolean) to make the decision to do another iteration or not.

In this case it seems that we have no other choice than letting the Array API consuming library decide explicitly when it needs to trigger the evaluation to collect the scalar value, even when the outcome of this iterative loop is a collection of n-dimensional arrays of the same type as the input.