stack-safe fromFoldable

[matthewleon]

with benchmarks that reflect no performance degradation

addresses #108

[paf31]

Thanks!

To be clear, this is only safe if the Foldable instance for f is too, right?

It's a bit of a shame to have to materialize the lazy list, when we use it one element at a time. I feel like we should try to build some sort of general build/fold like thing in the core libraries for this sort of thing.

But later perhaps. I'm happy with this, thanks! @garyb @natefaubion Any thoughts?

[natefaubion]

I'd be interested to see if this is better than using Array.fromFoldable and foreachE.

[paf31]

Ah yes, good idea.

[matthewleon]

I'd be interested to see if this is better than using Array.fromFoldable and foreachE.

Good idea indeed. Sorry, I wasn't avoiding your suggestion just for the sake of it.

I'll make a branch with some comparisons and link to it below.

[matthewleon]

Actually, it looks like I screwed up the benchmark by importing Map instead of StrMap. Stack is exploding once again. I'll have the array-based benchmark up soon to compare.

[matthewleon]

I'd be interested to see if this is better than using Array.fromFoldable and foreachE.

I've switched to Array.fromFoldable and foreachE. The benchmarks show them to be significantly faster. Thanks @natefaubion

for reference, Data.Map.fromFoldable is now:

fromFoldable (100000)
mean   = 936.02 ms
stddev = 53.54 ms
min    = 892.65 ms
max    = 1.04 s

and Data.StrMap.fromFoldable is now:

fromFoldable (100000)
mean   = 198.91 ms
stddev = 27.84 ms
min    = 180.95 ms
max    = 276.26 ms

With the old approach, performance of StrMap.fromFoldable was about 20% faster than Map.fromFoldable.

[matthewleon]

To be clear, this is only safe if the Foldable instance for f is too, right?

That certainly makes intuitive sense. Shall we put it to the test to be sure?

[matthewleon]

It's a bit of a shame to have to materialize the lazy list, when we use it one element at a time. I feel like we should try to build some sort of general build/fold like thing in the core libraries for this sort of thing.

Strongly agreed. Actually I leaned toward the lazy list before the array version because I figured there would be a way to use it without retaining a reference to the head, allowing the GC to sweep up progressively. But that didn't work out...

The array version, though faster, still has this flaw of requiring us to build up a data structure that we just end up iterating through, before handing it to the GC.

@paf31 do you have any ideas here? I remember I once experimented with a version of Lazy that never actually memoized. Something along those lines? On another note, is there an appropriate repo or other place to have a conversation about this?

[matthewleon]

On further thought, perhaps the simplest solution here is to create a foreachF for Foldables in purescript-eff? This would, sadly, make foldable-traversable a dependency. But it should serve to simplify this kind of code.

[matthewleon]

a foreachF for Foldables in purescript-eff

I think just traverse_ specialized to Eff so that thunks don't accumulate.

[paf31]

Do you think this is ready to merge? If so, we can come back to this once we figure out what to do about traverseE.

[matthewleon]

Do you think this is ready to merge?

Yes.

[paf31]

Thanks!