Skip to content

Infinite Sequence of All Combinations #311

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

EthanTheMaster
Copy link

Adds feature requested by #293

Helper iterator adapter to make calculating all combinations easier.

Copy link
Contributor

@tobz1000 tobz1000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like a good start, although it effectively locks the meta-iterator used as std::iter::repeat. Whether or not we want to add this flexibility is up to @bluss.

Other than that, my only concern is collecting the source iterator upon construction, which may cause memory usage issues, e.g.:

(0..1_000_000_000).product_combination()

Even worse if the source iterator is infinite - which would seem pointless, but could happen if a user passes on a use of ProductCombination through their own library - a contrived example:

fn product_combination_plus_one(it: impl Iterator<Item=usize>) -> impl Iterator<Item=Vec<usize>> {
    it.map(|i| i + 1).product_combination()
}

// User code
let arrays = product_combination_plus_one(0..);

We would expect the above to yield [1], [2], [3], ..., but it will panic on construction.

It would also be good to add some quickcheck tests.

self.increment();

//Map each digit from the base data.len() to its corresponding element
let mut result: Self::Item = self.internal_state.iter().map(|idx| {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A short-form closure is more idiomatic here:

self.internal_state.iter().map(|&idx| self.data[idx].clone())

I don't think result needs to be mut either.

.gitignore Outdated
@@ -1,2 +1,3 @@
/target/
Cargo.lock
.idea/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This .gitignore change doesn't belong in a feature PR

Copy link
Contributor

@fyrchik fyrchik Apr 26, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a tip: you can put all your IDE-related ignores in .git/info/exclude

(you can do it once in a template and then completely forget :)

@EthanTheMaster
Copy link
Author

@tobz1000 Thanks for the feedback! Support for huge vectors and infinitely size iterators has been added. Rather than collecting the iterator on construction, I just store the iterator and clone it whenever I needed to retrieve data. By doing this, the size of the iterator becomes irrelevant and doing things like mappings for infinitely sized iterators just work. However, cloning the iterator every time there is a retrieval comes with a performance hit as you have to iterate through n elements to reach the nth item.

Copy link
Contributor

@tobz1000 tobz1000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cloning the iterator rather than the elements indeed solves the memory issue - but as you've mentioned, there's a time-performance issue now.

To get around this, you can keep track of the adaptor's state using clones of the input iterator itself, rather than an index counter. Then, in most iterations, you'd only be moving on the fastest iterator by 1. Once it's exhausted, re-clone it, advance the next iterator by 1, and so on.

I think this will probably end up quite similar to MultiProduct, so it would be worth having a look at that (and Product, which is a bit easier to grasp).

One more thing I noticed is that the leftmost iterator is the fastest-moving - we normally have the rightmost iterator moving fastest, mimicking number digits.

@@ -1,2 +1,2 @@
/target/
Cargo.lock
Cargo.lock
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you re-add the final line break, you can remove this file from the diff entirely (or just git checkout master -- .gitignore)

Copy link
Contributor

@tobz1000 tobz1000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good progress so far. I think some further decisions to be made are whether to panic when the input iterator is empty (as currently), or to simply yield None straight away; and possibly the final name for the adaptor.

However I can't really say one way or the other. Hopefully a maintainer will be able to pick up the open PRs soon enough and decide this kind of thing.

tack_on = true;
break;
},
Some(ref mut pair) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't use default match binding in Itertools since it's still on Rust 1.12, but the tuple deref/item re-ref can still be done within the pattern match:

Some(&mut (ref mut iter, ref mut item)) => {


fn next(&mut self) -> Option<Self::Item> {
self.increment();
Some(self.internal_state.iter().map(|ref pair| pair.1.clone()).collect())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The item yielded by iter() is already a reference; you don't need to use ref in the closure. Also, the item can be obtained with a pattern match:

.map(|&(_, ref item)| item.clone())

@jswrenn jswrenn self-assigned this Jul 18, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants