CS2 Discussion: Output: for-in/for-of

[rattrayalex]

CoffeeScript and ES6 have very different takes on these features. CoffeeScript's version is arguably more useful, but also more complicated – and non-standard.

What should we do?

[DomVinyard]

Non-standard yes, not sure I agree that it's more complicated. I'm not convinced that there is any compelling need to change these.

[carlsmith]

Personally, I think CoffeeScript made a mess of this. I would have opted for doing this like Python does. It's less complex, and more powerful.

[kirly-af]

Can you elaborate on how you would expect it to work ?

What do you mean Python loops are "more powerful" ? ?
Furthermore, this complexity (I guess you refer to the different for statements in CS) is the result of JS bad design.

The real issue is jashkenas/coffeescript#3832.

[carlsmith]

So, in Python, you only have one kind of for-loop, and it always iterates over a sequence. You can use it on strings, lists and sets.

$ for n in [1, 2, 3]: print n
1
2
3

You can also use it on a dictionary, where it will iterate over the keys.

$ for key in {"foo": 1, "bar": 2}: print key
foo
bar

If you want to iterate over the values, just call values to get a list of them:

$ for value in {"foo": 1, "bar": 2}.values(): print value
1
2

Python also supports unpacking sequences in assignment statements. It's a lot like CoffeeScript's version, but doesn't require any brackets.

$ x, y, z = [1, 2, 3]
$ print x + y + z
6

Python combines the two ideas, and supports unpacking a sequence inside of a for-loop.

$ sequence_of_tuples = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
$ for x, y, z in sequence_of_tuples: print x + y + z
6
15
24

While this doesn't seem that helpful at first, it is very powerful once you have some functions and methods that feed into it. For example, CoffeeScript has its magical fudge for getting the index:

$ (console.log index, char) for char, index in "abc"
0 a
1 b
2 c

In Python, there's no magic involved. You just use the enumerate function that takes a sequence, and returns an enumeration of the sequence, which is essentially a list of index-item tuples.

$ print list(enumerate("abc"))
[(0, 'a'), (1, 'b'), (2, 'c')]

So we can then do this to get the same effect as CoffeeScript, without any magic:

$ for index, character in enumerate("abc"): print index, character
0 a
1 b
2 c

The main advantage of not using magic is that we can use the same feature to iterate over anything we like, not just index-item tuples.

$ for key, value in {"foo": 1, "bar": 2}.items(): print key, value
foo 1
bar 2

$ colors = [(225, 0, 60), (0, 255, 100)]
$ for r, g, b in colors: use(r, g, b)

And so on. There's lots of ways to break a structure into a sequence of pairs or triples or n-length tuples.

Python doesn't need to distinguish between for-in and for-of. A for-loop is a for-loop. It also doesn't need magic to get the index. Python just combines its concept of a basic for-loop and its concept of unpacking a sequence, and everything else flows naturally from there.

Python actually takes iteration much further, but it's all possible because they got their for-loops right, and could build on them.

[carlsmith]

Doing things Pythonically would allow us to get rid of for-of loops altogether, and then just create methods or functions for turning whatever we have into whatever we need. In CoffeeScript, that'd look like this:

for key in hash.ownKeys()
for key in hash.allKeys()
for index in array.indexes()
for [item, index] in enumerate(array)
for [r, g, b] in elements.map (element) -> element.backgroundColorTuple()

Due to JavaScript's issues with defining methods on builtins, we'd probably use functions instead (they can be contained in a module these days), but you get the idea. You convert whatever structure you have into a sequence of tuples, and iterate over them.

You can use the same logic in list, dictionary and set comprehensions too.

foo = {
  key.toLowercase(): value * 2
  for key, value in someHash.items()
}

And generator expressions... Well, you get the point. Python nailed iteration.

If we supported unpacking a sequence inside a for-loop when the names are wrapped in brackets (for [key, value] in hash.items()), then we would have the option of using Python style for-loops without breaking backwards compatibility, assuming we want to keep something like the old behaviour as well. The square brackets are also consistent with CoffeeScript assignment syntax ([a, b, c] = [1, 2, 3]).

[JimPanic]

This is nice and a lot more intuitive. I support this a lot!

[kirly-af]

@carlsmith: I like the idea of unpacking sequences inside for loops.

Though I completely disagree we should introduce new functions and/or methods in the language like the enumerate() you suggest. I personally think CoffeeScript success is (partially) due its idiom It's just JavaScript.

Also, I don't consider the way CoffeeScript provides the index in for-in loops like magical.
The way I see it is that CoffeeScript treats Objects like Maps: a collection of sub-objects, iterable.

In the end, the only real difference we have with array-like values that are intended to work with for-in loops (ie. arrays and strings) is the key type (string vs number).

So to me the actual issue is the inconsistency we have between for-of and for-in:

(console.log key, val) for key, val of {'foo': 'bar'}
(console.log idx, val) for val, idx in ['foo', 'bar']

What would make the more sense to me would be to have only one for (I would choose for-in over for-of but this is not our concern here), and keep being consistent with the way we treat the index/key.

(console.log key, val) for key, val in {'foo': 'bar'}
(console.log idx, val) for idx, val in ['foo', 'bar']
(console.log idx, char) for idx, char in 'foobar'
(console.log count, item) for count, item in someIterrableObject # generator yield, es6 map, es6 set, etc.

Also, it would be easy to assume only value is required if we provide only one parameter to for (apologize I know parameter is not the proper term), we can assume only the value is requested.

(console.log key, val) for key, val in {'foo': 'bar'}
(console.log val) for val in {'foo': 'bar'}
(console.log idx, char) for idx, char in 'foobar'
(console.log char) for char in 'foobar'
# ... and so on

Now we have the main structure of our for loop: for <optionalKey>, value in someCollection.
Then, we could integrate the destructuring on both key and value:

(console.log foo, bar) for {foo: bar} in [{a: 1},{b: 2},{c: 3}]
(console.log thisIsA, mapKey) for {thisIsA: mapKey}, {andA: mapValue} in new Map(...) # maps keys can be any kind of values

This would require more efforts but in a perfect world, this is the way I would expect iteration to be.

[carlsmith]

Of course, there are different systems, and they each have there own merits, but Python is known for being especially good at iteration, so in an ideal world, I'd always opt for that approach. It has implications we didn't get into here as well. Python does a lot with iteration.

You don't have to add new functions like enum to the language. You just allow unpacking when more than one name inside brackets is used in a for-loop for [x, y] in coords and then people can add the rest of the functionality with library code and their own stuff. It's just sugar for for coord in coords then [x, y] = coord.

[kirly-af]

@carlsmith

Due to JavaScript's issues with defining methods on builtins, we'd probably use functions instead (they can be contained in a module these days), but you get the idea. You convert whatever structure you have into a sequence of tuples, and iterate over them.

That's the part of your post that confused me concerning the possible introduction of new functions.
I got your point about unpacking sequences and I completely agree this would be a great benefit to be able to use it in for loops as you demonstrated.

Though, my point was more about how to unify the different kind of CoffeeScript for loops, and so the inconsistency between for-in and for-of (ie. for val, index in vs for key, val of). Furthermore, we haven't talked yet about ES6 iterables. AFAIK those need to be integrated as neither for-in nor for-of loops can handle it.

Thus, a for-from form has been suggested to handle iterables in jashkenas/coffeescript#3832 (the whole issue is worth reading btw).
Should we introduce a new form ? Or should we unify it through a unique single for-in ?
I'd vote for the second option. The only issue is that it would involve if statements in the transpiled code, like suggested in that comment. I personally this would be better option.

Having 3 different kinds of for loops sounds crazy to me, there should be only one.

[DomVinyard]

I really like @kirly-af's example

(console.log key, val) for key, val in {'foo': 'bar'}
(console.log idx, val) for idx, val in ['foo', 'bar']

[carlsmith]

What would it compile to? We have to assume we don't know the type of the iterable, as it will often be an expression that evaluates to something that's defined elsewhere.

[carlsmith]

Having 3 different kinds of for loops sounds crazy to me, there should be only one.

Adding unpacking to loops doesn't create a new type of loop. It just makes the assignment in for-loops consistent with normal assignments. We could do what you're suggesting, and still allow unpacking in for-loops. We could have any kind or any number of for-loops, and still allow assignments to be unpacked within them.

I agree we should only have one kind of for-loop, but don't think it should have pseudo-unpacking, with only two names, and hardcode what gets unpacked based on the iterable's type. If we ever did create a dialect of CoffeeScript with one kind of for-loop, I'd much prefer Python's iteration, which handles all iterables, including instances of user-defined classes, and is far more flexible.

[carlsmith]

Sorry if that came across badly. I didn't mean to be rude. I appreciate you're trying to come up with something as close to CoffeeScript as possible, but sane as well, and in that context, what you have looks good.

I'm not certain, but I think CoffeeScript is inconsistent because it doesn't know the types, so it's difficult or impossible to cover every case correctly in the compiled code without injecting a function call for the type check, and they don't like to do that because of performance. Injecting a type check invocation wouldn't be too expensive as the iterable expression is only evaluated once (not per iteration, like while and until), so you probably could do what you're suggesting without any serious performance issues. Still, if you're changing the language enough to only have one kind of for-loop, why not just redo for-loops altogether?