Use cases

[chicoxyzzy]

Old React use cases by @threepointone found in September 2016 meeting agenda https://gist.github.com/threepointone/014954c9270749d0b1d1051c12a705af

[runarberg]

Say you want to zip the output of a few iterators, but some of the iterators might be undefined (in which case the yielded array is sparse):

function* nullIter() {}

function* zip(iterables) {
  const iterators = iterables.map(iterable =>
    iterable?.[Symbol.iterator]?.() ?? nullIter(),
  );

  while (true) {
    const next = iterators.map(iterator => iterator.next());

    if (next.every(({ done }) => done)) {
      return;
    }

    yield next.map(({ value }) => value);
  }
}

It is indeed annoying that I have to name the nullIter. I would much rather write:

const iterators = iterables.map(iterable =>
  iterable?.[Symbol.iterator]?.() ?? (generator () => {})(),
);

Sure I can write { next: () => ({ done: true }) } but that requires knowledge of how iterators work internally and it feels a bit magical. When you see (generator () => {})() you know immediately that this is an iterator made from a generator that finishes immediately (i.e. a finished iterator that never yielded).

[hax]

@runarberg
Not sure how (generator () => {})() better than (function* () {})().

The old react example need lexical this which seems a good use case. But React hooks do not need that anymore.

So to be honest, we do not have very good use cases now.

[funkjunky]

For anyone who wants to take every improvement to DX they can, it's enough of a reason to simply improve the conciseness of the code:

setTrackSwitchHandle(*() => yield trainSpeedAtJunction() < 100 ? yield switchLeft() : yield switchRight())

setTrackSwitchHandle(function *() => { return yield trainSpeedAtJunction() < 100 ? yield yield switchLeft() : switchRight()); }

Frankly, I feel like you're trying to justify using oil paint in art instead of arcylic. Generators are such a massive expansion of what can be expressed in javascript. I used them to handle streaming io in web browsers and currently I'm making a platforming game in javascript that feeds generators through Redux. I'm sure there's significantly more powerful ways to use generators that feed into other generators.

The fact that we're debating whether we should add a new brush for painting with oil, is absurd. Is there justification?

You're dooming generators from the start if you don't provide enough tools to effectively use them. When i present generators to other people they look at the syntax and say "I'll just figure out how to solve it with async arrow functions" Provide the tools and they will come!

[VojtaStanek]

Here I have a real-world use-case (although it is about async iterator and it is in typescript, which may help readability).

class FooClient {
  constructor() {}

  async get<T>(endpoint: String): Promise<T> {
    const response = await fetch(endpoint);
    return await response.json();
  }

  async getAll<T>(): Promise<CountableGenerator<T>> {
    const ids = (await this.get(`/api/items`)).map((item) => item.id);
    
    const self = this // <-- This is needed
    async function* generator(): AsyncGenerator<T> { // because here I have to use function
      for (let id of ids) {
        yield self.get<T>(`/api/items/${id}`); // I would like to use `this` directly instead of `self`
      }
    }

    return {
      count: ids.length,
      generator: generator()
    }
  }
}

interface CountableGenerator<T> {
  count: number
  generator: AsyncGenerator<T>
}

[hax]

@cactucs

I'm not sure about the requirements of this case and how getAll need to be an async generator, it seems the name getAll implies Promise.all-like semantic?

Even without consider that, it seems this example could be wrote as

class FooClient {
  constructor() {}

  async get<T>(endpoint: String): Promise<T> {
    const response = await fetch(endpoint);
    return await response.json();
  }

  async *getMany<T>(endpoints: Iterable<String>): AsyncGenerator<T> {
    for (const endpoint of endpoints) yield this.get<T>(endpoint)
  }

  async getAllItems<T>(): Promise<CountableGenerator<T>> {
    const ids = (await this.get(`/api/items`)).map((item) => item.id);
    
    const endpoints = ids.map(id => `/api/items/${id}`)

    const generator = this.getMany<T>(endpoints)

    return {
      count: ids.length,
      generator,
    }
  }
}

[Jack-Works]

This is extremely useful when I working on do expression. Currently, I need to transform code like this:

class T extends Array {
    *f() {
        const a = do {
            yield super.values()
        }
    }
}

into

class T extends Array {
    *f() {
        var _a, _b;
        _b = (..._c) => super.values(..._c);
        const a = (yield* function* () {
            _a = yield _b();
        }.call(this), _a);
    }
}

If we have generator arrow function earlier, I can transform the result into

class T extends Array {
    *f() {
        var _a;
        const a = (yield* ()* => {
            _a = yield super.values();
        }.call(this), _a);
    }
}

And the transformer for generator arrow function will take care of super.* calls for me.

[dead-claudia]

Node's stream.pipeline and stream.compose requires passing not simply iterables, but functions returning them. Here's a quick example adapted from Node's docs, just using a class instead of a simple function (note the const self = this.

import {pipeline} from "node:stream/promises"
import * as fs from "node:fs"

class Transformer {
	// Initialize some input statistics

    async processChunk(chunk, {signal}) {
		// Transform, while updating input statistics in the process
	}

	async transform(sourceFile, targetFile) {
		const self = this
  		await pipeline(
			fs.createReadStream(source),
			async function* (source, {signal}) {
				source.setEncoding('utf8')
				for await (const chunk of source) {
					yield await self.processChunk(chunk, {signal})
				}
			},
			fs.createWriteStream(targetFile)
		)
	}
}

With an async generator arrow function, the self is no longer necessary.

class Transformer {
	// Initialize some input statistics

    async processChunk(chunk, {signal}) {
		// Transform, while updating input statistics in the process
	}

	async transform(sourceFile, targetFile) {
  		await pipeline(
			fs.createReadStream(source),
			async *(source, {signal}) => {
				source.setEncoding('utf8')
				for await (const chunk of source) {
					yield await this.processChunk(chunk, {signal})
				}
			},
			fs.createWriteStream(targetFile)
		)
	}
}

Of course, Node does have an (unstable) async map, as does https://github.com/tc39/proposal-iterator-helpers, so it isn't as compelling.

class Transformer {
	// Initialize some input statistics

    async processChunk(chunk, {signal}) {
		// Transform, while updating input statistics in the process
	}

	async transform(sourceFile, targetFile) {
  		await pipeline(
			fs.createReadStream(source)
				.setEncoding('utf8')
				.map((chunk, {signal}) => this.processChunk(chunk, {signal})),
			fs.createWriteStream(targetFile)
		)
	}
}

Here's a more persuasive one: the .flatMap for iterables in https://github.com/tc39/proposal-iterator-helpers and the existing Array.prototype.flatMap, if combined with this, could avoid a lot of boilerplate in more advanced use cases. It'd also allow for more flexible conversion for arrays and general iterables by allowing people to better switch between procedural and functional paradigms without having to create entire new named functions for it:

// Old
const result = list.flatMap(v => [
	v.one,
	...v.shouldIncludeTwo() ? [v.two] : [],
])

// New
const result = list.flatMap(*v => {
	yield v.one
	if (v.shouldIncludeTwo()) yield v.two
})

It's pretty easy to imagine this in a class as well, combined with several conditions and returning a promise to its result (say, it's streaming):

class Visitor {
	// ...

	async render(items, {signal}) {
		return items
			.filter(v => v.isRenderable)
			// apply however many more conditions and transformations to each
			// `v` as needed
			// and finally, render each item
			.flatMap(async *v => {
				if (this.shouldYieldA(v)) yield await this.renderA(v, {signal})
				if (this.shouldYieldB(v)) yield await this.renderB(v, {signal})
				if (this.shouldYieldC(v)) yield* this.renderAllCs(v, {signal})
				if (this.shouldYieldD(v)) yield await this.renderD(v, {signal})
				if (this.shouldYieldE(v)) yield await this.renderE(v, {signal})
				if (this.shouldYieldF(v)) yield await this.renderF(v, {signal})
				if (this.shouldYieldG(v)) yield await this.renderG(v, {signal})
				// ...
			})
			.join("")
	}
}

If using a generator function literal, you'd have to save this, and in this case it could lie significantly away from the code as well:

class Visitor {
	// ...

	render(items, {signal}) {
		const self = this
		return items
			.filter(v => v.isRenderable)
			// apply however many more conditions and transformations to each
			// `v` as needed
			// and finally, render each item
			.flatMap(async function *(v) {
				if (self.shouldYieldA(v)) yield await self.renderA(v, {signal})
				if (self.shouldYieldB(v)) yield await self.renderB(v, {signal})
				if (self.shouldYieldC(v)) yield* self.renderAllCs(v, {signal})
				if (self.shouldYieldD(v)) yield await self.renderD(v, {signal})
				if (self.shouldYieldE(v)) yield await self.renderE(v, {signal})
				if (self.shouldYieldF(v)) yield await self.renderF(v, {signal})
				if (self.shouldYieldG(v)) yield await self.renderG(v, {signal})
				// ...
			})
	}
}

And of course, a helper method could also be used here, but that's tantamount to const self = this on steroids:

class Visitor {
	// ...

    async *renderItem(v, {signal}) {
        if (this.shouldYieldA(v)) yield await this.renderA(v, {signal})
        if (this.shouldYieldB(v)) yield await this.renderB(v, {signal})
        if (this.shouldYieldC(v)) yield* this.renderAllCs(v, {signal})
        if (this.shouldYieldD(v)) yield await this.renderD(v, {signal})
        if (this.shouldYieldE(v)) yield await this.renderE(v, {signal})
        if (this.shouldYieldF(v)) yield await this.renderF(v, {signal})
        if (this.shouldYieldG(v)) yield await this.renderG(v, {signal})
        // ...
    }

	async render(items, {signal}) {
		return items
			.filter(v => v.isRenderable)
			// apply however many more conditions and transformations to each
			// `v` as needed
			// and finally, render each item
			.flatMap(v => this.renderItem(v, {signal}))
			.join("")
	}
}

Contrast this with using arrays to similar effect. While it's technically similarly concise and even lets you continue to use this, it ends up hard to read, and if this.renderAllCs(v) could generate a large structure, it also could cause a performance problem:

I've personally worked on a code base where this kind of spreading was a particularly common idiom, including within a few classes. These were all synchronous cases mind you.

class Visitor {
	// ...

	render(items, {signal}) {
		return items
			.filter(v => v.isRenderable)
			.flatMap(async v => [
				...(this.shouldYieldA(v) ? [await this.renderA(v)] : []),
				...(this.shouldYieldB(v) ? [await this.renderB(v)] : []),
				...(this.shouldYieldC(v) ? await this.renderAllCs(v).toArray() : []),
				...(this.shouldYieldD(v) ? [await this.renderD(v)] : []),
				...(this.shouldYieldE(v) ? [await this.renderE(v)] : []),
				...(this.shouldYieldF(v) ? [await this.renderF(v)] : []),
				...(this.shouldYieldG(v) ? [await this.renderG(v)] : []),
				// ...
			})
	}
}

And of course, just for clarity, here's the above not using any of the various helper methods:

class Visitor {
	// ...

	async *render(items) {
        let result = ""

        for await (const v of items) {
            if (v.isRenderable) {
				// apply however many more conditions and transformations to each
				// `v` as needed - nesting related to that is omitted for brevity
				// and finally, render each item
				if (this.shouldYieldA(v)) yield this.renderA(v)
				if (this.shouldYieldB(v)) yield this.renderB(v)
				if (this.shouldYieldC(v)) yield* this.renderAllCs(v)
				if (this.shouldYieldD(v)) yield this.renderD(v)
				if (this.shouldYieldE(v)) yield this.renderE(v)
				if (this.shouldYieldF(v)) yield this.renderF(v)
				if (this.shouldYieldG(v)) yield this.renderG(v)
            }
        }
	}
}

Use cases that aren't easily serviceable with either the iterable helpers or adding utility methods are fairly uncommon nowadays, but they do exist.

[dead-claudia]

@hax re: #1 (comment)

Nit: since endpoint is necessarily newly duplicated in your replacement code, it's hardly more of an argument than saying const self = this makes arrow functions redundant. You're doing the equivalent of const innerEndpoint = endpoint by passing it via a parameter.

I personally would've inlined the generator, which would've made it more immediately obvious of a use case:

class FooClient {
  constructor() {}

  async get<T>(endpoint: String): Promise<T> {
    const response = await fetch(endpoint);
    return await response.json();
  }

  async getAll<T>(): Promise<CountableGenerator<T>> {
    const ids = (await this.get(`/api/items`)).map((item) => item.id);
    const self = this // <-- This is needed

    return {
      count: ids.length,
      generator: async function *() {
        for (let id of ids) {
            yield self.get<T>(`/api/items/${id}`); // I would like to use `this` directly instead of `self`
        }
      }
    }
  }
}

Of course, unlike my Visitor, it'll become less compelling if the iterator helpers proposal hits stage 4:

class FooClient {
  constructor() {}

  async get<T>(endpoint: String): Promise<T> {
    const response = await fetch(endpoint);
    return await response.json();
  }

  async getAll<T>(): Promise<CountableGenerator<T>> {
    const ids = (await this.get(`/api/items`)).map((item) => item.id);

    return {
      count: ids.length,
      generator: () =>
        AsyncIterator.from(ids)
          .map(id => this.get(`/api/items/${id}`))
      }
    }
  }
}

[James-E-A]

Could be appropriate when writing "thin wrappers" around objects which are semantically Iterators but don't literally implement the Symbol.iterator interface, e.g. line 20 here:

/**
  * Convert any XPathResult object into its corresponding Javascript object.
  * @param {XPathResult} xpr
  * @returns {?Element|Array.<Element>|Iterable.<Element>|?number|?string|?boolean}
  */
export function xpathresultToNativeType(xpr) {
	switch (xpr.resultType) {
		case XPathResult.UNORDERED_NODE_SNAPSHOT_TYPE:
		case XPathResult.ORDERED_NODE_SNAPSHOT_TYPE:
			// XPath Snapshot, as JS Array
			return Array.from(
				// TODO benchmark further
				{ length: xpr.snapshotLength },
				(_0, i) => xpr.snapshotItem(i)
			);

		case XPathResult.UNORDERED_NODE_ITERATOR_TYPE:
		case XPathResult.ORDERED_NODE_ITERATOR_TYPE:
			// XPath Iterator, as JS Generator
			return (function* () {
				// TODO benchmark
				for (
					var value = xpr.iterateNext() ;
					value ;
					value = xpr.iterateNext()
				) {
					yield value;
				}
			})();

		case XPathResult.ANY_UNORDERED_NODE_TYPE:
		case XPathResult.FIRST_ORDERED_NODE_TYPE:
			// Single node value
			return xpr.singleNodeValue || null;

		case XPathResult.NUMBER_TYPE:
			return xpr.numberValue ?? null;

		case XPathResult.STRING_TYPE:
			return xpr.stringValue ?? null;

		case XPathResult.BOOLEAN_TYPE:
			return xpr.booleanValue ?? null;
	}
}