Detailed explanation of asynchronous generator and asynchronous iteration in Node. js
- 2021-10-25 05:48:15
- OfStack
Preface
The appearance of the generator function in JavaScript predates the introduction of async/await, which means that while creating asynchronous generators (generators that always return Promise and can be await), many considerations are also introduced.
Today, we will study asynchronous generators and their close relatives-asynchronous iteration.
Note: While these concepts should apply to all javascript that follow modern specifications, all of the code in this article was developed and tested for Node. js releases 10, 12, and 14.
Asynchronous generator function
Look at this applet:
// File: main.js
const createGenerator = function*(){
yield 'a'
yield 'b'
yield 'c'
}
const main = () => {
const generator = createGenerator()
for (const item of generator) {
console.log(item)
}
}
main()
This code defines a generator function, creates a generator object with this function, and then iterates through the generator object with for... of. Quite standard stuff-although you would never use a generator to handle such trivial things in real life. If you're not familiar with generators and for... of loops, see the articles "Javascript generators" and "ES 6 loops and iterative objects". Before using asynchronous generators, you need to have a solid understanding of generators and for... of loops.
Suppose we want to use await in a generator function, and Node. js supports this functionality whenever we need to declare the function with the async keyword. If you are not familiar with asynchronous functions, see "Writing asynchronous tasks in modern JavaScript" 1.
Let's modify the program and use await in the generator.
// File: main.js
const createGenerator = async function*(){
yield await new Promise((r) => r('a'))
yield 'b'
yield 'c'
}
const main = () => {
const generator = createGenerator()
for (const item of generator) {
console.log(item)
}
}
main()
Again, in practice, you wouldn't do this--you might have await from a third-party API or library function. In order to make it easy for everyone to master, our examples should be kept as simple as possible.
If you try to run the above program, you will encounter problems:
$ node main.js
/Users/alanstorm/Desktop/main.js:9
for (const item of generator) {
^
TypeError: generator is not iterable
JavaScript tells us that this generator is "non-iterative". At first glance, it seems that making a generator function asynchronous also means that the generator it generates is non-iterative. This is a bit confusing, because the purpose of the generator is to generate objects that can be iterated "programmatically".
Next, find out what happened.
Check generator
If you look at the iterative objects of Javascript generator [1]. When an object has an next method, the object implements the iterator protocol, and the next method returns an object with an value attribute, one of the done attributes, or both value and done attributes.
If you use the following code to compare the generator objects returned by asynchronous generator functions with those returned by regular generator functions:
// File: test-program.js
const createGenerator = function*(){
yield 'a'
yield 'b'
yield 'c'
}
const createAsyncGenerator = async function*(){
yield await new Promise((r) => r('a'))
yield 'b'
yield 'c'
}
const main = () => {
const generator = createGenerator()
const asyncGenerator = createAsyncGenerator()
console.log('generator:',generator[Symbol.iterator])
console.log('asyncGenerator',asyncGenerator[Symbol.iterator])
}
main()
You will see that the former does not have the Symbol. iterator method, while the latter does.
$ node test-program.js
generator: [Function: [Symbol.iterator]]
asyncGenerator undefined
Both generator objects have one next method. If you modify the test code to call this next method:
// File: test-program.js
/* ... */
const main = () => {
const generator = createGenerator()
const asyncGenerator = createAsyncGenerator()
console.log('generator:',generator.next())
console.log('asyncGenerator',asyncGenerator.next())
}
main()
You see another problem:
$ node test-program.js
generator: { value: 'a', done: false }
asyncGenerator Promise { <pending> }
To make objects iterable, the next method needs to return objects with value and done attributes. An async function will always return an Promise object. This feature is applied to generators created with asynchronous functions--these asynchronous generators always have an yield 1 Promise object.
This behavior prevents the generator of the async function from implementing the javascript iterative protocol.
Asynchronous iteration
Fortunately, there is a way to solve this contradiction. If you look at 1, look at the constructor or class returned by the async generator
// File: test-program.js
/* ... */
const main = () => {
const generator = createGenerator()
const asyncGenerator = createAsyncGenerator()
console.log('asyncGenerator',asyncGenerator)
}
You can see that it is an object whose type or class or constructor is AsyncGenerator instead of Generator:
asyncGenerator Object [AsyncGenerator] {}
Although the object may not be iterative, it is asynchronous iterative.
For an object to be able to iterate asynchronously, it must implement an Symbol. asyncIterator method. This method must return an object that implements the asynchronous version of the iterator protocol. That is, the object must have an next method that returns Promise, and this promise must eventually resolve to an object with done and value attributes.
1 AsyncGenerator object satisfies all these conditions.
This leaves a question--how can we traverse an object that is not iterable but can be iterated asynchronously?
for await … of cycle
Asynchronous iterative objects can be iterated manually using only the next method of the generator. (Note that the main function here is now async main--this allows us to use await inside the function.)
// File: main.js
const createAsyncGenerator = async function*(){
yield await new Promise((r) => r('a'))
yield 'b'
yield 'c'
}
const main = async () => {
const asyncGenerator = createAsyncGenerator()
let result = {done:false}
while(!result.done) {
result = await asyncGenerator.next()
if(result.done) { continue; }
console.log(result.value)
}
}
main()
However, this is not the most direct circulation mechanism. I don't like the loop condition of while, nor do I want to manually check result. done. In addition, the result. done variable must exist within the scope of both inner and outer blocks.
Fortunately, most (maybe all?) javascript implementations that support asynchronous iterators also support the special for await... of loop syntax. For example:
// File: main.js
const createGenerator = async function*(){
yield await new Promise((r) => r('a'))
yield 'b'
yield 'c'
}
const main = () => {
const generator = createGenerator()
for (const item of generator) {
console.log(item)
}
}
main()
If you run the above code, you will see that the asynchronous generator and iterable object have been successfully looped, and the fully resolved value of Promise is obtained in the loop body.
$ node main.js
a
b
c
This for await... of loop prefers objects that implement the asynchronous iterator protocol. But you can use it to traverse any kind of iterative object.
// File: main.js
const createGenerator = async function*(){
yield await new Promise((r) => r('a'))
yield 'b'
yield 'c'
}
const main = () => {
const generator = createGenerator()
for (const item of generator) {
console.log(item)
}
}
main()
When you use for await, Node. js will first look for the Symbol. asyncIterator method on the object. If it cannot be found, it rolls back to the method using Symbol. iterator.
Nonlinear code execution
Like await 1, the for await loop introduces nonlinear code execution into the program. In other words, your code will run in a different order from the code you wrote.
The first time your program encounters the for await loop, it calls next on your object.
This object will be yield 1 promise, then code execution will leave your async function, and your program will continue to execute outside that function.
1 Once your promise is resolved, code execution will use this value to return to the loop body.
Node. js will call next on the object when the loop ends and the next 1 trip is made. This call will produce another promise, and code execution will leave your function again. Repeat this pattern until Promise resolves to an object where done is true, and then continue code execution after the for await loop.
The following example can illustrate one point:
// File: main.js
const createGenerator = async function*(){
yield await new Promise((r) => r('a'))
yield 'b'
yield 'c'
}
const main = () => {
const generator = createGenerator()
for (const item of generator) {
console.log(item)
}
}
main()
In this code, you use numbered logging statements, which allow you to track its execution. As an exercise, you need to run the program yourself and see what the execution results are.
If you don't know how it works, it will confuse the execution of the program, but asynchronous iteration is indeed a powerful technology.