Nodejs Array Queue and forEach Application Detailed Explanation

This paper mainly records the problems and solutions caused by array characteristics in the development of Nodejs, as well as the flexible application of array.

The code test results in this paper are all based on node v 6.9. 5

Arrays and queues

The queue FIFO feature is implemented using the array object methods push/shift, such as:

>a=[]
[]
>a.push(2.3.4)
3
>a.push(2)
3
>a
[2.3.4.2]
>a.shift()
2
>a
>[3.4.2]

Arrays and forEach

There are two common ways to delete arrays: delete and using splice methods, and it is necessary to make clear the difference between them.

操作/方法	说明
splice	删除并返回指定的数组元素，数组本身长度会改变；但不会free元素对象
delete	删除（free）元素对象，数组元素不变，值变为undefined

If you want to completely delete an element from the array, use splice:

> a=[1,2,3]
[ 1, 2, 3 ]
> a.splice(1,1)
[ 2 ]
> a
[ 1, 3 ]
> a.length
2
> a.forEach(function(item, index){console.info("index[", index,"]:", item)});
index[ 0 ]: 1
index[ 1 ]: 3
undefined
> 

So, what is the effect of executing forEach when an element object is deleted using delete?

forEach Handling Mechanism for Arrays with Empty Elements

The test results are as follows

> a=[1,2,3]
[ 1, 2, 3 ]
> delete a[1]
true
> a
[ 1, , 3 ]
> a.length
3
> a.forEach(function(item, index){console.info("index[", index,"]:", item)});
index[ 0 ]: 1
index[ 2 ]: 3
undefined

From the test results, forEach does not traverse which item has the value of undefined. How to judge whether forEach is over or not is a big challenge in practical application.

Solve the application of asynchronous characteristics of forEach, and add effective data of prototype self-management for array;

The effect is as follows:

> a=[1,2,3]
[ 1, 2, 3 ]
> a.validnum=3
3
> delete a[2]
true
> a.validnum=2
2
> a
[ 1, 2, , validnum: 2 ]
> a.length
3
> a.validnum
2
> a.forEach(function(item, index){console.info("index[", index,"]:", item)});
index[ 0 ]: 1
index[ 1 ]: 2
undefined
> 

Added: Node. js Array forEach Synchronization Context Statement

Accustomed to the thinking mode of C language department, I just came into contact with Node. js, and its asynchronous processing made my head big.

When writing code, you encounter such a scenario, and you need to process the elements in an array in a loop, and then perform an last operation after all the processing is completed. But the asynchronous nature of JS makes this last statement execute first, so take a moment to study forEach.

Talk is cheap. Show me the code.

Usage of forEach

forEach is used to traverse the array structure, and some people say that the bottom layer of forEach is implemented with for, without further study, at least the effect is 1. The three parameters of the forEach callback function are: value, sequence number and original array. Serial numbers start at 0.

(() => {
  let arr = [2, 3, 1];
  arr.forEach(function (value, index, array) {
    console.log(value);
    console.log(index);
    console.log(array);
    console.log('-----');
  });
})();

Output

2
0
[ 2, 3, 1 ]
-----
3
1
[ 2, 3, 1 ]
-----
1
2
[ 2, 3, 1 ]
-----

From the result, forEach loops are synchronized, that is to say, they are all executed in sequence. But the thought that it is JS makes it impossible to synchronize. . Can verify under 1.

forEach asynchronously handles multiple loops

This time, a timing task is added to forEach, and each cycle operation delays the relevant time of value, simulating the time-consuming operation.

(() => {
  let arr = [2, 3, 1];
  arr.forEach(function (value, index, array) {
    setTimeout(function () {
      console.log(value);
    }, value*100);
  });
})();

Output

1
2
3

It can be seen from the results that the task that takes the shortest time is completed first, and the tasks of each loop are not executed in the order of loops, that is to say, multiple loops are processed asynchronously.

The forEach context is also executed asynchronously

Back to the question at the beginning, and regardless of whether multiple loops are executed in sequence or not, I need to execute a piece of data after all tasks in forEach are completed to inform me that all tasks are completed.

(() => {
  let arr = [2, 3, 1];
  arr.forEach(function (value, index, array) {
    setTimeout(function () {
      console.log(value);
    }, value*100);
  });
  console.log('All the work is done');
})();

Output

All the work is done
1
2
3

As a result, the contextual statements are not synchronized, and the tasks in the forEach loop are notified that all tasks are completed before they are completed, which obviously does not meet expectations.

Read a lot of blog for this problem, did not find a suitable solution, finally can only think of Promise. all to reluctantly achieve this function.

Promise. all Implementation of forEach Context Statement Synchronization

Change the above code to the structure of Promise. all. resolve () is called at the end of each loop. We know the then function of Promise. all, which will only be triggered when all Promise are executed, which seems to meet our needs.

> a=[1,2,3]
[ 1, 2, 3 ]
> a.splice(1,1)
[ 2 ]
> a
[ 1, 3 ]
> a.length
2
> a.forEach(function(item, index){console.info("index[", index,"]:", item)});
index[ 0 ]: 1
index[ 1 ]: 3
undefined
> 

0

Output

> a=[1,2,3]
[ 1, 2, 3 ]
> a.splice(1,1)
[ 2 ]
> a
[ 1, 3 ]
> a.length
2
> a.forEach(function(item, index){console.info("index[", index,"]:", item)});
index[ 0 ]: 1
index[ 1 ]: 3
undefined
> 

1

From the results, it meets our needs.

Possible remaining problems

Thinking of the asynchronous nature of JS, it suddenly occurred that there might be a problem with this method.

Here, every time forEach just enters, the Promise array is assigned. This operation time should be very short, and the last Promise. all statement is called after the assignment is completed three times.

However, if the array is very large and the loop assignment is very time consuming, if only one and a half assignment operations are completed, the Promise array passed in when the last Promise. all is executed may not be the array containing all Promise.

In this case, Promise. all waits for only one and a half operations. When Promise. all waits, Promise assigned to the back of this array does not know whether it will be waited.

Just contact JS do not understand the implementation mechanism, can only experiment to verify whether there is this problem under 1. Next, make this array one bigger. Please forgive me for making it bigger in the most stupid way.

> a=[1,2,3]
[ 1, 2, 3 ]
> a.splice(1,1)
[ 2 ]
> a
[ 1, 3 ]
> a.length
2
> a.forEach(function(item, index){console.info("index[", index,"]:", item)});
index[ 0 ]: 1
index[ 1 ]: 3
undefined
> 

2

After testing, when the array length is 2 ^ 18 * 10 on my computer, Promise reports an error RangeError: Too many elements passed to Promise. all.

When the array length is 2 ^ 17 * 10, that is, 2621440, it will work normally. Tested several times, the last All the work is done to execute the command output is always the last output (node xx. js is used because the terminal buffer is too small > log. txt redirects the output to a file for viewing).

Of course, there will not be such a large array in the application. From the result, there is no possible problem in the practical application.

That is to say, Promise context statement synchronization can be realized with Promise. all.