Serial and Parallel in JavaScript Asynchronous Operation
- 2021-12-09 08:08:44
- OfStack
1. Preface
Write this article under 1
js
Medium
es5
And
es6
Schemes for asynchronous functions, serial execution and parallel execution. Examples have been used in combination with serial and parallel.
2. es5 mode
Before es6 came out, the community
nodejs
For callback hell, there is already
promise
Plan. If there are multiple asynchronous functions, how to arrange the execution sequence, how to execute all asynchronous functions faster, and then execute the next step? Here, the serial execution and parallel execution of js appear.
3. Serial execution of asynchronous functions
var items = [ 1, 2, 3, 4, 5, 6 ];
var results = [];
function async(arg, callback) {
console.log(' Parameter is ' + arg +' , 1 Returns results in seconds ');
setTimeout(function () { callback(arg * 2); }, 1000);
}
function final(value) {
console.log(' Finish : ', value);
}
function series(item) {
if(item) {
async( item, function(result) {
results.push(result);
return series(items.shift());// Recursively execute all the data
});
} else {
return final(results[results.length - 1]);
}
}
series(items.shift());
4. Parallel execution of asynchronous functions
The above functions are executed one by one, and the last one is executed and then the next one is executed, similar
es6
async and await in (es5 later collectively referred to as es6) are there any similarities
promise.all
What about this, all parallel execution?
It can be written as follows:
var items = [ 1, 2, 3, 4, 5, 6 ];
var results = [];
function async(arg, callback) {
console.log(' Parameter is ' + arg +' , 1 Returns results in seconds ');
setTimeout(function () { callback(arg * 2); }, 1000);
}
function final(value) {
console.log(' Finish : ', value);
}
items.forEach(function(item) {// Loop completion
async(item, function(result){
results.push(result);
if(results.length === items.length) {// Judging whether the number of finished executions is equal to the number of functions to be executed
final(results[results.length - 1]);
}
})
});
5. Combination of serial execution and parallel execution of asynchronous functions
If many pieces of asynchronous data (hundreds of pieces) are executed in parallel, and there are many (https) request data in each asynchronous data, it will inevitably lead to insufficient number of tcp connections, or numerous call stacks piled up, resulting in memory overflow. Therefore, it is not easy to execute too much data in parallel, so there is a combination of parallel and serial.
The code can be written as follows:
var items = [ 1, 2, 3, 4, 5, 6 ];
var results = [];
var running = 0;
var limit = 2;
function async(arg, callback) {
console.log(' Parameter is ' + arg +' , 1 Returns results in seconds ');
setTimeout(function () { callback(arg * 2); }, 1000);
}
function final(value) {
console.log(' Finish : ', value);
}
function launcher() {
while(running < limit && items.length > 0) {
var item = items.shift();
async(item, function(result) {
results.push(result);
running--;
if(items.length > 0) {
launcher();
} else if(running == 0) {
final(results);
}
});
running++;
}
}
launcher();
6. es6 mode
es6
Naturally, it comes with serial and parallel execution modes. For example, serial can be used
async
And
await
(As explained earlier), you can use promise. all and so on for parallelism. Then for the combination of serial and parallel, the restriction
es50
Concurrent number, the community also has 1 scheme, for example
tiny-async-pool , es6-promise-pool , p-limit
Simple package 1
es50
Concurrency limit solution function
function PromiseLimit(funcArray, limit = 5) { // Concurrent execution 5 Bar data
let i = 0;
const result = [];
const executing = [];
const queue = function() {
if (i === funcArray.length) return Promise.all(executing);
const p = funcArray[i++]();
result.push(p);
const e = p.then(() => executing.splice(executing.indexOf(e), 1));
executing.push(e);
if (executing.length >= limit) {
return Promise.race(executing).then(
() => queue(),
e => Promise.reject(e)
);
}
return Promise.resolve().then(() => queue());
};
return queue().then(() => Promise.all(result));
}
Use:
// Test code
const result = [];
for (let index = 0; index < 10; index++) {
result.push(function() {
return new Promise((resolve, reject) => {
console.log(" Begin " + index, new Date().toLocaleString());
setTimeout(() => {
resolve(index);
console.log(" End " + index, new Date().toLocaleString());
}, parseInt(Math.random() * 10000));
});
});
}
PromiseLimit(result).then(data => {
console.log(data);
});
Modify the test code and add random failure logic
// Modify the test code Random failure or success
const result = [];
for (let index = 0; index < 10; index++) {
result.push(function() {
return new Promise((resolve, reject) => {
console.log(" Begin " + index, new Date().toLocaleString());
setTimeout(() => {
if (Math.random() > 0.5) {
resolve(index);
} else {
reject(index);
}
console.log(" End " + index, new Date().toLocaleString());
}, parseInt(Math.random() * 1000));
});
});
}
PromiseLimit(result).then(
data => {
console.log(" Success ", data);
},
data => {
console.log(" Failure ", data);
}
);
7. async and await combined with promise all
async function PromiseAll(promises,batchSize=10) {
const result = [];
while(promises.length > 0) {
const data = await Promise.all(promises.splice(0,batchSize));
result.push(...data);
}
return result;
}
There are two problems with writing this way:
1. When callingPromise.all
It was already created before
promises
, actually
promise
Has been implemented
2. Your implementation must wait ahead
batchSize个promise resolve
In order to run the next batch
batchSize
A, that is,
es50
All success is enough.
The improvements are as follows:
async function asyncPool(array,poolLimit,iteratorFn) {
const ret = [];
const executing = [];
for (const item of array) {
const p = Promise.resolve().then(() => iteratorFn(item, array));
ret.push(p);
if (poolLimit <= array.length) {
const e = p.then(() => executing.splice(executing.indexOf(e), 1));
executing.push(e);
if (executing.length >= poolLimit) {
await Promise.race(executing);
}
}
}
return Promise.all(ret);
}
Use:
const timeout = i => new Promise(resolve => setTimeout(() => resolve(i), i));
return asyncPool( [1000, 5000, 3000, 2000], 2,timeout).then(results => {
...
});