JS Array Deduplication Details

  • 2021-12-04 18:01:17
  • OfStack

Directory 1 test cases 2 JS Array Deduplication 4 Major Types 2.1 Element Comparison Type 2.1. 1 Bilayer for Loop 1-by-1 Comparison (es5 Commonly Used) 2.1. 2 Sort Adjacent Comparison 2.2 Find Element Position Type 2.2. 1 indexOf2.2. 2 findIndex2.3 Element Existence Type 2.3. 1 includes2.3. 2 some2.4 Depending on Data Structure Features 2.4. 1 Map2.4. 2 Set (ES6 Most Commonly Used) 3 Supplement

1 test cases


//  Test case 
const a = {};
const b = { c: 1 };
const array = [
  1, 1, "1", "1",
  {}, {}, { c: 1 }, { c: 1},
  a, a, b, b, 
  [], [], [1], [1],
  undefined, undefined,
  null, null,
  NaN, NaN,
];

2 JS array deduplication 4 major types

2.1 Element comparison type

This type is de-duplicated by comparing between array elements

2.1. 1 Bilayer for Cycle 1-by-1 Comparison (commonly used in es5)

Compare array elements one by one using a two-tier for loop, using the splice Method to remove duplicate elements


//  Double layer for Cycle 
function uniq1(arr) {
    for (let i = 0; i < arr.length; i++) {
        for (let j = i + 1; j < arr.length; j++) {
            if (arr[i] === arr[j]) {
                arr.splice(j, 1)
                j--
            }
        }
    }
    return arr
}

//  Deduplication result 
// [1,'1',{},{},{c:1},{c:1},{},{c:1},[],[],[1],[1],undefined,null,NaN,NaN]

By comparing the results before and after deduplication, repeated NaN Didn't get rid of it, because NaN === NaN For false

2.1. 2 Sort Adjacent Comparison

Use sort() Method to sort the elements of the array, and then compare adjacent elements, using the splice Method to remove duplicate elements.


function uni2(arr) {
    arr.sort();
    for (let i = 0; i < arr.length - 1; i++) {
        arr[i] === arr[i + 1] && arr.splice(i + 1, 1) && i--;
    }
    return arr;
}


You can also create a new array and put non-repeating elements into the new array


function uniq3(arr) {
    arr = arr.sort()
    const newArr = [arr[0]]
    for (let i = 1; i < arr.length; i++) {
        if (arr[i] !== arr[i - 1]) {
            newArr.push(arr[i])
        }
    }
    return newArr
}

//  Deduplication result 
// [[],[],1,'1',[1],[1],NaN,NaN,{},{},{c:1},{c:1},{},{c:1},null,undefined]

Repetitive NaN Didn't get rid of it, because NaN === NaN为false
sort
The default sort order is to convert elements to strings and objects to strings [object Object] , so sort Method can't sort the objects in the array, and it is possible that duplicate objects can't be removed unless duplicate objects are inherently adjacent

2.2 Find Element Position Type

This type removes duplication by finding the position where the element occurs for the first time

2.2.1 indexOf

Pass NaN0 Find out if the first occurrence of the current element is the current position, and if so, put a new array


function uniq4(arr) {
    let res = []
    for (let i = 0; i < arr.length; i++) {
        if (arr.indexOf(arr[i]) === i) {
            res.push(arr[i])
        }
    }
    return res
}

//  Deduplication result 
// [1,'1',{},{},{c:1},{c:1},{},{c:1},[],[],[1],[1],undefined,null]

Again, because NaN === NaN For false , so use NaN0 Find NaN The result is always-1, so there will be no NaN in the new array

2.2.2 findIndex

Pass findIndex Find out if the first occurrence of the current element is the current position, and if so, put a new array


function uniq5(arr) {
    let res = []
    for (let i = 0; i < arr.length; i++) {
        if (arr.findIndex(item => item === arr[i]) === i) {
            res.push(arr[i])
        }
    }
    return res
}
//  Deduplication result 
// [1,'1',{},{},{c:1},{c:1},{},{c:1},[],[],[1],[1],undefined,null]


Again, because NaN === NaN For false , so use findIndex Find NaN The result is always-1, so there is no NaN

2.3 Does the Element Exist Type

This type deduplicates by determining whether the current element exists in the new array

2.3.1 includes

includes Method is used to determine whether an array contains a specified value


function uniq6(arr) {
    let res = []
    for (let i = 0; i < arr.length; i++) {
        if (!res.includes(arr[i])) {
            res.push(arr[i])
        }
    }
    return res
}
//  Deduplication result 
// [1,'1',{},{},{c:1},{c:1},{},{c:1},[],[],[1],[1],undefined,null,NaN]


includes The zero-value equality algorithm is used to determine whether a given element is found, so it is possible to determine whether NaN exists in the new array

2.3.2 some

some Method is used to test whether at least 1 element in the array passes the provided function test


function uniq7(arr) {
    let res = []
    for (let i = 0; i < arr.length; i++) {
        if (!res.some(item => item === arr[i])) {
            res.push(arr[i])
        }
    }
    return res
}
//  Deduplication result 
// [1,'1',{},{},{c:1},{c:1},{},{c:1},[],[],[1],[1],undefined,null,NaN,NaN]


Again, it is still used here === To compare elements, because NaN === NaN For false Therefore, there will be multiple in the new array NaN

2.4 Depending on the characteristics of data structure

This type uses the data structure provided by ES6 Map , Set Intrinsic non-repeatable characteristics to remove duplication

2.4.1 Map

false0 The provided Map structure can use various types of values (including objects) as keys, and the keys are only 1


function uniq8(arr) {
    const map = new Map()
    for (let i = 0; i < arr.length; i++) {
        !map.has(arr[i]) && map.set(arr[i], true)
    }
    return [...map.keys()]
}
//  Deduplication result 
// [1,'1',{},{},{c:1},{c:1},{},{c:1},[],[],[1],[1],undefined,null,NaN]


false1 Method pair NaN Also effective

2.4. 2 Set (ES6 most commonly used)

Set The values of members of the structure are only 1, and there are no duplicate values.


function uniq9(arr) {
    return [...new Set(arr)]
}

//  Deduplication result 
// [1,'1',{},{},{c:1},{c:1},{},{c:1},[],[],[1],[1],undefined,null,NaN]

3 Supplements

The method described above can use different Api Make changes, such as using splice Method to remove array elements, we can use the filter Method to filter the array to get a new array;

Another example is includes You don't use it in the method of for Loop through the array, passing through reduce Method to replace and so on.

In a word, there are many methods, but they are always the same

Some deduplication methods are right NaN Invalid because NaN === NaN For false If there is a need, you can use the Object.is(NaN, NaN) For true To make modifications

In practical applications, the most commonly used method is to use Set, and you can also use the third-party library lodash To deal with


Related articles: