Summary of JavaScript array deduplication methods (with code)

This article brings you a summary of the method of deduplicating JavaScript arrays (with code). It has certain reference value. Friends in need can refer to it. I hope it will be helpful to you.

Array deduplication is still used frequently in daily development, and it is also a topic that can be easily caught on the Internet. Therefore, the purpose of writing this article is to summarize and summarize. Since many people are How much do you know about the mentioned array deduplication? Or if you encounter the need to remove duplicates during development, can you think of a better solution?

This time we will figure out how to do the most appropriate deduplication of arrays. We must consider not only compatibility, but also performance and code elegance.

My learning path is to imitate the learning method of Yan Yu (github: mqyqingfeng). I am grateful to outstanding people like Yan Yu for taking the lead. I don’t want to just do it, so I will practice more and output more. I hope that in the future To be able to make my own way.

1. Getting Started Solution

function unique(origin) {
    var result = [];
    for(var i = 0; i < origin.length; i++) {
        var arrayItem = origin[i];

        for(var j= 0; j< result.length; j++) {
            var resultItem = result[j];
            
            // 如果在结果数组循环中找到了该元素，则跳出循环，进入下一个源数组元素的判断
            if(resultItem === arrayItem) {
                break;
            }
        }
        
        // 如果把结果数组循环完都没有找到该元素，就将该元素压入结果数组中
        if(j === result.length) {
            result.push(arrayItem);
        }
    }
    return result;
}

var array = [&#39;a&#39;, &#39;b&#39;, &#39;c&#39;, &#39;1&#39;, 0, &#39;c&#39;, 1, &#39;&#39;, 1, 0];
console.log(unique(array));  // ["a", "b", "c", "1", 0, 1, ""]

The above code is the simplest way to implement array deduplication. The advantage is that it has excellent compatibility. The disadvantage is that it requires two for loops and the time complexity is O(n ^2), poor performance.

2. The indexOf attribute of the array

The indexOf attribute in the array is the ES5 specification. Only IE8 and earlier versions do not support this method. Relatively speaking, if you don't need to be compatible with IE8, try to use indexOf to determine whether an element is in the array.

function unique(origin){
    var result = [];
    for(var i = 0; i< origin.length; i++) {
        var item = origin[i];
        if(result.indexOf(item) === -1) {
            result.push(item);
        }
    }
    return result;
}

3. The filter attribute of the array

The filter() method of the array creates a new array. The elements in the new array are checked for all elements in the specified array that meet the conditions.

filter's callback has three parameters, the third parameter is the array object to which the current element belongs, so we can continue to use the indexOf attribute.

function unique(origin) {
    var result = origin.filter(function (item, index, array){
        // 获取元素在源数组的位置，只返回那些索引等于当前元素索引的值。
        return array.indexOf(item) === index;
    });
    return result;
}

filter is compatible with IE9. This method does not have a for loop and mainly uses the filter and indexOf attributes, so the code is relatively elegant.

4. Using the key value of Object

function unique(origin) {
    var result = [];
    var hashTable = {};
    for(var i = 0; i< origin.length; i++) {
        // 如果键对应的值，为真，意味着对象的键中已经有重复的键了。
        if(!hashTable[origin[i]]) {
        // 将元素作为对象的键，默认键对应的值为 true, 
            hashTable[origin[i]] = true;
            
            // 如果对象中没有这个键，则将这个元素放入结果数组中去。
            result.push(origin[i]);
        }
    }
    return result;
}

The event complexity of this solution is O(n), but the key of the object defaults to the string type. What does this mean? The number 1 and the string '1' are equal in the key, so the above method is not suitable for deduplication of mixed strings and numbers.

So we also put the type of the element into the key of the object:

function unique(origin) {
    var result = [];
    var hashTable = {};
    for(var i = 0; i< origin.length; i++) {
        var current = origin[i];
        // 字符串拼接元素的类型和元素
        var key = typeof(current) + current;
        if(!hashTable[key]) {
            hashTable[key] = true;
            result.push(current);
        }
    }
    return result;
}

5. Sort method of array

function unique(origin) {
    return origin.concat.sort().filter(function(item, index, array) {
        // !index 表示第 0 个元素应该被返回。
        return !index || item !== origin[index-1]
    })
}

function unique(array) {
    array.sort(); // 排序字符串
    array.sort(function(a, b) {
        return a-b; // 排序数字
    })
    
    for(let i=0; i<array.length; i++) {
        if(array[i] === array[i+1]) {
            array.splice(i, 1);
            i--; // 应该将前一个数删除，而不是删除后一个数。是因为元素被删除之后，后面元素的索引会迁移，所以要 i--;
        }
    }
    return array;
}

The advantage of the sort method is that it uses sorting and returns The next element is not equal to the previous one. Relatively simple and intuitive. The disadvantage is that it changes the original sorting position of the elements.

6. ES6 Set

ES6 provides a new data structure Set, which is similar to an array, but the values ​​of the members are unique and there are no duplicate values. When adding a value to the Set, no type conversion occurs, so 5 and '5' are two different values. Set internally determines whether two values ​​are the same, using an algorithm similar to "===", but the difference is that NaN is considered equal to NaN internally;

Set can be converted to an array, so it is easy to implement Deduplication

function unique(origin) {
    return Array.from(new Set(origin));
}

7. ES6 Map

ES6 adds new Map data results, and the previous object key value solution can be easily optimized through the has and set methods.

function unique(origin){
    const map = new Map()
    return origin.filter((item) => !map.has(item) && map.set(item, true))
}

8. Type judgment

Some common data types are === and indexOf which cannot be detected. For example:

console.log({} === {})  // false;

console.log(NaN === NaN)  // false;

console.log(/a/ === /a/);  // false;

console.log(1 === new String('1'))  // false;

var arr = [NaN];
console.log(arr.indexOf(NaN)); // -1

So when judging, if there are NaN and objects in the data, avoid using indexOf and ===;

in front of Set there As mentioned before, the Set method can remove duplicate NaNs.

Summary

I am tired of reading data deduplication on the Internet, but I still want to write an article to practice and summarize it. It would be great to have more ideas in my work. Thanks to those who love sharing and enjoying output.

The above is the detailed content of Summary of JavaScript array deduplication methods (with code). For more information, please follow other related articles on the PHP Chinese website!