How to remove duplicates from a Python list

Python list deduplication method: 1. For loop to implement list deduplication, the original order remains unchanged after deduplication; 2. List derivation deduplication, the original order remains unchanged after deduplication; 3. The set conversion function "set()" implements list deduplication. The principle is that duplication of elements in the same set is not allowed. 4. The new dictionary method implements list deduplication. The principle is that the "key" of the dictionary is Duplication is not allowed; 5. There are duplicate data in the deletion list. As long as there are duplications, delete them all.

The operating system for this tutorial: Windows 10 system, Python version 3.11.2, DELL G3 computer.

How to remove duplicates from a Python list:
1. Use a for loop to remove duplicates from a list
After this method removes duplicates, the original order remains unchanged.

# for循环实现列表去重
list1 = ['a', 'b', 1, 3, 9, 9, 'a']
list2 = []
for l1 in list1:
    if l1 not in list2:
        list2.append(l1)
print(list2)

Result: ['a', 'b', 1, 3, 9]

2. Use list comprehension to remove duplicates
After deduplication using this method, the original order remains unchanged.

# 使用列表推导式去重
list1 = ['a', 'b', 1, 3, 9, 9, 'a']
res = []
[res.append(i) for i in list1 if i not in res]
print(res)

Result: ['a', 'b', 1, 3, 9]

3. Use the set conversion function set() to implement list deduplication
Principle: Duplication is not allowed between elements of the same set

# set()列表去重
list1 = ['a', 'b', 1, 3, 9, 9, 'a']
list2 = list(set(list1))
print(list2)

Result: [1, 3, 9, 'b', 'a']
Problem: Use the set() function After deduplication, it will be sorted automatically, and the order of the original list will change.
There are two solutions:
The first method is to use the sort() method

# # 第一种方法，sort()
list1 = ['a', 'b', 1, 3, 9, 9, 'a']
list2 = list(set(list1))
list2.sort(key=list1.index)
print(list2)

Result: ['a ', 'b', 1, 3, 9]
Note: The sort() method has no return value, and the list elements are sorted in place
The second method is to use the sorted() function

# 第二种方法，sored()
list1 = ['a', 'b', 1, 3, 9, 9, 'a']
list2 = sorted(list(set(list1)), key=list1.index)
print(list2)

Result: ['a', 'b', 1, 3, 9]
Note: The python built-in function sorted() function returns a new list and does not make any modifications to the original list

4. Use the new dictionary method to achieve list deduplication
Principle: The "keys" of the dictionary are not allowed to be repeated
After deduplication with this method, the original order remains unchanged.

# 使用新建字典实现列表去重
list1 = ['a', 'b', 1, 3, 9, 9, 'a']
dic = {}
dic = dic.fromkeys(list1).keys()
print(list(dic))

Result: ['a', 'b', 1, 3, 9]

5. Delete duplicate data in the list
The above There are 4 deduplication methods, all of which keep one and delete the others
The following method is to keep none as long as there are duplicates

# 删除存在重复的值，不保留
list1 = ['a', 'b', 1, 3, 9, 9, 'a']
list2 = [i for i in list1 if list1.count(i) == 1]
print(list2)

Result: ['b', 1, 3]

The above are 5 methods for removing duplicates from lists. You can choose the corresponding method according to your needs.

The above is the detailed content of How to remove duplicates from a Python list. For more information, please follow other related articles on the PHP Chinese website!