Home > Common Problem > How to remove duplicates from a Python list

How to remove duplicates from a Python list

百草
Release: 2023-06-14 13:25:07
Original
13190 people have browsed it

Python list deduplication method: 1. For loop to implement list deduplication, the original order remains unchanged after deduplication; 2. List derivation deduplication, the original order remains unchanged after deduplication; 3. The set conversion function "set()" implements list deduplication. The principle is that duplication of elements in the same set is not allowed. 4. The new dictionary method implements list deduplication. The principle is that the "key" of the dictionary is Duplication is not allowed; 5. There are duplicate data in the deletion list. As long as there are duplications, delete them all.

How to remove duplicates from a Python list

The operating system for this tutorial: Windows 10 system, Python version 3.11.2, DELL G3 computer.

How to remove duplicates from a Python list:
1. Use a for loop to remove duplicates from a list

After this method removes duplicates, the original order remains unchanged.

# for循环实现列表去重
list1 = ['a', 'b', 1, 3, 9, 9, 'a']
list2 = []
for l1 in list1:
    if l1 not in list2:
        list2.append(l1)
print(list2)
Copy after login

Result: ['a', 'b', 1, 3, 9]

2. Use list comprehension to remove duplicates
After deduplication using this method, the original order remains unchanged.

# 使用列表推导式去重
list1 = ['a', 'b', 1, 3, 9, 9, 'a']
res = []
[res.append(i) for i in list1 if i not in res]
print(res)
Copy after login

Result: ['a', 'b', 1, 3, 9]

3. Use the set conversion function set() to implement list deduplication
Principle: Duplication is not allowed between elements of the same set

# set()列表去重
list1 = ['a', 'b', 1, 3, 9, 9, 'a']
list2 = list(set(list1))
print(list2)
Copy after login

Result: [1, 3, 9, 'b', 'a']
Problem: Use the set() function After deduplication, it will be sorted automatically, and the order of the original list will change.
There are two solutions:
The first method is to use the sort() method

# # 第一种方法,sort()
list1 = ['a', 'b', 1, 3, 9, 9, 'a']
list2 = list(set(list1))
list2.sort(key=list1.index)
print(list2)
Copy after login

Result: ['a ', 'b', 1, 3, 9]
Note: The sort() method has no return value, and the list elements are sorted in place
The second method is to use the sorted() function

# 第二种方法,sored()
list1 = ['a', 'b', 1, 3, 9, 9, 'a']
list2 = sorted(list(set(list1)), key=list1.index)
print(list2)
Copy after login

Result: ['a', 'b', 1, 3, 9]
Note: The python built-in function sorted() function returns a new list and does not make any modifications to the original list

4. Use the new dictionary method to achieve list deduplication
Principle: The "keys" of the dictionary are not allowed to be repeated
After deduplication with this method, the original order remains unchanged.

# 使用新建字典实现列表去重
list1 = ['a', 'b', 1, 3, 9, 9, 'a']
dic = {}
dic = dic.fromkeys(list1).keys()
print(list(dic))
Copy after login

Result: ['a', 'b', 1, 3, 9]

5. Delete duplicate data in the list
The above There are 4 deduplication methods, all of which keep one and delete the others
The following method is to keep none as long as there are duplicates

# 删除存在重复的值,不保留
list1 = ['a', 'b', 1, 3, 9, 9, 'a']
list2 = [i for i in list1 if list1.count(i) == 1]
print(list2)
Copy after login

Result: ['b', 1, 3]

The above are 5 methods for removing duplicates from lists. You can choose the corresponding method according to your needs.

The above is the detailed content of How to remove duplicates from a Python list. For more information, please follow other related articles on the PHP Chinese website!

Related labels:
source:php.cn
Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Popular Tutorials
More>
Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template