How to quickly remove duplicate data in php

php method to remove duplicate data: 1. Use the "array_unique" method to deduplicate array elements, and use the "array_values" function to reorder the key values; 2. Use the "array_flip" method to deduplicate.

Recommended: "PHP Video Tutorial"

php Quickly remove duplicates from array elements

1. Use the array_unique method to deduplicate

To deduplicate array elements, we generally use the array_unique method. This method can deduplicate the elements in the array.

<?php
$arr = array(1,1,2,3,3,3,4,4,5,6,6,7,8,8,9,9,9);
$arr = array_unique($arr);
$arr = array_values($arr);
print_r($arr);
?>

Output:

Array
(
    [0] => 1
    [1] => 2
    [2] => 3
    [3] => 4
    [4] => 5
    [5] => 6
    [6] => 7
    [7] => 8
    [8] => 9
)

After deduplication, the key values ​​will be out of order. You can use array_values ​​to reorder the key values.

2. Efficiency of using array_unique method to remove duplicates

<?php
$arr = array();
// 创建100000个随机元素的数组
for($i=0; $i<100000; $i++){
    $arr[] = mt_rand(1,99);
}
// 记录开始时间
$starttime = getMicrotime();
// 去重
$arr = array_unique($arr);
// 记录结束时间
$endtime = getMicrotime();
$arr = array_values($arr);
echo &#39;unique count:&#39;.count($arr).&#39;<br>&#39;;
echo &#39;run time:&#39;.(float)(($endtime-$starttime)*1000).&#39;ms<br>&#39;;
echo &#39;use memory:&#39;.getUseMemory();
/**
 * 获取使用内存
 * @return float
 */
function getUseMemory(){
    $use_memory = round(memory_get_usage(true)/1024,2).&#39;kb&#39;;
    return $use_memory;
}
/**
 * 获取microtime
 * @return float
 */
function getMicrotime(){
    list($usec, $sec) = explode(&#39; &#39;, microtime());
    return (float)$usec + (float)$sec;
}
?>

unique count:99
run time:653.39303016663ms
use memory:5120kb

Using array_unique method to remove duplicates, the running time takes about 650ms and the memory usage is about 5m

3. Faster array removal Duplication method

php has a key-value exchange method array_flip. We can use this method to remove duplicates. Because of key-value exchange, the original duplicate values ​​will become the same key.

Then perform a key-value exchange again, and exchange the keys and values ​​back to complete deduplication.

<?php
$arr = array();
// 创建100000个随机元素的数组
for($i=0; $i<100000; $i++){
    $arr[] = mt_rand(1,99);
}
// 记录开始时间
$starttime = getMicrotime();
// 使用键值互换去重
$arr = array_flip($arr);
$arr = array_flip($arr);
// 记录结束时间
$endtime = getMicrotime();
$arr = array_values($arr);
echo &#39;unique count:&#39;.count($arr).&#39;<br>&#39;;
echo &#39;run time:&#39;.(float)(($endtime-$starttime)*1000).&#39;ms<br>&#39;;
echo &#39;use memory:&#39;.getUseMemory();
/**
 * 获取使用内存
 * @return float
 */
function getUseMemory(){
    $use_memory = round(memory_get_usage(true)/1024,2).&#39;kb&#39;;
    return $use_memory;
}
/**
 * 获取microtime
 * @return float
 */
function getMicrotime(){
    list($usec, $sec) = explode(&#39; &#39;, microtime());
    return (float)$usec + (float)$sec;
}
?>

unique count:99
run time:12.840032577515ms
use memory:768kb

Using the array_flip method to remove duplicates takes about 18ms to run and takes up about 2m of memory.

Therefore, using the array_flip method to delete duplicates takes 98% less running time than using the array_unique method, and the memory usage is reduced by 4/ 5;

The above is the detailed content of How to quickly remove duplicate data in php. For more information, please follow other related articles on the PHP Chinese website!