Examples of memory optimization when using PHP for large number of loops

Sometimes we need to process a large amount of data. Too many loops may cause memory exhaustion. How to solve this problem? The editor below will help you solve the problem of memory exhaustion when looping a large amount of data in PHP. Friends in need can refer to it. I hope to be helpful.

I recently encountered the following problem when developing a PHP program:

PHP Fatal error: Allowed memory size of 268 435 456 bytes exhausted

The error message shows that the maximum allowed memory has been exhausted. I was surprised to encounter such an error at first, but after thinking about it, it is not surprising, because the program I am developing is to use a foreach loop statement to search for specific characteristics in a table with 40,000 records. data, that is to say, 40,000 pieces of data must be taken out at a time, and then the daily data must be checked one by one. It is conceivable that all 40,000 pieces of data are loaded into the memory. It would be strange if the memory does not burst.

After all these years of programming, I vaguely remember that PHP provides an API for loading data all at once. It is like processing streaming media. You can take and lose data as you go. Query methods that do not accumulate in memory. After a simple search, I found the correct usage on the official website.

This problem is called Buffered and Unbuffered queries on PHP's official website. PHP's default query mode is buffered mode. In other words, the query data results will be extracted into memory all at once for processing by the PHP program. This gives the PHP program additional functions, such as counting the number of rows, pointing the pointer to a certain row, etc. What's more important is that the program can repeatedly perform secondary queries and filtering operations on the data set. However, the disadvantage of this buffered query mode is that it consumes memory, that is, it trades space for speed.

In contrast, another PHP query mode is a non-buffered query. The database server will return data one by one instead of all at once. The result is that the PHP program consumes less memory, but it increases The pressure on the database server, because the database will wait for PHP to fetch the data until all the data is fetched.

Obviously, the buffered query mode is suitable for small data volume queries, while non-buffered query is suitable for large data volume queries.

Everyone knows about PHP’s buffered mode query. The following example is how to execute the non-buffered query API.

Non-buffered query method one: mysqli

##

<?php 
$mysqli = new mysqli("localhost", "my_user", "my_password", "world"); 
$uresult = $mysqli->query("SELECT Name FROM City", MYSQLI_USE_RESULT); 
if ($uresult) { 
  while ($row = $uresult->fetch_assoc()) { 
    echo $row[&#39;Name&#39;] . PHP_EOL; 
  } 
} 
$uresult->close(); 
?>

Non-buffered query method two: pdo_mysql

##

<?php 
$pdo = new PDO("mysql:host=localhost;dbname=world", &#39;my_user&#39;, &#39;my_pass&#39;); 
$pdo->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, false); 

$uresult = $pdo->query("SELECT Name FROM City"); 
if ($uresult) { 
  while ($row = $uresult->fetch(PDO::FETCH_ASSOC)) { 
    echo $row[&#39;Name&#39;] . PHP_EOL; 
  } 
} 
?>

Non-buffered query method three: mysql

<?php 
$conn = mysql_connect("localhost", "my_user", "my_pass"); 
$db  = mysql_select_db("world"); 

$uresult = mysql_unbuffered_query("SELECT Name FROM City"); 
if ($uresult) { 
  while ($row = mysql_fetch_assoc($uresult)) { 
    echo $row[&#39;Name&#39;] . PHP_EOL; 
  } 
} 
?>

Related recommendations:

The above is the detailed content of Examples of memory optimization when using PHP for large number of loops. For more information, please follow other related articles on the PHP Chinese website!