Shell script as a daemon process example to ensure that PHP scripts do not hang up_PHP tutorial

I started running a data list a few days ago. The list requires user names, mobile phone numbers, and email addresses. I easily obtained the user list, but there were as many as 2,000,000 users. To detect whether a user has a mobile phone number and email address, one must request each user through a secure interface that is open to the outside world, and then analyze the return value to know.

Here’s what I did:

1. Save the 2000w list to the temporary data table

2. Use PHP program to obtain 500 users from the table each time, and generate SQL update original record after detection

3. In order to prevent the PHP program from suddenly disconnecting, use a shell script to detect it every 1 minute. If PHP hangs, restart it

The reason why I use a shell script as a daemon is that the detection interface between mobile phones and mailboxes is slow, and it is impossible to detect 20 million users in 1 to 2 days.

Plan details:

1. Temporarily save the user list table users. The table structure is as follows:

CREATE TABLE `users` (
  `account` varchar(50) COMMENT '用户名',
  `has_phone` tinyint(3) unsigned NOT NULL default '0' COMMENT '是否有手机号',
  `has_email` tinyint(3) unsigned NOT NULL default '0' COMMENT '是否有邮箱',
  `flag` tinyint(3) unsigned  NOT NULL default '0' COMMENT '标志位',
  PRIMARY KEY  (`account`),
  KEY `flag` (`flag`)
 ) ENGINE=InnoDB  DEFAULT CHARSET=utf8 COMMENT='名单表';

I first imported more than 2,000 usernames into this temporary table. The two fields of has_phone and has_email are both 0 (none) by default. The flag indicates whether the user has been detected.

The following is part of the table data:

9873aaa,0,0,0

adddwwwd876222,0,0,0

testalexlee,0,0,0

codejia.net,0,0,0

haohdouywaa21,0,0,0

2. PHP script check_users.php

After importing the user list into the table, write a simple PHP script. The idea is as follows: each loop retrieves 500 users with flag=0 from the table, and then requests the interface to determine whether the user has a mobile phone number and email address. Generate a SQL and save it to a SQLS array. After all 500 users have been detected, cycle the SQLS array, update the 500 lists in the table, and set the flag flag to 1, indicating that the detection has been completed and will not be performed next time. Got it.

Since the PHP script code is long, here is a simple code description:

<?php
class Users{
    private $data;
    private $sqls;
    private $nums;         //判断是否有500用户
    private $total_nums;   //当前已经检测完的用户数量
    
    //每次取500个用户
    private function getUsers(){...}

    //检测这500个用户并生成SQL
    private function checkUserInfo(){...}

    //更新这500个用户
    private function updateUserInfo(){...}

    //运行
    public function run(){
        $flag = true;
        while($flag){
             if($this->nums != 500){ $flag = false; }
             if($this->total_nums == 10000){ 
                exit(0); //跑完1w个用户就退出，由守护进程启动
             }
             $this->getUsers();
             $this->checkUserInfo();
             $this->updateUserInfo();
             sleep(1); //跑完500用户休息1秒，保护用户检测接口     
         }
    }
}

$user = new Users();
$user->run();
?>

The above is a concise version of the PHP script. You probably get the idea. The initial version did not have the $total_nums variable. This is because when I first started running this script, I found that it only ran out of more than 40,000 scripts before it failed. Later, At first glance, it turned out that the script was hanging there because the connection to the database failed. Adding this variable cannot solve this problem, but after running 10,000 users each time, the PHP script exits and is restarted by the following shell script.

3. Shell script as daemon process

I added this shell script to crontab and executed it every 1 minute. This shell script is very simple. It detects whether the process id exists in check_users.php. If it exists, it means that the PHP script is still running, and the shell script does not perform any operation; If it does not exist, it means that the PHP script has finished running exit(0) and 1w users have exited. Then the shell script starts the script and enters the detection of the next 1w user list.

I mentioned above that if the PHP script cannot connect to the database when it is connected, PHP will always hang the ball there and cannot exit. I added a time detection in the shell script. When the PHP script process exists, calculate how long it has existed. If it exceeds the time I expected, kill the PHP script and restart it.

The example data at the beginning, the results are similar to the following:

testalexlee,1,0,1

codejia.net,0,0,1

haohdouywaa21,1,1,1

9873aaa,0,1,1

adddwwwd876222,1,0,1

At the end: The above user list data is just an example. Don’t take it too seriously. With 20 million data, I estimate that it will take a while because the detection interface is relatively slow. After receiving the request, the interface has to connect to the table, look up the table, and then return. . In fact, the best way is to pull a list directly from the table requested by the interface, and then use shell commands to process it, and you will get the result quickly. But this is the case in the company, some things are not open, you know~~~