My brother is currently working on a multi-process crawling service. It is now a main process and 10 child processes. If the child process exits unexpectedly, create another child process.
But this way, if the main process accidentally exits the child process, it will not exit with it. Next time I open this service, I will create 10 child processes, which adds up to 20 child processes. Not only will the data in redis be messed up, but it will also take up a lot of memory. Is there anything wrong? What is a good way to ensure that if the main process ends unexpectedly, the following child processes will be shut down together? ?
My idea is that every time the main process is started, exec executes the next shell script. The shell script closes all related processes except this main process. I wonder if you guys have any advice?
My brother is currently working on a multi-process crawling service. It is now a main process and 10 child processes. If the child process exits unexpectedly, create another child process.
But this way, if the main process accidentally exits the child process, it will not exit with it. Next time I open this service, I will create 10 child processes, which adds up to 20 child processes. Not only will the data in redis be messed up, but it will also take up a lot of memory. Is there anything wrong? What is a good way to ensure that if the main process ends unexpectedly, the following child processes will be shut down together? ?
My idea is that every time the main process is started, exec executes the next shell script. The shell script closes all related processes except this main process. I wonder if you guys have any advice?