Home > Article > Backend Development > Detailed introduction to positioning and destruction in Python threads (with examples)
This article brings you a detailed introduction to positioning and destruction in Python threads (with examples). It has certain reference value. Friends in need can refer to it. I hope it will be helpful to you.
I felt something was not right before work started, and I felt like I was going to take the blame. No, it’s the third day of work that makes me feel guilty.
We have an amazing background program that can dynamically load modules and run them in threads. In this way, the functions of the plug-in can be realized. When the module is updated, the background program itself will not exit. It will only close the thread corresponding to the module, update the code and then start it. 6 is not possible.
So I wrote a module to show off my skills, but I forgot to write the exit function, which resulted in a new thread being created every time the module was updated. Unless the program was restarted, those threads would remain alive.
This is not possible. We have to find a way to clean it up, otherwise I am afraid it will explode.
So how to clean it? All I can think of is two steps:
Find out the thread numbers tid that need to be cleaned up;
Destroy them;
Similar to the usual troubleshooting, first check the thread status of the target process through the ps command. Because the thread name has been set by setName, normally it should I saw the corresponding thread. Directly use the following code to simulate this thread:
Python version of multi-threading
#coding: utf8
import threading
import os
import time
def tt():
info = threading.currentThread()
while True:
print 'pid: ', os.getpid()
print info.name, info.ident
time.sleep(3)
t1 = threading.Thread(target=tt)
t1.setName('OOOOOPPPPP')
t1.setDaemon(True)
t1.start()
t2 = threading.Thread(target=tt)
t2.setName('EEEEEEEEE')
t2.setDaemon(True)
t2.start()
t1.join()
t2.join()
Output:
root@10-46-33-56:~# python t.py pid: 5613 OOOOOPPPPP 139693508122368 pid: 5613 EEEEEEEEE 139693497632512 ...
You can see that the thread name output in Python is what we set That, but the result of Ps made me doubt life:
root@10-46-33-56:~# ps -Tp 5613 PID SPID TTY TIME CMD 5613 5613 pts/2 00:00:00 python 5613 5614 pts/2 00:00:00 python 5613 5615 pts/2 00:00:00 python
Normally it shouldn’t be like this. I’m a little confused. Did I remember it wrong all along? Use another language version of multi-threading to test:
C version of multi-threading
#include<stdio.h>
#include<sys>
#include<sys>
#include<pthread.h>
void *test(void *name)
{
pid_t pid, tid;
pid = getpid();
tid = syscall(__NR_gettid);
char *tname = (char *)name;
// 设置线程名字
prctl(PR_SET_NAME, tname);
while(1)
{
printf("pid: %d, thread_id: %u, t_name: %s\n", pid, tid, tname);
sleep(3);
}
}
int main()
{
pthread_t t1, t2;
void *ret;
pthread_create(&t1, NULL, test, (void *)"Love_test_1");
pthread_create(&t2, NULL, test, (void *)"Love_test_2");
pthread_join(t1, &ret);
pthread_join(t2, &ret);
}</pthread.h></sys></sys></stdio.h>
Output:
root@10-46-33-56:~# gcc t.c -lpthread && ./a.out pid: 5575, thread_id: 5577, t_name: Love_test_2 pid: 5575, thread_id: 5576, t_name: Love_test_1 pid: 5575, thread_id: 5577, t_name: Love_test_2 pid: 5575, thread_id: 5576, t_name: Love_test_1 ...
Use the PS command to verify again:
root@10-46-33-56:~# ps -Tp 5575 PID SPID TTY TIME CMD 5575 5575 pts/2 00:00:00 a.out 5575 5576 pts/2 00:00:00 Love_test_1 5575 5577 pts/2 00:00:00 Love_test_2
This is correct. The thread name can indeed be seen through Ps!
But why can’t I see the Python one? Since the thread name is set through setName, let’s take a look at the definition:
[threading.py] class Thread(_Verbose): ... @property def name(self): """A string used for identification purposes only. It has no semantics. Multiple threads may be given the same name. The initial name is set by the constructor. """ assert self.__initialized, "Thread.__init__() not called" return self.__name @name.setter def name(self, name): assert self.__initialized, "Thread.__init__() not called" self.__name = str(name) def setName(self, name): self.name = name ...
Seeing here is actually just setting the properties of the Thread object, and If it has not been touched to the root, it must be invisible~
It seems that we can no longer use ps or /proc/ to externally The python thread name is searched, so we can only solve it within Python.
So the question becomes, how to get all running threads inside Python?
threading.enumerate can perfectly solve this problem! Why?
Because it is clearly stated in the doc of the following function, all active thread objects are returned, excluding terminated and unstarted ones.
[threading.py] def enumerate(): """Return a list of all Thread objects currently alive. The list includes daemonic threads, dummy thread objects created by current_thread(), and the main thread. It excludes terminated threads and threads that have not yet been started. """ with _active_limbo_lock: return _active.values() + _limbo.values()
Because we get the Thread object, we can get the information related to the thread through this!
Please see the complete code example:
<pre class="brush:php;toolbar:false">#coding: utf8 import threading import os import time def get_thread(): pid = os.getpid() while True: ts = threading.enumerate() print '------- Running threads On Pid: %d -------' % pid for t in ts: print t.name, t.ident print time.sleep(1) def tt(): info = threading.currentThread() pid = os.getpid() while True: print 'pid: {}, tid: {}, tname: {}'.format(pid, info.name, info.ident) time.sleep(3) return t1 = threading.Thread(target=tt) t1.setName('Thread-test1') t1.setDaemon(True) t1.start() t2 = threading.Thread(target=tt) t2.setName('Thread-test2') t2.setDaemon(True) t2.start() t3 = threading.Thread(target=get_thread) t3.setName('Checker') t3.setDaemon(True) t3.start() t1.join() t2.join() t3.join()</pre>Output:
root@10-46-33-56:~# python t_show.py pid: 6258, tid: Thread-test1, tname: 139907597162240 pid: 6258, tid: Thread-test2, tname: 139907586672384 ------- Running threads On Pid: 6258 ------- MainThread 139907616806656 Thread-test1 139907597162240 Checker 139907576182528 Thread-test2 139907586672384 ------- Running threads On Pid: 6258 ------- MainThread 139907616806656 Thread-test1 139907597162240 Checker 139907576182528 Thread-test2 139907586672384 ------- Running threads On Pid: 6258 ------- MainThread 139907616806656 Thread-test1 139907597162240 Checker 139907576182528 Thread-test2 139907586672384 ------- Running threads On Pid: 6258 ------- MainThread 139907616806656 Checker 139907576182528 ...
The code looks a bit long, but the logic is quite simple, Thread-test1 and Thread-test2 prints out the current pid, thread id and thread name, and then exits after 3 seconds. This is to simulate the normal exit of the thread.
The Checker thread outputs all active threads in the current process through threading.enumerate every second.
It can be clearly seen that the information of Thread-test1 and Thread-test2 can be seen at the beginning. After they exit, only is left. MainThread and Checker are just themselves.
Since we can get the name and thread id, we can also kill the specified thread!
Suppose now Thread-test2 has gone black and gone crazy, and we need to stop it, then we can solve it in this way:
In the above code Basically, add and add the following code:
def _async_raise(tid, exctype):
"""raises the exception, performs cleanup if needed"""
tid = ctypes.c_long(tid)
if not inspect.isclass(exctype):
exctype = type(exctype)
res = ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, ctypes.py_object(exctype))
if res == 0:
raise ValueError("invalid thread id")
elif res != 1:
ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, None)
raise SystemError("PyThreadState_SetAsyncExc failed")
def stop_thread(thread):
_async_raise(thread.ident, SystemExit)
def get_thread():
pid = os.getpid()
while True:
ts = threading.enumerate()
print '------- Running threads On Pid: %d -------' % pid
for t in ts:
print t.name, t.ident, t.is_alive()
if t.name == 'Thread-test2':
print 'I am go dying! Please take care of yourself and drink more hot water!'
stop_thread(t)
print
time.sleep(1)
Output
root@10-46-33-56:~# python t_show.py pid: 6362, tid: 139901682108160, tname: Thread-test1 pid: 6362, tid: 139901671618304, tname: Thread-test2 ------- Running threads On Pid: 6362 ------- MainThread 139901706389248 True Thread-test1 139901682108160 True Checker 139901661128448 True Thread-test2 139901671618304 True Thread-test2: I am go dying. Please take care of yourself and drink more hot water! ------- Running threads On Pid: 6362 ------- MainThread 139901706389248 True Thread-test1 139901682108160 True Checker 139901661128448 True Thread-test2 139901671618304 True Thread-test2: I am go dying. Please take care of yourself and drink more hot water! pid: 6362, tid: 139901682108160, tname: Thread-test1 ------- Running threads On Pid: 6362 ------- MainThread 139901706389248 True Thread-test1 139901682108160 True Checker 139901661128448 True // Thread-test2 已经不在了
After a while, although we treat Thread-test2 like this, it still cares about us :Drink more hot water,
PS: Although hot water is good, eight cups is enough, so don’t be greedy.
Back to the main story, the above method is extremely crude. Why do you say that?
Because its principle is: use Python's built-in API to trigger an exception on the specified thread so that it can automatically exit;
We can't even kill threads directly like killing processes through signals, because kill can only achieve our expectations by dealing with processes, but it is obviously not possible to deal with threads. No matter which thread is killed, the entire process will exit!
And because of the GIL, many children think that Python's threads are implemented by Python itself and do not actually exist. Python should be able to destroy them directly, right?
However, in fact, Python’s threads are genuine threads!
What does that mean? Python threads are native threads created by the operating system through pthread. Python only uses GIL to constrain these threads to decide when to start scheduling. For example, after running a number of instructions, the GIL will be handed over. As for who wins the oiran, it depends on the operating system.
If it is a simple thread, the system actually has ways to terminate it, such as: pthread_exit,pthread_kill or pthread_cancel, for details, see: https://www.cnblogs.com/Creat...
What a pity: There is no encapsulation of these methods at the Python level! Oh my god, so angry! Maybe people think that threads should be treated gently.
If you want to exit a thread gently, it is almost nonsense~
Either exit after running, or set the flag bit, and check the flag bit frequently. If you need to quit, just quit.
"How to correctly terminate a running child thread": https://www.cnblogs.com/Creat...
"Don't destroy python threads roughly" ：http://xiaorui.cc/2017/02/22/...
Welcome all experts to give advice and exchange, QQ discussion group: 258498217
Please note for reprinting Source: https://segmentfault.com/a/11...
Lin_R
posted on
Posted 3 days ago
Python: Thread positioning and destruction
c
linuxPython
# 266 Reading It takes 30 minutes to read
8
BackgroundI felt something was not right before the start of work, and I felt like I was going to take the blame. No, it’s the third day of work that makes me feel guilty. We have an amazing background program that can dynamically load modules and run them in threads. In this way, the functions of the plug-in can be realized. When the module is updated, the background program itself will not exit. It will only close the thread corresponding to the module, update the code and then start it. 6 is not possible.
So I wrote a module to show off my skills, but I forgot to write the exit function, which resulted in a new thread being created every time the module was updated. Unless the program was restarted, those threads would remain alive.
So how to clean it? All I can think of is two steps:
Find out the thread numbers tid that need to be cleaned up;
Destroy them;
Similar to the usual troubleshooting, first check the thread status of the target process through the ps command. Because the thread name has been set by setName, normally it should I saw the corresponding thread. Directly use the following code to simulate this thread:
#coding: utf8
import threading
import os
import time
def tt():
info = threading.currentThread()
while True:
print 'pid: ', os.getpid()
print info.name, info.ident
time.sleep(3)
t1 = threading.Thread(target=tt)
t1.setName('OOOOOPPPPP')
t1.setDaemon(True)
t1.start()
t2 = threading.Thread(target=tt)
t2.setName('EEEEEEEEE')
t2.setDaemon(True)
t2.start()
t1.join()
t2.join()
Output:
root@10-46-33-56:~# python t.py pid: 5613 OOOOOPPPPP 139693508122368 pid: 5613 EEEEEEEEE 139693497632512 ...
root@10-46-33-56:~# ps -Tp 5613 PID SPID TTY TIME CMD 5613 5613 pts/2 00:00:00 python 5613 5614 pts/2 00:00:00 python 5613 5615 pts/2 00:00:00 python
C version of multi-threading
#include<stdio.h>
#include<sys>
#include<sys>
#include<pthread.h>
void *test(void *name)
{
pid_t pid, tid;
pid = getpid();
tid = syscall(__NR_gettid);
char *tname = (char *)name;
// 设置线程名字
prctl(PR_SET_NAME, tname);
while(1)
{
printf("pid: %d, thread_id: %u, t_name: %s\n", pid, tid, tname);
sleep(3);
}
}
int main()
{
pthread_t t1, t2;
void *ret;
pthread_create(&t1, NULL, test, (void *)"Love_test_1");
pthread_create(&t2, NULL, test, (void *)"Love_test_2");
pthread_join(t1, &ret);
pthread_join(t2, &ret);
}</pthread.h></sys></sys></stdio.h>
Output:
root@10-46-33-56:~# gcc t.c -lpthread && ./a.out pid: 5575, thread_id: 5577, t_name: Love_test_2 pid: 5575, thread_id: 5576, t_name: Love_test_1 pid: 5575, thread_id: 5577, t_name: Love_test_2 pid: 5575, thread_id: 5576, t_name: Love_test_1 ...
Use the PS command to verify again:
root@10-46-33-56:~# ps -Tp 5575 PID SPID TTY TIME CMD 5575 5575 pts/2 00:00:00 a.out 5575 5576 pts/2 00:00:00 Love_test_1 5575 5577 pts/2 00:00:00 Love_test_2
This is correct. The thread name can indeed be seen through Ps!
But why can’t I see the Python one? Since the thread name is set through
setName, let’s take a look at the definition:
[threading.py] class Thread(_Verbose): ... @property def name(self): """A string used for identification purposes only. It has no semantics. Multiple threads may be given the same name. The initial name is set by the constructor. """ assert self.__initialized, "Thread.__init__() not called" return self.__name @name.setter def name(self, name): assert self.__initialized, "Thread.__init__() not called" self.__name = str(name) def setName(self, name): self.name = name ...
Seeing here is actually just setting the properties of the
Threadobject, and If it has not been touched to the root, it must be invisible~
It seems that we can no longer use
ps or /proc/ to externally The python thread name is searched, so we can only solve it within Python.
So the question becomes, how to get all running threads inside Python?
can perfectly solve this problem! Why?Because it is clearly stated in the doc of the following function, all active thread objects are returned, excluding terminated and unstarted ones.
[threading.py] def enumerate(): """Return a list of all Thread objects currently alive. The list includes daemonic threads, dummy thread objects created by current_thread(), and the main thread. It excludes terminated threads and threads that have not yet been started. """ with _active_limbo_lock: return _active.values() + _limbo.values()
Because we get the Thread object, we can get the information related to the thread through this!
Please see the complete code example: <pre class="brush:php;toolbar:false">#coding: utf8
import threading
import os
import time
def get_thread():
pid = os.getpid()
while True:
ts = threading.enumerate()
print '------- Running threads On Pid: %d -------' % pid
for t in ts:
print t.name, t.ident
print
time.sleep(1)
def tt():
info = threading.currentThread()
pid = os.getpid()
while True:
print 'pid: {}, tid: {}, tname: {}'.format(pid, info.name, info.ident)
time.sleep(3)
return
t1 = threading.Thread(target=tt)
t1.setName('Thread-test1')
t1.setDaemon(True)
t1.start()
t2 = threading.Thread(target=tt)
t2.setName('Thread-test2')
t2.setDaemon(True)
t2.start()
t3 = threading.Thread(target=get_thread)
t3.setName('Checker')
t3.setDaemon(True)
t3.start()
t1.join()
t2.join()
t3.join()</pre>Output:
root@10-46-33-56:~# python t_show.py pid: 6258, tid: Thread-test1, tname: 139907597162240 pid: 6258, tid: Thread-test2, tname: 139907586672384 ------- Running threads On Pid: 6258 ------- MainThread 139907616806656 Thread-test1 139907597162240 Checker 139907576182528 Thread-test2 139907586672384 ------- Running threads On Pid: 6258 ------- MainThread 139907616806656 Thread-test1 139907597162240 Checker 139907576182528 Thread-test2 139907586672384 ------- Running threads On Pid: 6258 ------- MainThread 139907616806656 Thread-test1 139907597162240 Checker 139907576182528 Thread-test2 139907586672384 ------- Running threads On Pid: 6258 ------- MainThread 139907616806656 Checker 139907576182528 ...
The code looks a bit long, but the logic is quite simple, Thread-test1 and
Thread-test2prints out the current pid, thread id and thread name, and then exits after 3 seconds. This is to simulate the normal exit of the thread.
The
Checkerthread outputs all active threads in the current process through
threading.enumerate every second. It can be clearly seen that the information of Thread-test1 and
can be seen at the beginning. After they exit, only is left. MainThread and Checker are just themselves.
Destroy the specified threadSince we can get the name and thread id, we can also kill the specified thread! Suppose now Thread-test2 has gone black and gone crazy, and we need to stop it, then we can solve it in this way: In the above code Basically, add and add the following code:
def _async_raise(tid, exctype):
"""raises the exception, performs cleanup if needed"""
tid = ctypes.c_long(tid)
if not inspect.isclass(exctype):
exctype = type(exctype)
res = ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, ctypes.py_object(exctype))
if res == 0:
raise ValueError("invalid thread id")
elif res != 1:
ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, None)
raise SystemError("PyThreadState_SetAsyncExc failed")
def stop_thread(thread):
_async_raise(thread.ident, SystemExit)
def get_thread():
pid = os.getpid()
while True:
ts = threading.enumerate()
print '------- Running threads On Pid: %d -------' % pid
for t in ts:
print t.name, t.ident, t.is_alive()
if t.name == 'Thread-test2':
print 'I am go dying! Please take care of yourself and drink more hot water!'
stop_thread(t)
print
time.sleep(1)
root@10-46-33-56:~# python t_show.py pid: 6362, tid: 139901682108160, tname: Thread-test1 pid: 6362, tid: 139901671618304, tname: Thread-test2 ------- Running threads On Pid: 6362 ------- MainThread 139901706389248 True Thread-test1 139901682108160 True Checker 139901661128448 True Thread-test2 139901671618304 True Thread-test2: I am go dying. Please take care of yourself and drink more hot water! ------- Running threads On Pid: 6362 ------- MainThread 139901706389248 True Thread-test1 139901682108160 True Checker 139901661128448 True Thread-test2 139901671618304 True Thread-test2: I am go dying. Please take care of yourself and drink more hot water! pid: 6362, tid: 139901682108160, tname: Thread-test1 ------- Running threads On Pid: 6362 ------- MainThread 139901706389248 True Thread-test1 139901682108160 True Checker 139901661128448 True // Thread-test2 已经不在了
After a while, although we treat
Thread-test2 like this, it still cares about us :Drink more hot water,
PS: Although hot water is good, eight cups is enough, so don’t be greedy.
Back to the main story, the above method is extremely crude. Why do you say that?
Because its principle is: use Python's built-in API to trigger an exception on the specified thread so that it can automatically exit;
pthread_exit,pthread_kill or pthread_cancel, for details, see: https://www.cnblogs.com/Creat...
"Don't destroy python threads roughly" ：http://xiaorui.cc/2017/02/22/...
Welcome all experts to give advice and exchange, QQ discussion group: 258498217
Please note for reprinting Ming source: https://segmentfault.com/a/11...
If you find my article useful to you, please feel free to appreciate it
You may feel InterestedTime sorting
· 1 day ago
If it were me, I might kill -9, I would rather kill a thousand by mistake than let one go, hehe
The above is the detailed content of Detailed introduction to positioning and destruction in Python threads (with examples). For more information, please follow other related articles on the PHP Chinese website!