Installation (taking CentOS as an example)
gevent depends on libevent and greenlet:
1. Install libevent
Direct yum install libevent
Then configure the python installation
2. Install easy_install
(1)
wget -q http://peak.telecommunity.com/dist/ez_setup.py
(2) Use
python ez_setup.py
(3) Use easy_install to check whether the command is available. If not, you can add the path to PATH
3. Install greenlet
(1)
yum install python-devel
(2)
easy_install greenlet
4. Install gevent
pip install cython -e git://github.com/surfly/gevent.git@1.0rc2#egg=gevent
Tips
The Gevent library has high performance, but I have always been troubled by the fact that threads cannot seize multi-core resources due to Python's GIL model.
The multi-core mode of starting multiple python processes requires adding front-end load balancing, such as lvs, which is a bit troublesome.
The multiprocessing module and os.fork will cause the two processes to repeatedly register accept events in the event core, resulting in duplicate file handle exceptions.
As for the mode of one process monitoring and multiple process processing, the resources of the monitoring process are not easy to allocate - should one core be allocated independently or not? If allocated separately, a core will be wasted when the number of connections is small. If not allocated, the CPU will frequently switch processes when the number of connections is large.
I just discovered yesterday that gevent can easily distribute its network model to multiple processes for parallel processing.
The secret is gevent.fork().
I used to take it for granted that gevent.fork is just a wrapper for greenlet.spawn, but it turns out that's not the case. gevent.fork can replace os.fork, which not only starts a new process, but also communicates their underlying event processing for parallel processing.
import gevent from gevent.server import StreamServer def eat_cpu(): for i in xrange(10000): pass def cb(socket, address): eat_cpu() socket.recv(1024) socket.sendall('HTTP/1.1 200 OK\n\nHello World!!') socket.close() server = StreamServer(('',80), cb, backlog=100000) server.pre_start() gevent.fork() server.start_accepting() server._stopped_event.wait()
After adding monkey.patch_os, os.fork can be replaced by gevent.fork, so that the multiprocessing module can be used as usual and achieve the effect of parallel processing.
from gevent import monkey; monkey.patch_os() from gevent.server import StreamServer from multiprocessing import Process def eat_cpu(): for i in xrange(10000): pass def cb(socket, address): eat_cpu() socket.recv(1024) socket.sendall('HTTP/1.1 200 OK\n\nHello World!!') socket.close() server = StreamServer(('',80), cb, backlog=100000) server.pre_start() def serve_forever(): server.start_accepting() server._stopped_event.wait() process_count = 4 for i in range(process_count - 1): Process(target=serve_forever, args=tuple()).start() serve_forever()