Home > Backend Development > Python Tutorial > Installation and usage tips of Python's network programming library Gevent

Installation and usage tips of Python's network programming library Gevent

WBOY
Release: 2016-07-06 13:29:44
Original
2205 people have browsed it

Installation (taking CentOS as an example)
gevent depends on libevent and greenlet:
1. Install libevent
Direct yum install libevent
Then configure the python installation
2. Install easy_install
(1)

wget -q http://peak.telecommunity.com/dist/ez_setup.py
Copy after login

(2) Use

python ez_setup.py
Copy after login

(3) Use easy_install to check whether the command is available. If not, you can add the path to PATH
3. Install greenlet
(1)

yum install python-devel
Copy after login

(2)

easy_install greenlet
Copy after login

4. Install gevent

pip install cython -e git://github.com/surfly/gevent.git@1.0rc2#egg=gevent
Copy after login

Tips
The Gevent library has high performance, but I have always been troubled by the fact that threads cannot seize multi-core resources due to Python's GIL model.
The multi-core mode of starting multiple python processes requires adding front-end load balancing, such as lvs, which is a bit troublesome.
The multiprocessing module and os.fork will cause the two processes to repeatedly register accept events in the event core, resulting in duplicate file handle exceptions.
As for the mode of one process monitoring and multiple process processing, the resources of the monitoring process are not easy to allocate - should one core be allocated independently or not? If allocated separately, a core will be wasted when the number of connections is small. If not allocated, the CPU will frequently switch processes when the number of connections is large.
I just discovered yesterday that gevent can easily distribute its network model to multiple processes for parallel processing.
The secret is gevent.fork().
I used to take it for granted that gevent.fork is just a wrapper for greenlet.spawn, but it turns out that's not the case. gevent.fork can replace os.fork, which not only starts a new process, but also communicates their underlying event processing for parallel processing.

import gevent
from gevent.server import StreamServer

def eat_cpu():
  for i in xrange(10000): pass

def cb(socket, address):
  eat_cpu()
  socket.recv(1024)
  socket.sendall('HTTP/1.1 200 OK\n\nHello World!!')
  socket.close()

server = StreamServer(('',80), cb, backlog=100000)
server.pre_start()

gevent.fork()

server.start_accepting()
server._stopped_event.wait()

Copy after login

After adding monkey.patch_os, os.fork can be replaced by gevent.fork, so that the multiprocessing module can be used as usual and achieve the effect of parallel processing.

from gevent import monkey; monkey.patch_os()
from gevent.server import StreamServer
from multiprocessing import Process

def eat_cpu(): 
  for i in xrange(10000): pass

def cb(socket, address):
  eat_cpu()
  socket.recv(1024)
  socket.sendall('HTTP/1.1 200 OK\n\nHello World!!')
  socket.close()

server = StreamServer(('',80), cb, backlog=100000)
server.pre_start()

def serve_forever():
  server.start_accepting()
  server._stopped_event.wait()

process_count = 4

for i in range(process_count - 1):
  Process(target=serve_forever, args=tuple()).start()

serve_forever()
Copy after login

Related labels:
source:php.cn
Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Popular Tutorials
More>
Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template