Using python to implement high-performance testing tools (1)

People who have been developing or testing developers for several years often feel confused. The development of new functions or the maintenance of old functions are basically piles of code. This article mainly talks about performance optimization in system design and architecture for everyone to learn. Some of the content involves specific products, some changes have been made, or test code demonstrations have been written separately.

Project background:

Implement a high-performance diameter test tool, accept 1000 and send 1000, and support up to 2000 messages per second in both directions. The source code of the diameter protocol is downloaded from http://sourceforge.net/projects/pyprotosim/. This open source package also supports SMPP, RADIUS, DHCP, LDAP, and the newly added protocol fields can be configured in the dictionary. It is really convenient if you need to modify the code. In the initial stage, in order to implement functions, we did not consider performance issues. Single threads were used in many places, and the initial performance could only support 50 messages. Hardware environment: SunFire 4170, 16 cores, 2.4 G per core

Several directions for Python performance optimization:

1. Change the python parser: Common python parsers include pysco, pypy, cython, jython and pysco no longer support python 2.7, so there is no test. It is said that it runs as fast as C language. I did a simple test on pypy and jython. pypy can be improved to 5-10 times on different machines. Although Jython can avoid the problem of python GIL (because jython runs on a java virtual machine), it seems from the test that Efficiency gains are minimal.

2. Optimize the code

3. Change the system architecture, multi-threading, multi-process or coroutine

Solution 1 : Changing the Python parser

If changing the Python parser can meet the performance requirements, it is the cheapest solution and does not require any changes to the code. The following code is just to illustrate the effect of pypy. It is a test code written separately and the result of running under windows. The running effect will be better on a Linux machine.

#!/usr/bin/env python #coding=utf-8 import time def check(num): a = list(str(num)) b = a[::-1] if a == b: return True return False def test(): all = xrange(1,10**7) for i in all: if check(i): if check(i**2): i**2 if __name__ == '__main__': start=time.time() test() print time.time()-start

The results of using python and pypy respectively

C:\Python27\python.exeD:/RCC/mp/src/test.py
14.4940001965

C:\pypy-2.1\pypy.exeD:/RCC/mp/src/test.py
4.37800002098

You can see the running results of pypy The effect is still obvious, although it can be increased by 5 times (on a Linux machine), 50*5, which is still far from 2000. pypy has no obvious effect on python multi-threading support, which will be mentioned later.

Let’s end it first. It’s too long and everyone seems tired. The next article will introduce the code optimization part.

[Recommended course:Python video tutorial]

The above is the detailed content of Using python to implement high-performance testing tools (1). For more information, please follow other related articles on the PHP Chinese website!