python high performance programming method one

Read Zen of Python and type import this into the Python parser. A keen Python newbie might notice the word "parse" and think Python is just another scripting language. "It must be slow!"

Hint There is no doubt that Python programs are not as efficient and fast as compiled languages. Even Python advocates will tell you that Python is not suitable for these fields. However, YouTube has used Python to serve 40 million video requests per hour. All you have to do is write efficient code and use external implementation (C/C++) code when needed. Here are some tips to help you become a better Python developer:

1. Use built-in functions: You can write efficient code in Python, But it's hard to beat the built-in functions. Verified. They are very fast. 2. Use join() to concatenate strings. You can use "+" to concatenate strings. But since strings are immutable in Python, each " The +" operation will create a new string and copy the old content. A common usage is to use Python's array module to modify individual characters; when completed, use the join() function to create the final string.

>>> #This is good to glue a large number of strings

>>> for chunk in input():

>>> my_string.join(chunk)

3. Use Python multiple assignment to exchange variables

This is in Python Elegant and fast:   >>> x, y = y, x

  This is very slow:

  >>> temp = x

                                                              use   use   use use using use using using using using out out out out   out out out out out out ’       ’’’   out     ’   ’ through ‐   ‐‐‐ ‐ ​ ​ ​ ​ ​ ​ ​      . Try to use local variables

Python retrieves local variables faster than retrieving global variables. This means, avoid the "global" keyword.

5. Try to use "in"

Use the "in" keyword. Simple and fast.

>>> for key in sequence:

>>> print “found”

6. Use lazy loading to speed up

Move the "import" declaration into the function and only import it when needed. In other words, if a certain Some modules do not need to be used immediately, and you can import them later. For example, you do not have to import a large number of modules at the beginning to speed up the program startup. This technique cannot improve the overall performance. But it can help you distribute the loading time of the modules more evenly.

7. Use "while 1" for infinite loops

Sometimes you need an infinite loop in a program (such as an instance of a listening socket). Although "while true" can accomplish the same thing, "while 1" " is a single-step operation. This trick can improve your Python performance. >>> while 1:

>>> #do stuff, faster with while 1

>>> while True:

>>> # do stuff, slower with wile True

8. Use list comprehension

Starting with Python 2.0, you can use list comprehension to replace a lot of "for" and "while" blocks. Using list comprehension is usually faster, and the Python parser can It is optimized for finding a predictable pattern in the loop. The added benefit is that list comprehension is more readable (functional programming) and in most cases it saves an extra counting variable. For example, let’s count the even numbers between 1 and 10:

>>> # the good way to iterate a range

>>> evens = [ i for i in range(10) if i%2 == 0]

>>> [0, 2, 4, 6, 8]

>>> # the following is not so Pythonic

>>> i = 0

>>> evens = []

> >> while i

>>> if i %2 == 0: evens.append(i)

>>> i += 1

>>> [0, 2, 4, 6, 8 ]

9. Use xrange() to process long sequences:

This can save you a lot of system memory, because xrange() only produces one integer element per call in the sequence. In contrast, range() will give you a complete list of elements directly, which will cause unnecessary overhead when used for looping.

10. Use Python generator:

This can also save memory and improve performance. For example, for a video stream, you can send it in chunks of bytes instead of the entire stream. For example,

  >>> chunk = ( 1000 * i for i in xrange(1000))                                                              >>> chunk. next()

1000

>>> chunk.next()

2000

11. Understand the itertools module:

This module is very effective for iteration and combination. Let’s generate all permutations of a list [1, 2, 3] with just three lines of Python code:

>>> import itertools

>>> list(iter)

[(1, 2, 3), (1, 3, 2), (2, 1, 3), (2, 3, 1), (3, 1, 2) , (3, 2, 1)]

12. Learn the bisect module to keep the list sorted:

This is a free binary search implementation and a tool for quickly inserting ordered sequences. That is, you can use:

>>> import bisect

>>> bisect.insort(list, element)

You have inserted an element into the list, and you don’t need to call sort() again to keep it Sorting of containers, because this can be very expensive in long sequences.

13. Understand that a Python list is actually an array:

The list implementation in Python is not the ordinary singly linked list that people usually talk about in computer science realized. A list in Python is an array. That is, you can retrieve an element of a list in constant time O(1) without having to search from scratch. What's the point? Python developers need to think twice when using insert() of list objects. For example: >>> list.insert(0, item)

Inserting an element at the front of the list is not efficient, because all subsequent subscripts in the list have to Change. However, you can use list.append() to effectively add elements at the end of the list. Pick deque first if you want to be quick between inserts or when. It is fast because deque in Python is implemented as a doubly linked list. Say no more.

14. Test members using dict and set: Check if an element exists in a dictonary or set. This is very fast in Python. This is because dict and set are implemented using hash tables. The search efficiency can reach O(1). Therefore, if you need to check members frequently, use set or dict as your container.

>>> mylist = ['a', 'b', 'c'] #Slower, check membership with list:

> >> 'c' in mylist

>>> True

>>> myset = set(['a', 'b', 'c']) # Faster, check membership with set:

>>> ' c' in myset:

>>> True

15. Sort() using Schwartzian Transform:

The native list.sort() function is very fast. Python sorts lists in natural order. Sometimes, you need sorting in a non-natural order. For example, you want to sort IP addresses based on server location. Python supports custom comparisons, you can use list.sort(CMP()), which will be slower than list.sort() due to the added overhead of function calls. If performance is a problem, you can apply Guttman-Rosler Transform, based on Schwartzian Transform. It is only interested in the actual algorithm to be used, and the brief working principle of it is that you can transform the list and call Python's built-in list.sort() -> Faster without using list.sort(CMP()) -> Slower.

16. Python decorator cache results:

The “@” symbol is Python’s decoration syntax. It's not just for tracing, locking or logging. You can decorate a Python function to remember the result of the call for subsequent use. This technique is called memoization. Here is an example:

>>> from functools import wraps

>>> def memo(f):

>>> cache = { }

>>> @wraps(f)

>>> def wrap(*arg):

>>> if arg not in cache: cache['arg'] = f(*arg)

>>> return cache['arg']

>>> return wrap

We can also use decorators for the Fibonacci function:

  >>> @memo

    >>> def fib(i):

      >>>   if i

      >>>   return fib(i- 1) + fib(i-2)

The key idea here is: enhance the function (decoration) function to remember each Fibonacci value that has been calculated; if they are in the cache, there is no need to calculate them again.

17. Understand Python’s GIL (Global Interpreter Lock):

GIL is necessary because CPython’s memory management is not thread-safe. You can't simply create multiple threads and hope that Python will run faster on a multi-core machine. This is because the GIL will prevent multiple native threads from executing Python bytecode at the same time. In other words, the GIL will serialize all your threads. However, you can use threads to manage multiple spawned processes to speed up programs that run independently of your Python code.

18. Be as familiar with the Python source code as you are with the documentation:

Some Python modules are implemented in C for performance. When performance is critical and official documentation is insufficient, feel free to explore the source code. You can find the underlying data structures and algorithms. Python’s source code library is a great place: http://svn.python.org/view/python/trunk/Modules

Conclusion:

These are no substitutes for thinking with your brain. It’s the developer’s job to open the hood and fully understand so they don’t quickly cobble together a crap design. The Python suggestions in this article can help you get good performance. If it’s not fast enough, Python will need Leveraging external forces: Analyzing and running external code. We’ll cover that in the second part of this article.