Backend Development
Python Tutorial
Solving the problem of Django database connection loss (explanation with examples)Solving the problem of Django database connection loss (explanation with examples)
The content of this article is about solving the problem of Django database connection loss (explanation with examples). It has certain reference value. Friends in need can refer to it. I hope it will be helpful to you.
Problem
When using mysql in Django, the database connection may occasionally be lost. The errors usually include the following two types
1. `OperationalError: (2006, 'MySQL server has gone away')` 1. `OperationalError: (2013, 'Lost connection to MySQL server during query')`
Query mysql global variables SHOW GLOBAL VARIABLES ;You can see wait_timeout, this variable represents the connection idle time. If the client uses one connection to query the database multiple times, there will be no problem if the query is continuous. If the query pauses for more than wait_timeout after several queries, the database connection will be lost and the database connection will be lost.
Reproduction
Use Django to reproduce the problem next time:
1. Set the wait_timeout of mysql to 10 seconds, and then enter the django shell simulation Query (only part of the following error message is retained)
In[1]:import time In[2]:from django.contrib.auth.models import User In[3]:list(User.objects.filter(id=1)) Out[3]:[<user:>] In[4]:time.sleep(15) # 模拟比较慢的代码(其中没有查询数据库的代码),或者空闲什么都不操作一段时间,此时间要比`wait_timeout`大一些 list(User.objects.filter(id=1)) Traceback (most recent call last): File "<ipython-input-4-3574ae8220ee>", line 1, in <module> list(User.objects.filter(id=1)) File "/usr/lib/python3.6/site-packages/pymysql/connections.py", line 1037, in _read_bytes CR.CR_SERVER_LOST, "Lost connection to MySQL server during query") django.db.utils.OperationalError: (2013, 'Lost connection to MySQL server during query')</module></ipython-input-4-3574ae8220ee></user:>
Seeking
Then the above problem basically shows that the error is caused by too long idle time.
In order to reduce unnecessary database connections and closures, Django reuses database connections. When a request is started, a connection pool is established to store the connection. After that, a connection is reused for each request. My guess is that Django saves the connection longer than wait_timeout. If the save time is shorter, the connection can be re-established to avoid this error.
Yes, the official document has also explained this problem, setting the database CONN_MAX_AGE parameter, example:
DATABASES = {
"default": {
'ENGINE': 'django.db.backends.mysql',
'NAME': '',
'USER': '',
'PASSWORD': '',
'HOST': '',
'CONN_MAX_AGE': 9 # 比wait_timeout小一些
}
}
When we tested it, we found that things were not as simple as we thought. Why does the error still occur? Behind all this, is it the distortion of human nature or the loss of morality? Please watch the next episode "Breakthrough".
Breakthrough
After searching CONN_MAX_AGE in the django source code, I followed the clues and found out how django closes the failed connectiondjango. db.close_old_connections():
# Register an event to reset transaction state and close connections past # their lifetime. def close_old_connections(**kwargs): for conn in connections.all(): conn.close_if_unusable_or_obsolete() signals.request_started.connect(close_old_connections) signals.request_finished.connect(close_old_connections)
The focus is on the last two lines. This method is executed when specific events are implemented through signal. The two specific events, as the name implies, are the start of the request and the end of the request. The error we reported was in one request, so this method is usually ineffective. It only closes and re-establishes the connection for each request.
Solution
Do not close the django shell that reproduces the problem. Continue to execute the following code:
In[5]:from django.db import close_old_connections In[6]:close_old_connections() In[7]:list(User.objects.filter(id=1)) Out[7]: [<user:>]</user:>
Call django.db.close_old_connections and query again. No more errors.
Then if we want to avoid this error, we must call the django.db.close_old_connections method before executing each database query.
1. Generally, this kind of problem will not occur, because the database query is continuously performed in one request, and there is no need to call this method for every request. This is a groundless worry.
2. Sometimes there is a large amount of data in a request, and the database will be queried and then other processing (not involving the database) will be performed for a period of time. For example, some data will be queried first, then the data will be processed, excel will be generated, the file will be saved, and Generate url. It is known that this takes a very long time, so it is best to call django.db.close_old_connections first to prevent the connection from being lost when the final URL is saved in the database.
The above is the detailed content of Solving the problem of Django database connection loss (explanation with examples). For more information, please follow other related articles on the PHP Chinese website!
Refactoring Python Code EffectivelyJul 24, 2025 am 03:38 AMRefactoring is not rewriting, but improving the code structure and readability without changing the function. Common reconstruction situations include too large functions or classes, many repetitive codes, fuzzy variable naming, and complex control processes. Refactoring should start with details, such as splitting large functions, extracting duplicate code, simplifying conditional judgment, and improving variable naming. Tools and testing are the key. Using pytest, black, isort, flake8, mypy and other tools to cooperate with unit testing can ensure that the changes are safe. Refactoring should be continuously optimized from a small way, rather than rewriting it all at once.
Python Memory Management ExplainedJul 24, 2025 am 03:38 AMPython's memory management consists of automatic allocation and recycling mechanisms. When creating variables, memory will be allocated from the memory pool or system malloc according to the object size. Small objects preferentially use memory pools to improve efficiency. Memory recycling mainly relies on reference counting and garbage collector (gc module). Reference counting is zeroed and memory is released, while circular references are processed by garbage collector. To reduce memory usage, array, NumPy array, generator, and \_\_slots\_\_\_ can be used. The memory is not released immediately at the end of the del or function, which may be caused by garbage collection delay, external memory usage or object cache. You can use tracemalloc or memory\_profiler tools to analyze the memory situation.
python recursion exampleJul 24, 2025 am 03:36 AMRecursion is a method for function calls to solve problems in Python, and is suitable for scenarios such as factorial, Fibonacci sequence, nested list traversal and binary search. 1. Factorial is recursively calculated by n*factorial(n-1), and the basic situation is n==0 or 1, and the basic situation is n==0 or 1; 2. The Fibonacci sequence defines f(n)=f(n-1) f(n-2), and the basic situation is f(0)=0, f(1)=1, but the naive recursive efficiency is low, and it is recommended to use lru_cache to optimize; 3. When traversing the nested list, if the elements are lists, they will be processed recursively, otherwise they will be printed; 4. The binary search recursive version looks for the target value in an ordered array, and determines the recursive left and right intervals based on the comparison between the intermediate value and the target. The basic situation is low>hig
Customizing Logging Handlers in PythonJul 24, 2025 am 03:33 AMThe core of custom loggingHandler is to inherit logging.Handler and implement the emit() method, which is suitable for scenarios such as sending logs to emails, writing to databases, or pushing remote servers. 1. The situations that need to be customized include: pushing logs to Slack or DingTalk, recording to database or API, processing by level, and adding additional information; 2. The implementation method is to inherit logging.Handler and rewriting emit(), where you write custom logic such as sending HTTP requests; 3. When using it, you need to pay attention to exception handling, formatting output, setting appropriate levels and formatters, and avoid duplicate output and propagation problems.
What is the difference between python `break` and `continue`?Jul 24, 2025 am 03:33 AMIn Python, the difference between break and continue is that: 1.break is used to terminate the entire loop immediately, which is often used to exit the loop early or complete the search task; 2.continue only skips the current iteration and continues to execute the next loop, which is suitable for ignoring specific elements or filtering data. For example, use break after finding a match when searching a list, and skip invalid entries with continue when cleaning data. Although both control the cycle flow, their functions are completely different.
How to flatten a list of lists in PythonJul 24, 2025 am 03:32 AMThere are three ways to tile nested lists in Python: First, use list comprehension, the syntax is [itemforsublistinlist_of_listsforiteminsublist], which is suitable for two-dimensional lists; Second, use itertools.chain, which includes itertools.chain.from_iterable(list_of_lists) or itertools.chain(*list_of_lists), which has better performance; Third, when dealing with irregular nesting, judgment statements need to be added, for example, using isinstance(sublist, list) to distinguish lists from non-
python threading lock exampleJul 24, 2025 am 03:29 AMThreading.Lock is needed to prevent race conditions for shared resources in multi-threading environments. 1. Create lock object lock=threading.Lock(); 2. Use withlock: Ensure the operation atomicity of shared variables; 3. Multiple threads accumulate 100,000 times for counters, and the final result is correct 500,000; 4. It is recommended to use the with statement to automatically manage the acquisition and release of locks; 5. Avoid nested acquisition of locks, and use threading.RLock() if necessary; 6. The scope of the lock should be as small as possible to improve performance; 7. Pay attention to avoid deadlocks due to inconsistent locking order.
Scientific Computing with PythonJul 24, 2025 am 03:25 AMPython is widely used in scientific computing because its mature libraries and tool chains can handle various tasks. Key points include: 1. Install core libraries such as NumPy (efficient arrays and mathematical functions), SciPy (advanced mathematical operations), Matplotlib (data visualization) and Pandas (table data processing), which can be installed through pip or conda; 2. Replace native lists with NumPy to improve performance, and support vectorized operations, broadcast mechanisms and linear algebra functions; 3. SciPy provides complex mathematical tools such as integral, optimization, and Fourier transform, such as using quad function to calculate definite integrals; 4. Matplotlib and its encapsulation library Seaborn are used for graph display, supporting style settings and professionalism


Hot AI Tools

Undress AI Tool
Undress images for free

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

ZendStudio 13.5.1 Mac
Powerful PHP integrated development environment

Notepad++7.3.1
Easy-to-use and free code editor

SublimeText3 Chinese version
Chinese version, very easy to use

mPDF
mPDF is a PHP library that can generate PDF files from UTF-8 encoded HTML. The original author, Ian Back, wrote mPDF to output PDF files "on the fly" from his website and handle different languages. It is slower than original scripts like HTML2FPDF and produces larger files when using Unicode fonts, but supports CSS styles etc. and has a lot of enhancements. Supports almost all languages, including RTL (Arabic and Hebrew) and CJK (Chinese, Japanese and Korean). Supports nested block-level elements (such as P, DIV),

MantisBT
Mantis is an easy-to-deploy web-based defect tracking tool designed to aid in product defect tracking. It requires PHP, MySQL and a web server. Check out our demo and hosting services.






