Detailed introduction to Python Web framework Tornado operation and deployment

The example in this article shares the details of the operation and deployment of the Python Web framework Tornado for your reference. The specific content is as follows

1. Operation and deployment
Because Tornado has its own HTTPServer built in, running and deploying it is not the same as other Python web frameworks. You need to write a main() function to start the service instead of configuring a WSGI container to run your application:

def main():
  app = make_app()
  app.listen(8888)
  IOLoop.current().start()

if __name__ == '__main__':
  main()

Configure your operating system or process Manager to run this program to start the service. Note that it may be necessary to increase the maximum number of file handles allowed to be open per process (to avoid "Too many open files" errors). To increase this limit (e.g. to 50000) you can use the ulimit command, modify /etc/security/limits.conf or set minfds in your supervisord configuration.

2. Processes and ports
Due to Python's GIL (Global Interpreter Lock), in order to make full use of multi-CPU machines, it is necessary to run multiple Python processes. Generally, it's best to run one process per CPU.

Tornado includes a built-in multi-process mode to start multiple processes at once, which requires a slight change in the main function:

def main():
  app = make_app()
  server = tornado.httpserver.HTTPServer(app)
  server.bind(8888)
  server.start(0) # forks one process per cpu
  IOLoop.current().start()

This is the simplest way to start multiple processes and have them share the same port, although it has some limitations. First, each child process will have its own IOLoop, so it is important not to touch the global IOLoop instance before forking (even indirectly). Secondly, in this model, it is difficult to achieve zero-downtime updates. Finally, because all processes share the same port, it is more difficult to monitor them individually.

For more complex deployments, it is recommended to start independent processes and let them each listen to different ports. The "process groups" function of supervisord is a good way. When each process uses a different port, an external load balancer such as HAProxy or nginx usually needs to provide a single address to outbound visitors.

3. Running behind a load balancer
When running behind a load balancer such as nginx, it is recommended to pass xheaders=True to the constructor of HTTPServer. This will tell Tornado to use HTTP headers like X-Real-IP to obtain the user's IP address instead of assuming all traffic comes from the load balancer's IP address.

This is an original nginx configuration file, similar in structure to the configuration we use in FriendFeed. This assumes that nginx and Tornado server are running on the same machine, and four Tornado servers are running on ports 8000 - 8003:

user nginx;
worker_processes 1;
 
error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;
 
events {
  worker_connections 1024;
  use epoll;
}
 
http {
  # Enumerate all the Tornado servers here
  upstream frontends {
    server 127.0.0.1:8000;
    server 127.0.0.1:8001;
    server 127.0.0.1:8002;
    server 127.0.0.1:8003;
  }
 
  include /etc/nginx/mime.types;
  default_type application/octet-stream;
 
  access_log /var/log/nginx/access.log;
 
  keepalive_timeout 65;
  proxy_read_timeout 200;
  sendfile on;
  tcp_nopush on;
  tcp_nodelay on;
  gzip on;
  gzip_min_length 1000;
  gzip_proxied any;
  gzip_types text/plain text/html text/css text/xml
        application/x-javascript application/xml
        application/atom+xml text/javascript;
 
  # Only retry if there was a communication error, not a timeout
  # on the Tornado server (to avoid propagating "queries of death"
  # to all frontends)
  proxy_next_upstream error;
 
  server {
    listen 80;
 
    # Allow file uploads
    client_max_body_size 50M;
 
    location ^~ /static/ {
      root /var/www;
      if ($query_string) {
        expires max;
      }
    }
    location = /favicon.ico {
      rewrite (.*) /static/favicon.ico;
    }
    location = /robots.txt {
      rewrite (.*) /static/robots.txt;
    }
 
    location / {
      proxy_pass_header Server;
      proxy_set_header Host $http_host;
      proxy_redirect off;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Scheme $scheme;
      proxy_pass http://frontends;
    }
  }
}

4. Static files and file caching
In Tornado, you can provide static file services by specifying a special static_path in the application:

settings = {
  "static_path": os.path.join(os.path.dirname(__file__), "static"),
  "cookie_secret": "__TODO:_GENERATE_YOUR_OWN_RANDOM_VALUE_HERE__",
  "login_url": "/login",
  "xsrf_cookies": True,
}
application = tornado.web.Application([
  (r"/", MainHandler),
  (r"/login", LoginHandler),
  (r"/(apple-touch-icon\.png)", tornado.web.StaticFileHandler,
   dict(path=settings['static_path'])),
], **settings)

These settings will automatically route all requests starting with /static/ to the static directory. For example, //m.sbmmt.com/:8888/static/foo.png will provide foo through the specified static directory. png file. We also automatically serve /robots.txt and /favicon.ico from the static directory (although they do not start with the /static/ prefix).

In the above settings, we explicitly configure Tornado to obtain the apple-touch-icon.png file from the StaticFileHandler root, although the file is in the static file directory. (The regex capture group must tell the StaticFileHandler the filename requested, and calling the capture group passes the filename as a method argument to the handler.) You could do the same thing, such as serving a sitemap.xml file from the root of the website. Of course, you can also avoid forging the root's apple-touch-icon.png by using the tag in your HTML.

To improve performance, it is usually a good idea to let the browser actively cache static resources so that the browser does not send unnecessary If-Modified-Since or Etag requests that may block when rendering the page. Yes, Tornado uses static content versioning to support this feature.

To use these features, use the static_url method in your template instead of entering the URL of the static file directly in your HTML:

<html>
  <head>
    <title>FriendFeed - {{ _("Home") }}</title>
  </head>
  <body>
    <p><img src="{{ static_url("images/logo.png") }}"/></p>
  </body>
</html>

static_url() function will translate the relative path into a URI similar to /static/images/logo.png?v=aae54. The v parameter is the hash of the logo.png content, and its existence Causes the Tornado service to send cache headers to the user's browser, which will cause the browser to cache the content indefinitely.

Because the parameter v is based on the file content, if you update a file and restart the service, it will send a new v value, so the user's browser will automatically pull the new file. If the contents of the file have not changed, the browser will continue to use the locally cached copy without checking for updates from the server, significantly improving rendering performance.

In production, you may want to serve static files through a better static server, such as nginx. You can configure any web server to recognize the version tag provided via static_url() and set the cache header accordingly. The following is part of the nginx related configuration we use in FriendFeed:

location /static/ {
  root /var/friendfeed/static;
  if ($query_string) {
    expires max;
  }
 }

五、Debug模式和自动重载
如果传递 debug=True 配置给 Application 的构造函数，应用程序将会运行在debug/开发模式。 在这个模式下，为了方便于开发的一些功能将被启用( 每一个也可以作为独立的标签使用，如果它们都被专门指定，那它们都将获得独立的优先级):

1、autoreload=True: 应用程序将会观察它的源文件是否改变，并且当任何文件改变的时候便重载它自己。这减少了在开发中需要手动重启服务的需求。然而，在debug模式下，某些错误(例如import的时候有语法错误)会导致服务 关闭，并且无法自动恢复。
2、compiled_template_cache=False: 模板将不会被缓存。
3、static_hash_cache=False: 静态文件哈希 (被 static_url 函数使用) 将不会被缓存。
4、serve_traceback=True: 当一个异常在 RequestHandler 中没有捕获，将会生成一个包含调用栈信息的错误页。
自动重载(autoreload)模式和 HTTPServer 的多进程模式不兼容，你不能给 HTTPServer.start 传递 1 以外的参数(或者调用 tornado.process.fork_processes) 当你使用自动重载模式的时候。

debug模式的自动重载功能可作为一个独立的模块位于 tornado.autoreload。以下两者可以结合使用，在语法错误之时提供额外的健壮性: 设置 autoreload=True 可以在app运行时检测文件修改，还有启动 python -m tornado.autoreload myserver.py 来捕获任意语法错误或者其他的启动时错误。

重载会丢失任何Python解释器命令行参数(-u). 因为它使用 sys.executable 和 sys.argv 重新执行Python。此外，修改这些变量将造成重载错误。

在一些平台(包括Windows 和Mac OSX 10.6之前)，进程不能被“原地”更新，所以当检测到代码更新，旧服务就会退出然后启动一个新服务。这已经被公知来混淆一些IDE。

六、WSGI和Google App Engine
Tornado通常是独立运行的，不需要一个WSGI容器。然而，在一些环境中 (例如Google App Engine)，只运行WSGI，应用程序不能独立运行自己的服务。在这种情况下，Tornado支持一个有限制的操作模式，不支持异步操作但允许一个Tornado's功能的子集在仅WSGI环境中。以下功能在WSGI模式下是不支持的，包括协程，@asynchronous 装饰器，AsyncHTTPClient，auth 模块和WebSockets。

你可以使用 tornado.wsgi.WSGIAdapter 把一个Tornado Application 转换成WSGI应用。在这个例子中， 配置你的WSGI容器发 现 application 对象:

import tornado.web
import tornado.wsgi

class MainHandler(tornado.web.RequestHandler):
  def get(self):
    self.write("Hello, world")

tornado_app = tornado.web.Application([
  (r"/", MainHandler),
])
application = tornado.wsgi.WSGIAdapter(tornado_app)

以上就是本文的全部内容，希望对大家的学习有所帮助。

更多PythonWeb框架Tornado运行和部署详细介绍相关文章请关注PHP中文网！