Integrate Lua in nginx to develop web services

Background introduction

During project development, a service we handled before stored the generated data in redis, and the client obtained the specific data in redis through a specific key. In previous development, the architecture solution of nginx+wsgi+python was adopted. Projects can also be quickly implemented through python, and pushing to the test environment has been in use.
As time passed, I slowly thought about the project and found that this implementation method also had certain drawbacks. Because there is no complicated logic for the service, nginx receives the request and forwards it to the background python service; then the python service gets the specific request, fetches the data from redis and returns it to the client for calling. The whole process is actually relatively simple and clear. There is no need to add another layer of python services between nginx and redis, so I am thinking about whether the implementation of this architecture can be optimized.

nginx and lua

Later, after a simple search for nginx+redis, I found some keywords related to nginx+lua+openresty. The next step is to start researching openresty. Looking at the relevant documents, we found that openresty is a project set that integrates nginx core modules and various third-party modules. If you use openresty, you avoid installing various additional third-party packages, and they are all inherited into its installation package. This is definitely a good choice for lazy people.

Introduction to lua

lua is a scripting language, also known as glue language. It is a very small language that can be easily coupled with other languages. Through Lua's own characteristics, coupled with the language it is attached to, using the characteristics of two languages ​​​​in one service is definitely not as powerful as usual. I have seen some awesome games using Lua as a glue-type functional language.
However, through simple use, I found that Lua's string processing is not powerful enough, because it directly uses the C library, and its string processing is only the interface provided by the C library, without additional enrichment and supplementation of the interface. In many scenarios, you need to implement some common interfaces yourself, such as split.

Integrating lua and ngxin

Since lua is a glue-type language, can lua be directly coupled in nginx? The answer is very yes. Lua can run directly in nginx to do some logical processing and log control. Then we can consider using nginx+lua to develop a web service to meet the needs and complete the convenience and rapid development that other server-side languages ​​cannot complete.
In addition to the traversal mentioned above, what advantages will nginx+lua bring:
1. Reduce one layer of forwarding and use other service languages ​​to develop services. A protocol will definitely be used to communicate directly between nginx and the server. Such as cgi, fcig, wsgi, etc. If you use lua, because lua runs directly in nginx, it is necessary to do an additional nginx forwarding.
2. Event-based response, because Lua is a runtime environment that directly runs nginx, so Lua inherits all the features of nginx. Under normal circumstances, nginx provides services based on events, select or epoll. It will definitely be awesome in terms of performance.
Based on the above reasons, nginx+lua was started. Because lazy people directly adopt the entire openresty installation strategy, of course you can also install nginx separately, then install lua, and then nginx_lua_module. After installing openresty, I started writing simple tests. In fact, there are related examples on openresty's github. You can directly do some simple test development based on the examples.
After completing the first step of installation and writing test examples, lua+nginx means that the integration has been successful. The next step is to develop your own business logic. What needs to be noted here is that openresty actually integrates nginx, so running two nginx on one machine may have corresponding problems. Therefore, after installing openresty, the existing nginx on the machine may have a certain impact.

Upgrade the original service

The development environment has been configured. The next step is to directly develop the business logic, that is, receive the request to access redis, read the data and return the client request. Because Lua can be written directly in the nginx configuration file, but this is not a good strategy. Although Lua is directly coupled with other languages, the degree of coupling must also consider software engineering-related issues. For example, the maintainability of later code and the readability of nginx configuration files.
In view of the above reasons, it is also recommended that the function implementation of Lua be written into a separate file, and then be declared in the nginx configuration file to tell nginx to load and run. On the basis of implementing functions, try to avoid the degree of coupling and maintainability of the code.
In the specific implementation, two files handle the entire service, of course because the service itself is simple.
-redis.conf #redis host, port
-init.lua #Initialization configuration file
The concept of nginx's shared memory is utilized in the implementation.

<code><span>--redis.conf</span>
host:<span>127.0</span><span>.0</span><span>.1</span>
port:<span>6379</span><span>--init.lua</span>
tmp = {}
<span>for</span> l <span>in</span> io.<span>lines</span>(<span>"lua/redis.conf"</span>) <span>do</span><span>for</span> i <span>in</span><span>string</span>.gmatch(l, <span>"([^:]+)"</span>) <span>do</span>
                table.insert(tmp, i)
        <span><span>end</span></span><span><span>end</span></span>
ngx.shared.config:<span>set</span>(tmp[<span>1</span>], tmp[<span>2</span>])
ngx.shared.config:<span>set</span>(tmp[<span>3</span>], tmp[<span>4</span>])</code>

Here we need to read redis.conf when nginx starts, so we need to add a configuration to nginx to tell nginx that init.lua needs to be executed when it starts. In addition, the shared memory config of nginx must be declared, so the configuration of nginx is as follows

<code>lu<span>a_shared</span>_dict config <span>1</span>m<span>;</span>
init_by_lu<span>a_file</span> 'lua/init.lua'<span>;</span></code>

第一步已经完成，就是redis相关配置的读取，共享内存的声明和初始化，那接下来就是具体的逻辑实现，几十行代码分分钟搞定。

<code>     location  /vector{
                content_by_lua '
                        <span>local</span> redis = require <span>"resty.redis"</span><span>local</span> server = redis:new()
                        <span>local</span> conf = ngx.shared.config 

                        <span>local</span> ok, err = server:connect(conf:<span>get</span>(<span>"host"</span>), conf:<span>get</span>(<span>"port"</span>))
                        ngx.header.content_type = <span>"text/plain"</span><span>if</span><span>not</span> ok <span>then</span>
                                ngx.<span>log</span>(ngx.ERR, err)
                                ngx.<span>exit</span>(ngx.HTTP_SERVICE_UNAVAILABLE)
                        <span>end</span><span>local</span> x = ngx.var.arg_x;
                        <span>local</span> y = ngx.var.arg_y;
                        <span>local</span> z = ngx.var.arg_z;

                        <span>if</span> x == nil <span>or</span> y == nil <span>or</span> z == nil <span>then</span>                                ngx.<span>say</span>(<span>"{\\\"ret\\\": -1}"</span>)
                                ngx.<span>exit</span>(ngx.HTTP_SERVICE_UNAVAILABLE)
                        <span>end</span><span>local</span> key = z..<span>"_"</span>..x..<span>"_"</span>..y
                        <span>local</span> data = server:<span>get</span>(key)
                ';
        }</code>