Home System Tutorial LINUX Nginx link tracking for microservice architecture

Nginx link tracking for microservice architecture

Aug 06, 2024 pm 04:34 PM
linux linux tutorial Red Hat linux system linux command linux certification red hat linux linux video

Nginx link tracking for microservice architecture

In most microservice architectures, Nginx is basically a commonly used access layer facility, so we hope that the request ID will be verified and populated from the Nginx layer and printed in the Nginx request log.

Reading Tips: This article does not provide a complete solution for link tracking, but only provides a support solution for link tracking at the Nginx layer!

1 Background introduction

The birth of microservices has solved many problems of traditional monolithic applications, such as poor maintainability, poor scalability, and poor flexibility (coarse-grained comparison). Although the microservice architecture is good, it also brings many challenges, among which Troubleshooting is one of the challenges that needs to be solved. So, how do you find the root cause of a failure across many applications and instances?

Based on the above requirements, we can centrally collect and display all the logs generated by each transaction in each application (but only if you have: Log Center). This way you can quickly see at which step the transaction failed. If done well, you can also directly carry out secondary development and data analysis. After analyzing the collected logs and faults, they can be displayed intuitively with a graphical interface.

For example, you can display the topology diagram of microservice calls and use colors to distinguish faults (such as commonly used red: indicates abnormality, green: normal, yellow: warning). Then you can classify common faults or exceptions and make a friendly display (to put it bluntly, there is no need to go directly to the stack), such as: NullPointerException: The interface directly and friendly prompts which line of code threw a null pointer, and the input parameter is What... (This is not the focus of this article, so I won’t go into too much nonsense. I will introduce it in detail later when I have the opportunity).

To do link tracking of the entire microservice architecture, we definitely hope that there will be a global transaction ID to associate all logs from the first point when a transaction enters the microservice center (for link tracking, such an ID is definitely not enough , but only this is introduced here). Of course, the most ideal thing is to plan the front-end logs (such as operation logs, data flows, etc.).

2 Nginx

In most microservice architectures, Nginx is basically a commonly used access layer facility, so we hope that the request ID will be verified and populated from the Nginx layer and printed in the Nginx request log. Here are only three ways to implement the transaction ID production method of the Nginx layer.

2.1 Option 2: Splicing based on built-in variables

In versions prior to 1.11.0, we can use splicing to assemble the request ID. The reference configuration is as follows:

<span class="hljs-section">server</span> {
    <span class="hljs-comment"># 定义$request_trace_id的值,在1.11.0之前,我们可以使用类似的方式声明</span>
    <span class="hljs-comment"># 只要能确保其值出现重复的可能性尽可能的小即可。 </span>
    <span class="hljs-attribute">set</span> <span class="hljs-variable">$request_trace_id</span> trace-id-<span class="hljs-variable">$pid</span>-<span class="hljs-variable">$connection</span>-<span class="hljs-variable">$bytes_sent</span>-<span class="hljs-variable">$msec</span>;
    <span class="hljs-attribute">location</span> / {
        <span class="hljs-comment"># ……</span>
        <span class="hljs-comment"># 将此trace_id传递给后端的server,通过header方式,此后我们既可以在环境中获取此header  </span>
        <span class="hljs-attribute">proxy_set_header</span> X-Request-Id <span class="hljs-variable">$request_trace_id</span>;  
    }
}
Copy after login

Parameter description:

  • $pid: nginx worker process ID
  • $connection: The number of link ids to the upstream server
  • $bytes_sent: Number of bytes sent
  • $msec: The current time, that is, the time obtained by this variable, including seconds and milliseconds (divided by .)
2.2 Option 3: Implementation based on LUA script

Use the random UUID generated by the system /dev/urandom. The reference script is as follows:

<span class="hljs-comment">---</span>
 <span class="hljs-comment">--- UUID</span>
 <span class="hljs-comment">--- Created by lry.</span>
 <span class="hljs-comment">--- DateTime: 2018/2/25 下午7:38</span>
 <span class="hljs-comment">--- Describe: 用系统/dev/urandom生成的随机uuid</span>
 <span class="hljs-comment">---</span>
 <span class="hljs-keyword">local</span> template =<span class="hljs-string">"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"</span>
 <span class="hljs-keyword">local</span> d = <span class="hljs-built_in">io</span>.open(<span class="hljs-string">"/dev/urandom"</span>, <span class="hljs-string">"r"</span>):read(<span class="hljs-number">4</span>)
 <span class="hljs-built_in">math</span>.randomseed(<span class="hljs-built_in">os</span>.time() + d:byte(<span class="hljs-number">1</span>) + (d:byte(<span class="hljs-number">2</span>) * <span class="hljs-number">256</span>) + (d:byte(<span class="hljs-number">3</span>) * <span class="hljs-number">65536</span>) + (d:byte(<span class="hljs-number">4</span>) * <span class="hljs-number">4294967296</span>))
<span class="hljs-keyword">local</span> uuid=<span class="hljs-built_in">string</span>.gsub(template, <span class="hljs-string">"x"</span>,
  <span class="hljs-function"><span class="hljs-keyword">function</span> <span class="hljs-params">(c)</span></span>
    <span class="hljs-keyword">local</span> v = (c == <span class="hljs-string">"x"</span>) <span class="hljs-keyword">and</span> <span class="hljs-built_in">math</span>.random(<span class="hljs-number">0</span>, <span class="hljs-number">0xf</span>) <span class="hljs-keyword">or</span> <span class="hljs-built_in">math</span>.random(<span class="hljs-number">8</span>, <span class="hljs-number">0xb</span>)
    <span class="hljs-keyword">return</span> <span class="hljs-built_in">string</span>.format(<span class="hljs-string">"%x"</span>, v)
  <span class="hljs-keyword">end</span>)
<span class="hljs-keyword">return</span> uuid
Copy after login
2.3 Solution 1: Based on $request_id Implementation

Nginx在 1.11.0版本中就提供了内置变量 $request_id ,其原理就是生成32位的随机字符串,虽不能比拟UUID的概率,但32位的随机字符串的重复概率也是微不足道了,所以一般可视为UUID来使用即可。参考配置如下:

<span class="hljs-comment"># Nnginx代理默认会把header中参数的 "_" 下划线去掉,所以后台服务器后就获取不到带"_"线的参数名</span>
<span class="hljs-attribute">underscores_<span class="hljs-keyword">in</span>_headers</span> <span class="hljs-literal">on</span>;

<span class="hljs-comment"># 设定日志格式</span>
<span class="hljs-attribute"><span class="hljs-built_in">log</span>_format</span> main  \<span class="hljs-string">'<span class="hljs-variable">$remote_addr</span> - <span class="hljs-variable">$remote_user</span> [<span class="hljs-variable">$time_local</span>] "<span class="hljs-variable">$request</span>" \'
                 \'<span class="hljs-variable">$status</span> <span class="hljs-variable">$body_bytes_sent</span> "<span class="hljs-variable">$http_referer</span>" <span class="hljs-variable">$upstream_http_request_id</span> \'
                 \'"<span class="hljs-variable">$http_user_agent</span>" "<span class="hljs-variable">$http_x_forwarded_for</span>"\';

server {
    location / {
        <span class="hljs-comment"># 如果请求头中已有该参数,则获取即可;如果没有,则使用</span><span class="hljs-variable"><span class="hljs-comment">$request_id</span></span><span class="hljs-comment">进行填充</span>
        <span class="hljs-built_in">set</span> <span class="hljs-variable">$temp_request_id</span> <span class="hljs-variable">$http_x_request_id</span>;
        <span class="hljs-keyword">if</span> (<span class="hljs-variable">$temp_request_id</span> = "") {
            <span class="hljs-built_in">set</span> <span class="hljs-variable">$temp_request_id</span> <span class="hljs-variable">$request_id</span>;
        }
        <span class="hljs-comment"># 屏蔽掉原来的请求头参数</span>
        proxy_<span class="hljs-built_in">set</span>_header  x_request_id        "";
        <span class="hljs-comment"># 设置向后转发的请求头参数</span>
        proxy_<span class="hljs-built_in">set</span>_header  X-Request-Id        <span class="hljs-variable">$temp_request_id</span>;
    }
}
</span>
Copy after login
3 最佳实践

生成交易ID的方式有很多种,但希望使用者结合自身实际情况进行合理取舍,而不要盲目的追求ID的唯一性、可读性和时序性等等。

比如,ID具有时序性虽然有一定的好处,但实际的架构根本没有去使用该时序性,则没必要花大量的精力和做出大量的开发,去实现一个有时序性的交易ID。又比如,觉得UUID可读性太差,从而花了很多成本去开发一个具有一定含义的交易ID(如前几位表示什么意思,多少位到多少位又表示什么意思之类的),开发出来后,实际架构根本没有去解读该ID的地方,则浪费了成本。

但也不是所有人都直接使用UUID就能满足的,比如我需要考虑日志的容量,则可以考虑适当缩减ID的长度(每个ID缩减10个字符串,每笔交易就可能少几百或几千个字符串,再往上规划,还是可以减少一些日志容量的)。

最后,如果有考虑想收集前端的日志的童鞋,建议交易ID就不要使用Long型,因为前端可能会有损失精度的问题。同时也建议使用 $request_id  来填充交易ID。

The above is the detailed content of Nginx link tracking for microservice architecture. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
1 months ago By 尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. Best Graphic Settings
1 months ago By 尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. How to Fix Audio if You Can't Hear Anyone
1 months ago By 尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. Chat Commands and How to Use Them
1 months ago By 尊渡假赌尊渡假赌尊渡假赌

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Difference between centos and ubuntu Difference between centos and ubuntu Apr 14, 2025 pm 09:09 PM

The key differences between CentOS and Ubuntu are: origin (CentOS originates from Red Hat, for enterprises; Ubuntu originates from Debian, for individuals), package management (CentOS uses yum, focusing on stability; Ubuntu uses apt, for high update frequency), support cycle (CentOS provides 10 years of support, Ubuntu provides 5 years of LTS support), community support (CentOS focuses on stability, Ubuntu provides a wide range of tutorials and documents), uses (CentOS is biased towards servers, Ubuntu is suitable for servers and desktops), other differences include installation simplicity (CentOS is thin)

How to install centos How to install centos Apr 14, 2025 pm 09:03 PM

CentOS installation steps: Download the ISO image and burn bootable media; boot and select the installation source; select the language and keyboard layout; configure the network; partition the hard disk; set the system clock; create the root user; select the software package; start the installation; restart and boot from the hard disk after the installation is completed.

Centos stops maintenance 2024 Centos stops maintenance 2024 Apr 14, 2025 pm 08:39 PM

CentOS will be shut down in 2024 because its upstream distribution, RHEL 8, has been shut down. This shutdown will affect the CentOS 8 system, preventing it from continuing to receive updates. Users should plan for migration, and recommended options include CentOS Stream, AlmaLinux, and Rocky Linux to keep the system safe and stable.

Detailed explanation of docker principle Detailed explanation of docker principle Apr 14, 2025 pm 11:57 PM

Docker uses Linux kernel features to provide an efficient and isolated application running environment. Its working principle is as follows: 1. The mirror is used as a read-only template, which contains everything you need to run the application; 2. The Union File System (UnionFS) stacks multiple file systems, only storing the differences, saving space and speeding up; 3. The daemon manages the mirrors and containers, and the client uses them for interaction; 4. Namespaces and cgroups implement container isolation and resource limitations; 5. Multiple network modes support container interconnection. Only by understanding these core concepts can you better utilize Docker.

Centos options after stopping maintenance Centos options after stopping maintenance Apr 14, 2025 pm 08:51 PM

CentOS has been discontinued, alternatives include: 1. Rocky Linux (best compatibility); 2. AlmaLinux (compatible with CentOS); 3. Ubuntu Server (configuration required); 4. Red Hat Enterprise Linux (commercial version, paid license); 5. Oracle Linux (compatible with CentOS and RHEL). When migrating, considerations are: compatibility, availability, support, cost, and community support.

What to do after centos stops maintenance What to do after centos stops maintenance Apr 14, 2025 pm 08:48 PM

After CentOS is stopped, users can take the following measures to deal with it: Select a compatible distribution: such as AlmaLinux, Rocky Linux, and CentOS Stream. Migrate to commercial distributions: such as Red Hat Enterprise Linux, Oracle Linux. Upgrade to CentOS 9 Stream: Rolling distribution, providing the latest technology. Select other Linux distributions: such as Ubuntu, Debian. Evaluate other options such as containers, virtual machines, or cloud platforms.

How to use docker desktop How to use docker desktop Apr 15, 2025 am 11:45 AM

How to use Docker Desktop? Docker Desktop is a tool for running Docker containers on local machines. The steps to use include: 1. Install Docker Desktop; 2. Start Docker Desktop; 3. Create Docker image (using Dockerfile); 4. Build Docker image (using docker build); 5. Run Docker container (using docker run).

How to mount hard disk in centos How to mount hard disk in centos Apr 14, 2025 pm 08:15 PM

CentOS hard disk mount is divided into the following steps: determine the hard disk device name (/dev/sdX); create a mount point (it is recommended to use /mnt/newdisk); execute the mount command (mount /dev/sdX1 /mnt/newdisk); edit the /etc/fstab file to add a permanent mount configuration; use the umount command to uninstall the device to ensure that no process uses the device.

See all articles