search

Home >Database >Redis >Redis source code analysis and in-depth understanding of Makefile files

Redis source code analysis and in-depth understanding of Makefile files

青灯夜游
青灯夜游forward
2022-01-27 10:26:562221browse

This article will talk about Redis source code compilation and analyze the Makefile file in depth and detail. I hope it will be helpful to everyone!

Redis source code analysis and in-depth understanding of Makefile files

To study this article, you need to have the Redis source code, and it is best to set up the relevant compilation environment, so that you can intuitively see the MakefileThe execution process of the file. The source code version used here is redis-6.2.1. [Related recommendations: Redis video tutorial]

Makefile file details

The content of the Makefile file in the source code root directory is as follows : 

default: all

.DEFAULT:
	cd src && $(MAKE) $@

install:
	cd src && $(MAKE) $@

.PHONY: install

The following information can be seen from the code:

  • The first target of this file is default, which has no actual effect. Depends on the all target
  • There is no so-called all target in the code, so when we use make directly, ## will be called first #default target, and then call the all target. Since the all target does not exist, the .DEFAULT target will be called instead, in the execution statement of the Makefile , $@ represents the target, $(MAKE) represents make, so the expanded code is as follows, readers can compile it by themselves , see if the first output statement is the same as what we analyzed 
    • cd src && make all
    The install goal is similar to the previous one, and ultimately it goes into the
  • src/ directory, and then calls the The only difference between the Makefile files in the directory is that the target of the call becomes install. The expanded code is as follows: 
    • cd src && make install
    When the incoming parameter is other, the call will go to
  • .DEFAULT, and then call the corresponding target of Makefile in the subdirectory to cleanFor example, the code is as follows: 
    • cd src && make clean

Detailed explanation of src/Makefile

This file is the file that really plays a compilation role. It has a lot of content and is complicated. , and in order to be compatible with a variety of compilers, there are many branch selection syntaxes. We only use the

gcc compiler under Linux as an example to explain. There is no difference in the rest, just through The judgment statement is just to change some compilation parameters

1. Makefile.dep target

Makefile is executing the corresponding Before the target, non-target instructions will be executed first, such as variable assignment, Shell statements, etc., so we will find that the Makefile file will not be executed completely in order. The related code is as follows: 

NODEPS:=clean distclean

# FINAL_CFLAGS里的各个变量原型
STD=-pedantic -DREDIS_STATIC=''
WARN=-Wall -W -Wno-missing-field-initializers
OPTIMIZATION?=-O2
OPT=$(OPTIMIZATION)
DEBUG=-g -ggdb
#CFLAGS 根据条件选择的，不重要的参数，忽略
#REDIS_CFLAGS 根据条件选择的，不重要的参数，忽略

FINAL_CFLAGS=$(STD) $(WARN) $(OPT) $(DEBUG) $(CFLAGS) $(REDIS_CFLAGS)
REDIS_CC=$(QUIET_CC)$(CC) $(FINAL_CFLAGS)

all: $(REDIS_SERVER_NAME) $(REDIS_SENTINEL_NAME) $(REDIS_CLI_NAME) $(REDIS_BENCHMARK_NAME) $(REDIS_CHECK_RDB_NAME) $(REDIS_CHECK_AOF_NAME)
	@echo ""
	@echo "Hint: It's a good idea to run 'make test' ;)"
	@echo ""

Makefile.dep:
	-$(REDIS_CC) -MM *.c > Makefile.dep 2> /dev/null || true

ifeq (0, $(words $(findstring $(MAKECMDGOALS), $(NODEPS))))
-include Makefile.dep
endif

First add the following points

Makefilebasis

    ##Makefile# The usage format of ##'s
  • findstring function is $(findstring FIND, IN), which means to find FIND in IN. If If it arrives, it will return FIND. If it cannot find it, it will return empty. Makefile
    • 's
  • words function indicates the number of words, for example, $( words, foo bar)'s return value is "2"Makefile
    • 's
  • MAKECMDGOALS variable represents the incoming parameters (all )Makefile
  • CCThe default value is ccMakefile
  • - MM is to output a rule for make, which describes the dependencies of source files, but does not include system header files
    • can be summarized The following information is output: The
all
target in

    is the default compilation target we mentioned in the previous section, but we can try to compile it ourselves, You will find that the
  • Makefile.dep file is generated first, because it first executes the bottom judgment statement, which calls the Makefile.dep target. Because at this time The value of MAKECMDGOALS
    • is
  • all, which is not in the range of NODEPS, so the above ifeq statement is established and Makefile.dep will be called TargetThe value of REDIS_CC
    • is composed of three variables.
  • QUIET_CC prints debugging information. Readers can go to the source code to see the relevant content. This part does not Important, we ignore that the value of CC represents the compiler, and the value in FINAL_CFLAGS is some parameters of the compilation. These values ​​have been extracted from the above codeIn summaryMakefile.dep
    • The function of the goal is to generate the dependencies of all files ending with
  • .c in the current directory and write them into Makefile. In the dep file, the content of the file generated after compilation is as follows. It looks quite messy, but the content inside actually lists the target files ultimately generated by each source file, and lists the dependencies it requires. <pre class="brush:js;toolbar:false;">acl.o: acl.c server.h fmacros.h config.h solarisfixes.h rio.h sds.h \ connection.h atomicvar.h ../deps/lua/src/lua.h ../deps/lua/src/luaconf.h \ ae.h monotonic.h dict.h mt19937-64.h adlist.h zmalloc.h anet.h ziplist.h \ intset.h version.h util.h latency.h sparkline.h quicklist.h rax.h \ redismodule.h zipmap.h sha1.h endianconv.h crc64.h stream.h listpack.h \ rdb.h sha256.h adlist.o: adlist.c adlist.h zmalloc.h ae.o: ae.c ae.h monotonic.h fmacros.h anet.h zmalloc.h config.h \ ae_epoll.c ae_epoll.o: ae_epoll.c ... zipmap.o: zipmap.c zmalloc.h endianconv.h config.h zmalloc.o: zmalloc.c config.h zmalloc.h atomicvar.h</pre>
2. The general target file generation target

code is as follows: 

.make-prerequisites:
	@touch $@

ifneq ($(strip $(PREV_FINAL_CFLAGS)), $(strip $(FINAL_CFLAGS)))
.make-prerequisites: persist-settings
endif

ifneq ($(strip $(PREV_FINAL_LDFLAGS)), $(strip $(FINAL_LDFLAGS)))
.make-prerequisites: persist-settings
endif

%.o: %.c .make-prerequisites
	$(REDIS_CC) -MMD -o $@ -c $<

The following is the code Analysis of this part of the code:

  • 这部分是通用的根据源文件生成目标文件的target，Makefile中%表示通配符，所以只要符合格式要求的都可以借助这段代码来生成对应的目标文件
  • .make-prerequisites没啥用忽略，而REDIS_CC的值在上一小节有说明了，是用于编译文件的指令
  • gcc的-MMD参数与前面说的那个-MM是基本一致的，只不过这个会将输出内容导入到对应的%.d文件中
  • Makefile中$@表示目标，$<表示第一个依赖，$^表示全部依赖
  • 综上，这个target的作用是依赖于一个源文件，然后根据这个源文件生成对应的目标文件，并且将依赖关系导入到对应的%.d文件中

下面是一个简单的例子：

# 假设生成的目标文件为acl.o，则代入可得
acl.o: acl.c .make-prerequisites
	$(REDIS_CC) -MMD -o acl.o -c acl.c

# 执行完成后在该目录下会生成一个acl.o文件和acl.d文件

3、all目标所依赖的各个子目标的名称设置

PROG_SUFFIX的值默认为空，可以忽略。这里设置的六个目标名都是会被all这个目标引用的，从名字可以看出这六个目标是对应着Redis不同的功能，依次是服务、哨兵、客户端、基础检测、rdf持久化以及aof持久化。
代码如下：

REDIS_SERVER_NAME=redis-server$(PROG_SUFFIX)
REDIS_SENTINEL_NAME=redis-sentinel$(PROG_SUFFIX)
REDIS_CLI_NAME=redis-cli$(PROG_SUFFIX)
REDIS_BENCHMARK_NAME=redis-benchmark$(PROG_SUFFIX)
REDIS_CHECK_RDB_NAME=redis-check-rdb$(PROG_SUFFIX)
REDIS_CHECK_AOF_NAME=redis-check-aof$(PROG_SUFFIX)

4、all目标所依赖的各个子目标的内容

  • REDIS_LD也是一个编译指令，和前面那个REDIS_CC有点像，只不过这个指定了另外的一些编译参数，比如设置了某些依赖的动态库、静态库的路径，读者有兴趣的话可以去看一下代码，看看REDIS_LD的详细内容
  • FINAL_LIBS是一系列动态库链接参数，读者有兴趣可以自行去Makefile里面查看该变量的内容，限于篇幅原因这里就不展开讲了
  • 将QUIET_INSTALL忽略（这个是自定义打印编译信息的），可以看出REDIS_INSTALL的值其实就是install，Linux下的install命令是用于安装或升级软件或备份数据的，这个命令与cp类似，但是install允许你控制目标文件的属性，这里不作深入分析了，有兴趣的读者可以自行查阅相关的介绍install命令的文章。基本用法为：install src des，表示将src文件复制到des文件去
    代码如下：
REDIS_SERVER_OBJ=adlist.o quicklist.o ae.o anet.o dict.o server.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o cluster.o crc16.o endianconv.o slowlog.o scripting.o bio.o rio.o rand.o memtest.o crcspeed.o crc64.o bitops.o sentinel.o notify.o setproctitle.o blocked.o hyperloglog.o latency.o sparkline.o redis-check-rdb.o redis-check-aof.o geo.o lazyfree.o module.o evict.o expire.o geohash.o geohash_helper.o childinfo.o defrag.o siphash.o rax.o t_stream.o listpack.o localtime.o lolwut.o lolwut5.o lolwut6.o acl.o gopher.o tracking.o connection.o tls.o sha256.o timeout.o setcpuaffinity.o monotonic.o mt19937-64.o
REDIS_CLI_OBJ=anet.o adlist.o dict.o redis-cli.o zmalloc.o release.o ae.o crcspeed.o crc64.o siphash.o crc16.o monotonic.o cli_common.o mt19937-64.o
REDIS_BENCHMARK_OBJ=ae.o anet.o redis-benchmark.o adlist.o dict.o zmalloc.o release.o crcspeed.o crc64.o siphash.o crc16.o monotonic.o cli_common.o mt19937-64.o

DEP = $(REDIS_SERVER_OBJ:%.o=%.d) $(REDIS_CLI_OBJ:%.o=%.d) $(REDIS_BENCHMARK_OBJ:%.o=%.d)
-include $(DEP)

INSTALL=install
REDIS_INSTALL=$(QUIET_INSTALL)$(INSTALL)

# redis-server
$(REDIS_SERVER_NAME): $(REDIS_SERVER_OBJ)
	$(REDIS_LD) -o $@ $^ ../deps/hiredis/libhiredis.a ../deps/lua/src/liblua.a $(FINAL_LIBS)

# redis-sentinel
$(REDIS_SENTINEL_NAME): $(REDIS_SERVER_NAME)
	$(REDIS_INSTALL) $(REDIS_SERVER_NAME) $(REDIS_SENTINEL_NAME)

# redis-check-rdb
$(REDIS_CHECK_RDB_NAME): $(REDIS_SERVER_NAME)
	$(REDIS_INSTALL) $(REDIS_SERVER_NAME) $(REDIS_CHECK_RDB_NAME)

# redis-check-aof
$(REDIS_CHECK_AOF_NAME): $(REDIS_SERVER_NAME)
	$(REDIS_INSTALL) $(REDIS_SERVER_NAME) $(REDIS_CHECK_AOF_NAME)

# redis-cli
$(REDIS_CLI_NAME): $(REDIS_CLI_OBJ)
	$(REDIS_LD) -o $@ $^ ../deps/hiredis/libhiredis.a ../deps/linenoise/linenoise.o $(FINAL_LIBS)

# redis-benchmark
$(REDIS_BENCHMARK_NAME): $(REDIS_BENCHMARK_OBJ)
	$(REDIS_LD) -o $@ $^ ../deps/hiredis/libhiredis.a ../deps/hdr_histogram/hdr_histogram.o $(FINAL_LIBS)

4.1、REDIS_SERVER_NAME目标

该目标依赖于REDIS_SERVER_OBJ，而REDIS_SERVER_OBJ的内容都是一些目标文件（上面代码有给出），这些目标文件最终都会通过3.2小节介绍的那个target来生成。可以看到REDIS_SERVER_NAME这个target需要使用REDIS_SERVER_OBJ、…/deps/hiredis/libhiredis.a、…/deps/lua/src/liblua.a以及FINAL_LIBS这些来编译链接生成最终的目标文件，即redis-server

4.2、REDIS_SENTINEL_NAME目标

可以看到REDIS_SENTINEL_NAME目标很简单，只是简单地使用install命令复制了REDIS_SERVER_NAME目标生成的那个文件，即redis-server，从这里可以知道哨兵服务redis-sentinel与Redis服务使用的是同一套代码

4.3、REDIS_CHECK_RDB_NAME目标

和前面的如出一辙，也是简单复制了redis-server文件到redis-check-rdb文件去

4.4、REDIS_CHECK_AOF_NAME目标

和前面的如出一辙，也是简单复制了redis-server文件到redis-check-aof文件去

4.5、REDIS_CLI_NAME目标

这个就不是简单复制了，而是使用和REDIS_SERVER_NAME目标相同的方法进行直接编译的，唯一的区别是REDIS_SERVER_NAME链接了…/deps/lua/src/liblua.a，而REDIS_CLI_NAME链接的是…/deps/linenoise/linenoise.o

4.6、REDIS_BENCHMARK_NAME目标

这个也是使用和REDIS_SERVER_NAME目标相同的方法进行直接编译的，唯一的区别是REDIS_SERVER_NAME链接了…/deps/lua/src/liblua.a，而REDIS_BENCHMARK_NAME链接的是…/deps/hdr_histogram/hdr_histogram.o

5、all目标

经过前面的介绍，all目标的作用也就一目了然了，最终会生成六个可执行文件，以及输出相应的调试信息
代码如下：

all: $(REDIS_SERVER_NAME) $(REDIS_SENTINEL_NAME) $(REDIS_CLI_NAME) $(REDIS_BENCHMARK_NAME) $(REDIS_CHECK_RDB_NAME) $(REDIS_CHECK_AOF_NAME)
	@echo ""
	@echo "Hint: It&#39;s a good idea to run &#39;make test&#39; ;)"
	@echo ""

6、安装和卸载Redis的目标

6.1、安装Redis的目标

这里逻辑很简单，先创建一个用于存放Redis可执行文件的文件夹（默认是/usr/local/bin），然后将REDIS_SERVER_NAME、REDIS_BENCHMARK_NAME、REDIS_CLI_NAME对应的可执行文件复制到/usr/local/bin中去，这里可以看到前面那几个照葫芦画瓢的文件并没有复制过去，而是直接通过创建软连接的方式去生成对应的可执行文件（内容相同，复制过去浪费空间）
代码如下：

PREFIX?=/usr/local
INSTALL_BIN=$(PREFIX)/bin

install: all
	@mkdir -p $(INSTALL_BIN)
	$(REDIS_INSTALL) $(REDIS_SERVER_NAME) $(INSTALL_BIN)
	$(REDIS_INSTALL) $(REDIS_BENCHMARK_NAME) $(INSTALL_BIN)
	$(REDIS_INSTALL) $(REDIS_CLI_NAME) $(INSTALL_BIN)
	@ln -sf $(REDIS_SERVER_NAME) $(INSTALL_BIN)/$(REDIS_CHECK_RDB_NAME)
	@ln -sf $(REDIS_SERVER_NAME) $(INSTALL_BIN)/$(REDIS_CHECK_AOF_NAME)
	@ln -sf $(REDIS_SERVER_NAME) $(INSTALL_BIN)/$(REDIS_SENTINEL_NAME)

6.2、卸载Redis的目标

这里就是删除前面复制的那些文件了，比较简单，就不细讲了
代码如下：

uninstall:
	rm -f $(INSTALL_BIN)/{$(REDIS_SERVER_NAME),$(REDIS_BENCHMARK_NAME),$(REDIS_CLI_NAME),$(REDIS_CHECK_RDB_NAME),$(REDIS_CHECK_AOF_NAME),$(REDIS_SENTINEL_NAME)}

7、clean和distclean目标

所有Makefile的clean或者distclean目标的作用都是大致相同的，就是删除编译过程中产生的那些中间文件，以及最终编译生成的动态库、静态库、可执行文件等等内容，代码比较简单，就不作过多的分析了
代码如下：

clean:
	rm -rf $(REDIS_SERVER_NAME) $(REDIS_SENTINEL_NAME) $(REDIS_CLI_NAME) $(REDIS_BENCHMARK_NAME) $(REDIS_CHECK_RDB_NAME) $(REDIS_CHECK_AOF_NAME) *.o *.gcda *.gcno *.gcov redis.info lcov-html Makefile.dep dict-benchmark
	rm -f $(DEP)

.PHONY: clean

distclean: clean
	-(cd ../deps && $(MAKE) distclean)
	-(rm -f .make-*)

.PHONY: distclean

8、test目标

执行完Redis编译之后，会有一段提示文字我们可以运行make test测试功能是否正常，从代码中我们可以看出其实不止一个test目标，还有另一个test-sentinel目标，这个是测试哨兵服务的。这两个目标分别运行了根目录的runtest和runtest-sentinel文件，这两个是脚本文件，里面会继续调用其他脚本来完成整个功能的测试，并输出测试信息到控制台。具体怎么测试的就不分析了，大家有兴趣的可以去看一下。
代码如下：

test: $(REDIS_SERVER_NAME) $(REDIS_CHECK_AOF_NAME) $(REDIS_CLI_NAME) $(REDIS_BENCHMARK_NAME)
	@(cd ..; ./runtest)

test-sentinel: $(REDIS_SENTINEL_NAME) $(REDIS_CLI_NAME)
	@(cd ..; ./runtest-sentinel)

总结

本文详细地分析了与Redis编译相关的Makefile文件，通过学习Makefile文件里的内容，我们可以更为全面地了解Redis的编译过程，因为Makefile文件中将很多编译命令用@给取消显示了，转而使用它自己特制的编译信息输出给我们看，代码如下：

ifndef V
QUIET_CC = @printf &#39;    %b %b\n&#39; $(CCCOLOR)CC$(ENDCOLOR) $(SRCCOLOR)$@$(ENDCOLOR) 1>&2;
QUIET_LINK = @printf &#39;    %b %b\n&#39; $(LINKCOLOR)LINK$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR) 1>&2;
QUIET_INSTALL = @printf &#39;    %b %b\n&#39; $(LINKCOLOR)INSTALL$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR) 1>&2;
endif

所以我们直接去编译的话很多细节会看不到，可以自己尝试修改Makefile文件，在前面这段代码之前定义V变量，这样就可以看到完整的编译信息了。修改如下：

V = 'good'

ifndef V
QUIET_CC = @printf &#39;    %b %b\n&#39; $(CCCOLOR)CC$(ENDCOLOR) $(SRCCOLOR)$@$(ENDCOLOR) 1>&2;
QUIET_LINK = @printf &#39;    %b %b\n&#39; $(LINKCOLOR)LINK$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR) 1>&2;
QUIET_INSTALL = @printf &#39;    %b %b\n&#39; $(LINKCOLOR)INSTALL$(ENDCOLOR) $(BINCOLOR)$@$(ENDCOLOR) 1>&2;
endif

本人之前也写过Nginx编译相关的文章，下面总结两者的几点区别：

  • Nginx使用了大量的Shell相关的技术，而Redis则很少使用这些
  • Nginx跨平台的相关参数是通过配置脚本进行配置的，而Redis则是直接在Makefile文件中将这件事给做了，这两者没有什么优劣之分，Nginx主要是为了可扩展性强才使用那么多配置脚本的，而Redis基本不用考虑这些，所以简单一点实现就行了
  • 由于Redis将其一些逻辑都放在了Makefile文件中了，所以看起来Nginx最终生成的Makefile文件要比Redis简单易懂很多（Nginx复杂逻辑在那些配置脚本里）
  • Nginx生成的配置文件足有1000多行，代码量比Redis的400多行要大很多，因为Nginx把全部依赖的生成方式全部列举了出来，而Redis借助了Makefile.dep、各种%.d文件来将依赖信息分散到中间文件中去，极大地减少了Makefile的代码量

本文转载自：https://blog.csdn.net/weixin_43798887/article/details/117674538

更多编程相关知识，请访问：编程入门！！

The above is the detailed content of Redis source code analysis and in-depth understanding of Makefile files. For more information, please follow other related articles on the PHP Chinese website!

lua nginx sentinel .net default redis https linux
Statement：
This article is reproduced at:csdn.net. If there is any infringement, please contact admin@php.cn delete
Previous article：A brief analysis of single-threaded and multi-threaded models in Redis6Next article：A brief analysis of single-threaded and multi-threaded models in Redis6

Related articles

See more