Nginx's Web caching service and Sina's open source NCACHE module

<span><span><span>#</span>Nginx的Web缓存服务与新浪网的开源NCACHE模块</span><span></span></span><span><span><span>##</span>什么是web缓存</span><span></span><span> Web缓存位于内容源web服务器和客户端之间，当用户访问一个	URL时，web缓存服务器回去后端web源服务器取回要输出的内容，然后，当下一个请求到来时，如果访问的是相同的URL，web缓存服务器直接输出内容给客户端，而不是像源服务器再次发送请求。web缓存降低了内容源web服务器、数据库的负载，减轻了网络延迟，提高了用户的响应速度，增强了用户体验。</span><span></span><span>最著名的还要数Squid Cache，其主要在Unix一类系统运行。</span><span></span></span><span><span><span>##</span>Nginx的Web缓存服务</span><span></span><span>Nginx从0.7.48后支持类似于Squid的缓存模块。这个缓存是把URL及相关组合当做key，用md5算法对key进行希哈，得到硬盘上对应的希哈路径，从而将缓存内容保存在该目录内。支持任意URL链接。同时也支持404/301/302这样的非200状态码。</span><span></span><span>Nginx的Web缓存服务主要用于proxy_cache相关指令集和fastcgi相关指令集构成，前者用于反向代理时，对后端内容源进行缓存，后者主要用于对FastCDI的动态程序进行缓存。两者功能基本一样。</span><span></span></span><span><span><span>###</span>proxy_cache相关指令集</span><span></span><span><span><span>**</span>1、proxy_cache指令<span>**</span></span></span><span></span><span>语法：proxy<span><span>_</span>cache zone<span>_</span></span>name;</span><span></span><span>默认值：none</span><span></span><span>使用环境：http,server,location</span><span></span><span>该指令用于设置那个缓存区将被应用，zone<span><span>_</span>name的值为proxy<span>_</span></span>cache_path指令创建的缓存区明称。</span><span></span><span><span><span>**</span>2、proxy_cache_path指令<span>**</span></span></span><span></span><span>语法：proxy<span><span>_</span>cache<span>_</span></span>path path<span><span><span>[</span><span>levels=number</span><span>]</span></span>keys_z <span><span>[</span><span>max_size=size</span><span>]</span></span></span>;</span><span></span><span>默认值：none</span><span></span><span>使用环境：HTTP</span><span></span><span><span><span>**</span>eg：<span>**</span></span></span><span></span><span>proxy<span><span>_</span>cache<span>_</span></span>path /data0/proxy<span><span>_</span>cache<span>_</span></span>dir  levels=1:2 keys<span><span>_</span>z>_</span></span>one: 500m inactive=1d max_size=30g;</span><span></span><span>注意该指令只能在http标签内配置，levels指定该缓存有两层hash目录，第一层为1个字母，第二层为2个字母，保存文件名类似于/data0/proxy<span><span>_</span>cache<span>_</span></span>dir/c/29/fdg35415fg35f4gsdf2g1535gh465h;key<span><span>_</span>zone参数用来为缓存区起名，500m指定内存空间大小为500MB；inactive的1d是如果缓存数据在1天之内没有被访问，将被删除；max<span>_</span></span>size的30g是指硬盘的缓存空间为30GB。</span><span></span><span><span><span>**</span>3proxy_cache_methods指令<span>**</span></span></span><span></span><span>语法：proxy<span><span>_</span>cache<span>_</span></span>methods [GET HEAD POST];</span><span></span><span>默认值：proxy<span><span>_</span>cache<span>_</span></span>methods GET HEAD;</span><span></span><span>使用环境：http，server，location</span><span></span><span>该指令用于设置用于缓存那些HTTP方法，默认缓存 HTTP GET/HEAD 方法，不缓存HTTP POST方法。</span><span></span><span><span><span>**</span>4proxy_cache_min_uses指令<span>**</span></span></span><span></span><span>语法：proxy<span><span>_</span>cache<span>_</span></span>min<span><span>_</span>uses the<span>_</span></span>number；</span><span></span><span>默认值：proxy<span><span>_</span>cache<span>_</span></span>min_uses 1；</span><span></span><span>使用环境：http，server，location</span><span></span><span>该指令设置缓存最小的使用次数，默认值是1.</span><span></span><span><span><span>**</span>5、proxy_cache_valid指令<span>**</span></span></span><span></span><span>语法：proxy<span><span>_</span>cache<span>_</span></span>valid reply<span><span>_</span>code [reply<span>_</span></span>code...]time;</span><span></span><span>默认值：none</span><span></span><span>使用环境：http，server，location</span><span></span><span>该指令用于对不同的返回状态码的URL设置不同的缓存时间，例如：</span><span></span><span>proxy<span><span>_</span>cache<span>_</span></span>valid 200 302 10m；</span><span></span><span>proxy<span><span>_</span>cache<span>_</span></span>valid 404      1m;</span><span></span><span>如果不指定状态吗，直接指定时间，则只有200、301、302状态的URL缓存5分钟。</span><span></span><span><span><span>**</span>6、proxy_cache_key指令<span>**</span></span></span><span></span><span>语法：proxy<span><span>_</span>cache<span>_</span></span>key line;</span><span></span><span>默认值：none</span><span></span><span>使用环境：http，server，location</span><span></span><span>该指令用来设置web缓存的key值，Nginx根据key值md5希哈存储缓存。一般根据<span><span>`</span>‘$host(域名)、$request_uri(请求路径)’<span>`</span></span>等组合变量合成proxy<span><span>_</span>cache<span>_</span></span>key.例如：<span><span>`</span>proxy_cache_key "$host:$server_port$uri$is_args$args";<span>`</span></span></span><span></span><span><span><span>##</span>proxy_cache完整示例</span><span><span></span><span></span>	su<span></span>	yum -y install pcre//安装pcre<span></span>	wget http://labs.frickle.com/files/ngx_cache_purge-2.3.tar.gz<span></span>	tar zxvf ngx_cache_purge-2.3.tar.gz//获取nginx_cache_purge<span></span>	cd nginx-1.6.3//进入你的nginx文件目录（nginx安装请参考前面的博客）<span></span>	 ./configure --user=www --group=www --addmodule=../ngx_cache_purge-2.3 --prefix=/usr/local/webserver/nginx --with-http_stub_status_module --with-http_ssl_module<span></span></span><span></span><span><span><span>**</span>配置nginx.conf<span>**</span></span></span><span></span><span><span><span>**</span>cd /usr/local/webserver/nginx/conf<span>**</span></span></span><span></span><span></span></span><span><span><span>```</span><span></span>#user  www www;<span></span>worker_processes  1;<span></span><span></span>#error_log  logs/error.log;<span></span>#error_log  logs/error.log  notice;<span></span>#error_log  logs/error.log  info;<span></span><span></span>#pid        logs/nginx.pid;<span></span><span></span><span></span>events {<span></span>    use epoll;<span></span>    worker_connections  1024;<span></span>}<span></span><span></span><span></span>http {<span></span>    include       mime.types;<span></span>    default_type  application/octet-stream;<span></span><span></span>    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '<span></span>    #                  '$status $body_bytes_sent "$http_referer" '<span></span>    #                  '"$http_user_agent" "$http_x_forwarded_for"';<span></span><span></span>    #access_log  logs/access.log  main;<span></span><span></span>    #charset utf-8;<span></span><span></span>    server_name_hash_bucket_size 128;<span></span>    client_header_buffer_size 32k;<span></span>    large_client_header_buffers 4 32k;<span></span><span></span>    sendfile        on;<span></span>    #tcp_nopush     on;<span></span><span></span>    keepalive_timeout  30;<span></span><span></span>    tcp_nodely  on;<span></span><span></span>    proxy_temp_path /data0/proxy_temp_path;<span></span><span></span>    proxy_temp_path /data0/proxy_temp_path levels=1:2 key_z inactive=1d max_size=30g;<span></span>    upstream my_sever_pool{<span></span>    	server 192.168.1.2:80 weight=1 max_fails=2 fail_timeout=30s;<span></span>    	server 192.168.1.3:80 weight=1 max_fails=2 fail_timeout=30s;<span></span>    	server 192.168.1.4:80 weight=1 max_fails=2 fail_timeout=30s;<span></span><span></span>	}<span></span><span></span><span></span><span></span>    #gzip  on;<span></span><span></span>    server {<span></span>        listen       80;<span></span>        server_name  localhost;<span></span><span></span>        #charset koi8-r;<span></span><span></span>        #access_log  logs/host.access.log  main;<span></span><span></span>        location / {<span></span>            proxy_set_header Host $host;<span></span>	    proxy_set_header X-Forward-For $remote_addr;<span></span>	    proxy_pass http://my_server_pool;<span></span>	   # root   html;<span></span>            #index  index.html index.htm;<span></span>        }<span></span>	location ~ .*\.(gif|jpg|jpeg|png|bmp|swf|js|css)$<span></span>	{<span></span>		#使用web缓存区cache_one<span></span>		proxy_cache cache_one;<span></span><span></span>		#对不同状态码设置不同缓存时间<span></span>		proxy_cache_valid 200 304 12h;<span></span>		proxy_cache_valid 301 302 1m;<span></span>		proxy_cache_valid any im;<span></span>		#设置web缓存的key值，nginx根据key值md5希哈存储缓存，这里根据“域名/URL 参数”组合成key。<span></span>		proxy_cache_key $host$uri$is_args$args;<span></span>		#反向代理，访问后端内容源服务器<span></span>		proxy_set_header Host $host;<span></span>		proxy_set_header X-Forwarded-For $remote_addr;<span></span>		proxy_pass http:my_server_pool;<span></span>	}<span></span>	#用于清除缓存，假设一个URL为http://my.domain.com/text.gif通过访问http://my.domain.com/purge/test.gif可以清除该URK缓存。<span></span>	location ~ /purge(/.*)<span></span>	{<span></span>		#设定只允许指定的IP或IP段才可以清除URL缓存。<span></span>		allow 		127.0.0.1<span></span>		allow		192.168.0.0/16;<span></span>		deny		all;<span></span>		proxy_cache_purge cache_one $shot$1$is-args$args;<span></span>	}<span></span>	access_log 0ff<span></span><span></span>        #error_page  404              /404.html;<span></span><span></span>        # redirect server error pages to the static page /50x.html<span></span>        #<span></span>        error_page   500 502 503 504  /50x.html;<span></span>        location = /50x.html {<span></span>            root   html;<span></span>        }<span></span><span></span>        # proxy the PHP scripts to Apache listening on 127.0.0.1:80<span></span>        #<span></span>        #location ~ \.php$ {<span></span>        #    proxy_pass   http://127.0.0.1;<span></span>        #}<span></span><span></span>        # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000<span></span>        #<span></span>        #location ~ \.php$ {<span></span>        #    root           html;<span></span>        #    fastcgi_pass   127.0.0.1:9000;<span></span>        #    fastcgi_index  index.php;<span></span>        #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;<span></span>        #    include        fastcgi_params;<span></span>        #}<span></span><span></span>        # deny access to .htaccess files, if Apache's document root<span></span>        # concurs with nginx's one<span></span>        #<span></span>        #location ~ /\.ht {<span></span>        #    deny  all;<span></span>        #}<span></span>    }<span></span><span></span><span></span>    # another virtual host using mix of IP-, name-, and port-based configuration<span></span>    #<span></span>    #server {<span></span>    #    listen       8000;<span></span>    #    listen       somename:8080;<span></span>    #    server_name  somename  alias  another.alias;<span></span><span></span>    #    location / {<span></span>    #        root   html;<span></span>    #        index  index.html index.htm;<span></span>    #    }<span></span>    #}<span></span><span></span><span></span>    # HTTPS server<span></span>    #<span></span>    #server {<span></span>    #    listen       443 ssl;<span></span>    #    server_name  localhost;<span></span><span></span>    #    ssl_certificate      cert.pem;<span></span>    #    ssl_certificate_key  cert.key;<span></span><span></span>    #    ssl_session_cache    shared:SSL:1m;<span></span>    #    ssl_session_timeout  5m;<span></span><span></span>    #    ssl_ciphers  HIGH:!aNULL:!MD5;<span></span>    #    ssl_prefer_server_ciphers  on;<span></span><span></span>    #    location / {<span></span>    #        root   html;<span></span>    #        index  index.html index.htm;<span></span>    #    }<span></span>    #}<span></span><span></span>}<span></span><span>```</span></span><span></span><span></span></span>