研究了下nginx+tomcat+memcached 的配置,练习了下脚本的书写,参考了好几篇文章,在此记录一下,有不对的地方敬请指正。
系统环境:
RHEL6.5 x64
iptables -F & selinux is disabled
主机角色:
node1 :192.168.122.101 :nginx tomcat memcached
node2 :192.168.122.102 :tomcat memcached
https://code.google.com/memcached-session-manager
memcached的session管理
nginx做反向代理两台tomcat,用memcached同步session,防止数据丢失
Tomcat1将session存储在memcacted2上.tomcat和Memcached使用交叉存储,只有当M2不可用时T1才将数据存储在M1上(M1是T1的failoverNode),使用这种方式可以避免单点故障.从而实现应用的高可用性.
注意:两台node上的tomcat应完全一致
1.在两台node主机上配置tomcat环境
#./jdk-6u32-linux-x64.bin# mv jdk1.6.0_32/ /usr/local/lnmp/jdk#vim /etc/profile 编写环境变量export JAVA_HOME=/usr/local/lnmp/jdk export CLASSPATH=:$JAVA_HOME/lib:$JAVA_HOME/jre/lib export PATH=$PATH:$JAVA_HOME/bin #source /etc/profile测试java能否正常工作 #vim test.javapublicclasstest{publicstaticvoid main(String[] args) {System.out.println("Hello!"); } } #javac test.java 编译#java test 执行后出现Hello!说明java环境配置好
安装tomcat服务器
#tar zxf apache-tomcat-7.0.37.tar.gz 解压就能用,不需要编译#mv apache-tomcat-7.0.37 tomcat#/TOMCAT_ROOT_DIR/tomcat/webapps/ROOT tomcat的默认发布目录#/TOMCAT_ROOT_DIR/tomcat/bin/startup.sh(shutdown.sh) tomcat默认的启动和关闭脚本
tomcat默认开启8080端口,
测试http://192.168.122.101:8080 访问到tomcat默认的测试页
#cd tomcat/webapps/ROOT#cat test.jsp #测试页this time is: new java.util.Date()%>
测试访问;http://192.168.122.101:8080 显示当前时间
tomcat访问8080端口,使用nginx反向代理。
使用nginx发布jsp动态网页:
nginx配置文件:
cat /usr/local/nginx/conf/nginx.conf
#user nginx nginx; worker_processes 4; #error_log logs/error.log;#error_log logs/error.log notice;#error_log logs/error.log info;#pid logs/nginx.pid;events { use epoll; worker_connections 1024; } http { upstream tomcat { #两台tomcat负载均衡 sticky; #session同步,为nginx的模块server192.168.0.1:8080; server192.168.0.2:8080; } include mime.types; default_type application/octet-stream; #log_format main '$remote_addr - $remote_user [$time_local] "$request" '# '$status $body_bytes_sent "$http_referer" '#access_log logs/access.log main; sendfile on; #tcp_nopush on;#keepalive_timeout 0; keepalive_timeout 65; #gzip on;server { listen 80; server_name localhost; #charset koi8-r;#access_log logs/host.access.log main; location / { root html; proxy_pass http://tomcat;indexindex.html index.htm; } #error_page 404 /404.html;# redirect server error pages to the static page /50x.html error_page 500502503504 /50x.html; location = /50x.html { root html; } location ~ \.jsp$ { proxy_pass http://tomcat; } } nginx -t && nginx -s reload
访问:http://192.168.122.101/test.jsp 测试
同步两台tomcat的内容,并且修改java环境变量/etc/profile
测试:http://192.168.122.102:8080/test.jsp
两台node(tomcat)准备好了
2、在node1上的nginx环境中发布网页
使用nginx的负载均衡功能,
体现在nginx.conf中:
upstream tomcat-lb { server192.168.122.101:8080; server192.168.122.102:8080; } location~ \.jsp$ { proxy_passhttp://tomcat-lb; } nginx -t && nginx -s reload
测试:http://192.168.122.101/test.jsp
实现两台主机的负载均衡(访问nginx所在的node)
问题:在后台实现了nginx的负载均衡,但是当一个用户刷新数据时,数据总是变化的,试想,在动态页面,如果用户提交数据刷新后并没有提交到服务器上,并且必须要重新填写表格,这样会造成糟糕的用户体验。
解决:给nginx增加一个sticky模块.(需要重新编译nginx)
重新在nginx在进行模块化编译:
nginx-sticky-modules.tar.gz(解压就行)
#tar zxf nginx-sticky-modules.tar.gz -C /root/nginx-1.4.2#cd nginx-1.4.2/#make clean#./configure --prefix=/usr/local/nginx --add-module=$NGINX_PKG_DIR/$DIR/nginx-sticky-module-1.0 --with-http_ssl_module --with-http_stub_status_module#make && make install#vim nginx.confupstream linux { sticky; #装了nginx-sticky后的功能 server 192.168.122.101:8080 ; server 192.168.122.102:8080 ; }
测试:http://192.168.122.101/test.jsp 后发现刷新不会来回负载(每个用户看到的其实不是一个tomcat上的数据,但是对用户是透明的)
3.nginx负载tomcat的jsp时,需要解决session共享:
使用memcache进行缓存(用户)后端数据,但是又要想到解决单点故障问题,因而可以采用两台memcache作为后端负载.
memcached默认端口11211,后端使用交叉存储(tomcat会将session同步,session自动寻找存储的memcached,但是默认是交叉存储,当一个memcached服务器坏掉,tomcat都会存到存活的memcached服务器上)
只要tomcat不宕掉,一切数据都还存在
但是当memcached宕掉,tomcat会向存活的memcached上存取
session 的序列化方案官方推荐的有 4 种:
java serialization
msm-kryo-serializer
msm-javolution-serializer
msm-xstream-serializer
其中性能最好的是Kryo,我们使用kryo来做
mecached服务器node1 and node2
#yum install memcached -y#/etc/init.d/memcached start
memcached默认开启11211端口
测试访问:telnet localhost 11211
stats 查看状态
set user 0 0 3 存储新值(add replace)
get user 获取值
配置两台memcached服务器(tomcat),下载jar包(必须支持相关的java程序)
提供测试环境的包,解压即可用:
链接: http://pan.baidu.com/s/1mgIF9NU 密码: n5wu
asm-3.2.jar
couchbase-client-1.2.2.jar
kryo-1.03.jar
kryo-serializers-0.11.jar
memcached-session-manager-1.6.5.jar
memcached-session-manager-tc7-1.6.5.jar
minlog-1.2.jar
msm-kryo-serializer-1.6.5.jar
reflectasm-0.9.jar
spymemcached-2.10.3.jar
node1 and node2 同步内容
#cd tomcat/lib/#mget jar/* 下载tomcat的session共享管理包#cd tomcat/conf/# vim context.xml"de.javakaffee.web.msm.MemcachedBackupSessionManager" memcachedNodes="n1:192.168.122.101:11211,n2:192.168.122.102:11211" failoverNodes="n1" #tomcat2需要写成n2 requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$" transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory" /> #tomcat/bin/shutdown.sh #重启tomcat,以识别memcached-session-manager# tail -f logs/catalina.out #默认日志INFO: MemcachedSessionService finished initialization, sticky true, operation timeout 1000, with node ids [n2] and failover node ids [n1]
正常启动
编辑测试文件:(提交用户信息的jsp测试页)
# vim tomcat/webapps/ROOT/test.jsp "text/html; charset=GBK" %>"java.util.*" %>html>head>title>Cluster App Testtitle>head>body>Server Info: request.getLocalAddr() + " : " + request.getLocalPort()+"
");%>"
ID " + session.getId()+"
"); String dataName = request.getParameter("dataName"); if (dataName != null && dataName.length() > 0) { String dataValue = request.getParameter("dataValue"); session.setAttribute(dataName, dataValue); } out.print("Session list"); Enumeration e = session.getAttributeNames(); while (e.hasMoreElements()) { String name = (String)e.nextElement(); String value = session.getAttribute(name).toString(); out.println( name + " = " + value+"
"); System.out.println( name + " = " + value); } %>formaction="test.jsp"method="POST">name:inputtype=textsize=20name="dataName">br>key:inputtype=textsize=20name="dataValue">br>inputtype=submit>form>body>html>
session共享完成:两台tomcat和memcached做相同的配置(jdk,memcache,tomcat)
测试:
http://192.168.122.101/test.jsp
使用一台tomcat和另外一台的memcached进行session共享
任何一台tomcat或者memcached挂了都无所谓
session会记录并且保持用户的数据信息
我同时写了一键安装的脚本,还有一些问题,也分享出来,希望有大神能指点~~
第一个,在有nginx的机器上运行:
#!/bin/bash setenforce 0 > /dev/null iptables -F > /dev/null sed -i 's/^SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config ############### nginx + tomcat + memcachedDIR_NOW=`pwd` IPADDR_NTM='192.168.122.101' IPADDR_TM='192.168.122.102'NGINX_PKG_DIR='/root/one_key_install' NGINX_PKG_NAME='nginx-1.6.1.tar.gz' NGINX_DIR='/usr/local/nginx'TOMCAT_PKG_DIR='/root/one_key_install' TOMCAT_PKG_NAME='apache-tomcat-7.0.37.tar.gz' TOMCAT_DIR='/usr/local/tomcat'STICKY_PKG_DIR='/root/one_key_install' STICKY_PKG_NAME='nginx-sticky-module-1.0.tar.gz'JDK_BIN_DIR='/root/one_key_install' JDK_BIN_NAME='jdk-6u32-linux-x64.bin'################### nginx + sticky installcd$NGINX_PKG_DIR tar zxf $NGINX_PKG_NAME DIR=`ls -F | grep /$ | grep nginx | awk -F '/''{print $1}'` tar zxf $STICKY_PKG_DIR/$STICKY_PKG_NAME -C $NGINX_PKG_DIR/$DIRsed -i 's/^CFLAGS="$CFLAGS -g"/#CFLAGS="$CFLAGS -g"/g' /$NGINX_PKG_DIR/$DIR/auto/cc/gcc sed -i 's/^#define NGINX_VER \"nginx\/\" NGINX_VERSION/#define NGINX_VER \"nginx\/\"/g' /$NGINX_PKG_DIR/$DIR/src/core/nginx.h yum install gcc pcre-devel openssl-devel -y cd$NGINX_PKG_DIR/$DIR ./configure --prefix=/usr/local/nginx --add-module=$NGINX_PKG_DIR/$DIR/nginx-sticky-module-1.0 --with-http_ssl_module --with-http_stub_status_module make && make install ln -s /usr/local/nginx/sbin/nginx /usr/local/sbin/ useradd -M -d /usr/local/nginx/ -s /sbin/nologin nginx cd$DIR_NOW rm -f /usr/local/nginx/conf/nginx.conf cp nginx.conf.exp /usr/local/nginx/conf/nginx.conf sed -i "s/server 192.168.0.1:8080;/server ${IPADDR_NTM}:8080;/g" /usr/local/nginx/conf/nginx.conf sed -i "s/server 192.168.0.2:8080;/server ${IPADDR_TM}:8080;/g" /usr/local/nginx/conf/nginx.conf ##################### tomcat installcd$JDK_BIN_DIR sh $JDK_BIN_NAME DIR=`ls -F | grep /$ | grep jdk | awk -F '/''{print $1}'` mv $DIR /usr/local/ mv /usr/local/$DIR /usr/local/jdk echo"export JAVA_HOME=/usr/local/jdk export CLASSPATH=:\$JAVA_HOME/lib export PATH=\$PATH:\$JAVA_HOME/bin ">>/etc/profile cd$TOMCAT_PKG_DIR tar zxf $TOMCAT_PKG_NAME -C /usr/local cd /usr/local mv /usr/local/`ls | grep tomcat` /usr/local/tomcat ln -s /usr/local/tomcat/bin/startup.sh /usr/local/sbin/tomcat-start ln -s /usr/local/tomcat/bin/shutdown.sh /usr/local/sbin/tomcat-stop cd$DIR_NOW tar zxf kryo_pkgs.tar.gz cd kryo_pkgs cp * /usr/local/tomcat/lib cd$DIR_NOW rm -f /usr/local/tomcat/conf/context.xml cp context.xml.exp /usr/local/tomcat/conf/context.xml cd$DIR_NOW rm -f /usr/local/tomcat/conf/context.xml cp context.xml.exp /usr/local/tomcat/conf/context.xml sed -i "s#memcachedNodes=\"n1:192.168.0.1:11211,n2:192.168.0.2:11211\"#memcachedNodes=\"n1:${IPADDR_NTM}:11211,n2:${IPADDR_TM}:11211\"#" /usr/local/tomcat/conf/context.xml #################### memcached installyum install memcached -y #################### start services /etc/init.d/memcached start source /etc/profile tomcat-start nginx
第二个,在只有tomcat和memcached的机器上运行
#!/bin/bash setenforce 0 > /dev/null iptables -F > /dev/null sed -i 's/^SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config DIR_NOW=`pwd` IPADDR_NTM='192.168.122.101' IPADDR_TM='192.168.122.102'TOMCAT_PKG_DIR='/root/no_ngx' TOMCAT_PKG_NAME='apache-tomcat-7.0.37.tar.gz' TOMCAT_DIR='/usr/local/tomcat'STICKY_PKG_DIR='/root/no_ngx' STICKY_PKG_NAME='nginx-sticky-module-1.0.tar.gz'JDK_BIN_DIR='/root/no_ngx' JDK_BIN_NAME='jdk-6u32-linux-x64.bin'cd$JDK_BIN_DIR sh $JDK_BIN_NAMEDIR=`ls -F | grep /$ | grep jdk | awk -F '/''{print $1}'` mv $DIR /usr/local/ mv /usr/local/$DIR /usr/local/jdk echo"export JAVA_HOME=/usr/local/jdk export CLASSPATH=:\$JAVA_HOME/lib export PATH=\$PATH:\$JAVA_HOME/bin ">>/etc/profile cd$TOMCAT_PKG_DIR tar zxf $TOMCAT_PKG_NAME -C /usr/local/ cd /usr/local mv /usr/local/`ls | grep tomcat` /usr/local/tomcat cd$DIR_NOW tar zxf kryo_pkgs.tar.gz cd kryo_pkgs cp * /usr/local/tomcat/lib cd$DIR_NOW rm -f /usr/local/tomcat/conf/context.xml cp context.xml.exp /usr/local/tomcat/conf/ mv /usr/local/tomcat/conf/context.xml.exp /usr/local/tomcat/conf/context.xml sed -i "s#memcachedNodes=\"n1:192.168.0.1:11211,n2:192.168.0.2:11211\"#memcachedNodes=\"n1:${IPADDR_NTM}:11211,n2:${IPADDR_TM}:11211\"#" /usr/local/tomcat/conf/context.xml sed -i 's/failoverNodes="n1"/failoverNodes="n2"/' /usr/local/tomcat/conf/context.xml yum install memcached -y /etc/init.d/memcached start source /etc/profile ln -s /usr/local/tomcat/bin/startup.sh /usr/local/sbin/tomcat-start ln -s /usr/local/tomcat/bin/shutdown.sh /usr/local/sbin/tomcat-stop tomcat-start
这两个脚本存在同样的问题,执行完毕之后发现
source /etc/profile
这一句没有执行,必须手动执行,一直百思不得其解,希望有谁能解答,多谢啦~~
以上就介绍了nginx+tomcat+memcached配置+脚本,包括了方面的内容,希望对PHP教程有兴趣的朋友有所帮助。