Nginx+keepalived负载均衡篇

对于中、小型企业，如果没有资金去购买昂贵的四/七层负载均衡交换机，那么Nginx是不错的七层负载均衡选择，并且可以通过Nginx + Keepalived实现Nginx 负载均衡器双机互备，任意一台机器发生故障，对方都能够将虚拟IP接管过去。下面将其安装步骤详细说明下：

一、环境：
centos5.3、nginx-0.7.51、keepalived-1.1.19
主nginx负载均衡器：192.168.0.154
辅nginx负载均衡器：192.168.9.155
vip：192.168.0.188

二、安装keepalived
#tar zxvf keepalived-1.1.19.tar.gz
#cd keepalived-1.1.19
#./configure --prefix=/usr/local/keepalived
#make
#make install
#cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
#cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
#cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
#mkdir /etc/keepalived
#cd /etc/keepalived/
vim keepalived.conf
! Configuration File for keepalived
global_defs {
   notification_email {
   yuhongchun027@163.com
        }
   notification_email_from keepalived@chtopnet.com
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}
vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    mcast_src_ip 192.168.0.155    <==辅nginx的IP地址
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass chtopnet
    }
    virtual_ipaddress {
        192.168.0.188                      <==vip地址
    }
}
#service keepalived start
我们来看一下日志：
[root@ltos ~]# tail /var/log/messages
Oct 6 03:25:03 ltos avahi-daemon[2306]: Registering new address record for 192.168.0.188 on eth0.
Oct 6 03:25:03 ltos avahi-daemon[2306]: Registering new address record for 192.168.0.154 on eth0.
Oct 6 03:25:03 ltos avahi-daemon[2306]: Registering HINFO record with values 'I686'/'LINUX'.
Oct 6 03:25:23 ltos avahi-daemon[2306]: Withdrawing address record for fe80::20c:29ff:feb9:eeab on eth0.
Oct 6 03:25:23 ltos avahi-daemon[2306]: Withdrawing address record for 192.168.0.154 on eth0.
Oct 6 03:25:23 ltos avahi-daemon[2306]: Host name conflict, retrying with <ltos-31>
Oct 6 03:25:23 ltos avahi-daemon[2306]: Registering new address record for fe80::20c:29ff:feb9:eeab on eth0.
Oct 6 03:25:23 ltos avahi-daemon[2306]: Registering new address record for 192.168.0.188 on eth0.
Oct 6 03:25:23 ltos avahi-daemon[2306]: Registering new address record for 192.168.0.154 on eth0.
Oct 6 03:25:23 ltos avahi-daemon[2306]: Registering HINFO record with values 'I686'/'LINUX'.

很显然vrrp已经启动，我们还可以通过命令来检查
[root@ltos html]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:0c:29:ba:9b:e7 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.154/24 brd 192.168.0.255 scope global eth0
    inet 192.168.0.188/32 scope global eth0
    inet6 fe80::20c:29ff:feba:9be7/64 scope link
       valid_lft forever preferred_lft forever
3: sit0: <NOARP> mtu 1480 qdisc noop
    link/sit 0.0.0.0 brd 0.0.0.0

说明vip已经启动，这样主服务器就配置好了，辅机的配置大致一样，除了配置文件有少部分的变化，下
面贴出辅机的配置文件：
! Configuration File for keepalived
global_defs {
   notification_email {
   yuhongchun027@163.com
        }
   notification_email_from keepalived@chtopnet.com
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}
vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    mcast_src_ip 192.168.0.154              <==主nginx的IP的地址
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass chtopnet
    }
    virtual_ipaddress {
        192.168.0.188
    }
}

检查其配置
[root@ltos html]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:0c:29:ba:9b:e7 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.155/24 brd 192.168.0.255 scope global eth0
    inet 192.168.0.188/32 scope global eth0
    inet6 fe80::20c:29ff:feba:9be7/64 scope link
       valid_lft forever preferred_lft forever
3: sit0: <NOARP> mtu 1480 qdisc noop
    link/sit 0.0.0.0 brd 0.0.0.0

测试其效果方法很简单，分别在主辅机上/var/www/html/下建立不同的主页及内容文件index.html，内容分别为192.168.0.154,192.168.0.155，然后用客户机上elinks http://192.168.0.188,主机down掉后辅机会马上接替提供服务，间隔时间几乎无法感觉出来。10月14日又测试了一次后发现，当master主机恢复故障后又能自动恢复成主nginx机，证明整个过程都是正确的。下面是用tcpdump -n -v host 192.168.0.155抓包结果，对keepalived有兴趣的同学请参阅一下《keepalived权威指南》
[root@ltos ~]# tcpdump -n -v host 192.168.0.155
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
13:24:02.826948 IP (tos 0x0, ttl 255, id 281, offset 0, flags [none], proto: VRRP (112), length: 40) 192.168.0.155 > 224.0.0.18: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20, addrs: 192.168.0.188 auth "chtopne^@"
13:24:03.826985 IP (tos 0x0, ttl 255, id 282, offset 0, flags [none], proto: VRRP (112), length: 40) 192.168.0.155 > 224.0.0.18: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20, addrs: 192.168.0.188 auth "chtopne^@"
13:24:04.828898 IP (tos 0x0, ttl 255, id 283, offset 0, flags [none], proto: VRRP (112), length: 40) 192.168.0.155 > 224.0.0.18: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20, addrs: 192.168.0.188 auth "chtopne^@"

这个环境用webbench做压力测试，在10000并发的情况下
[root@autolemp webbench-1.5]# webbench -c 10000 -t 10 http://192.168.0.188/index.html
Webbench - Simple Web Benchmark 1.5
Copyright (c) Radim Kolar 1997-2004, GPL Open Source Software.
Benchmarking: GET http://192.168.0.188/index.html
10000 clients, running 10 sec.
Speed=746202 pages/min, 2785675 bytes/sec.
Requests: 124331 susceed, 36 failed

由于其中之一的nginx主机只有64M内存，所以不敢用高于10000的并发测试；如果同志们想用于生产环境，可考虑更高的并发测试(webbench最多支持30000并发)。如果大家对Nginx+keepalived有疑问，请联系yuhongchun027@163.com(抚琴煮酒)