keepalive HA + docker + nginx 实战

虚拟机桥接模式，独立ip和网卡

IP	角色
192.168.0.107	master
192.168.0.111	slave

配置虚拟机网卡静态ip

TYPE="Ethernet"                                                                                             
PROXY_METHOD="none"                                                                                         
BROWSER_ONLY="no"                                                                                           
BOOTPROTO="static"                                                                                          
DEFROUTE="yes"                                                                                              
IPV4_FAILURE_FATAL="no"                                                                                     
IPV6INIT="yes"                                                                                              
IPV6_AUTOCONF="yes"                                                                                         
IPV6_DEFROUTE="yes"                                                                                         
IPV6_FAILURE_FATAL="no"                                                                                     
IPV6_ADDR_GEN_MODE="stable-privacy"                                                                         
NAME="ens33"                                                                                                
UUID="a4fa6313-d874-4d7e-ade4-463878ca48b6"                                                                 
DEVICE="ens33"                                                                                              
ONBOOT="yes"                                                                                                
IPADDR=192.168.0.107       
#以下配置参考宿主机配置                                                                                 
GATEWAY=192.168.0.1                                                                                         
DNS1=192.168.1.1                                                                                            
DNS2=192.168.0.1 

配置保存之后重启网卡

systemctl restart network

查看配置是否生效

ip addr 

生效的网卡为ens33

ens33:  mtu 1500 qdisc pfifo_fast state UP group default qlen 1000      link/ether 00:0c:29:8f:47:6f brd ff:ff:ff:ff:ff:ff                                                      inet 192.168.0.107/24 brd 192.168.0.255 scope global noprefixroute ens33                                valid_lft forever preferred_lft forever                                                              inet 192.168.0.200/32 scope global ens33                                                                valid_lft forever preferred_lft forever                                                              inet6 fe80::dc9e:7948:4e5e:3056/64 scope link noprefixroute                                             valid_lft forever preferred_lft forever 

环境直接安装docker

第一步：设置 yum 仓库地址

sudo yum-config-manager \--add-repo \https://download.docker.com/linux/centos/docker-ce.repo
sudo yum-config-manager \--add-repo \http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

第二步：更新 yum 缓存

sudo yum makecache fast #yum 是包管理器

第三步：安装新版 docker

sudo yum install -y docker-ce docker-ce-cli containerd.io

#启动docker服务
systemctl restart docker
#将docker服务添加到linux 开机启动
systemtctl enable docker

Docker镜像加速

cat <<EOF > /etc/docker/daemon.json
{"registry-mirrors": ["https://docker.mirrors.ustc.edu.cn","http://hub-mirror.c.163.com"],"max-concurrent-downloads": 10,"log-driver": "json-file","log-level": "warn","log-opts": {"max-size": "10m","max-file": "3"},"data-root": "/var/lib/docker"
}
EOF

# -v 是指卷挂载，将宿主机路径挂载到‘:’后面的容器路径上
docker run -d -p 80:80 --name nginx-web \
-v /usr/local/nginx/www:/usr/share/nginx/html \
-v /usr/local/nginx/conf/nginx.conf:/etc/nginx/nginx.conf \
-v /usr/local/nginx/logs:/var/log/nginx nginx

安装部署keepalived

 yum install -y keepalivedsystemctl enable keepalivedsystemctl start keepalived

修改keepalived配置

! Configuration File for keepalivedglobal_defs {notification_email {acassen@firewall.locfailover@firewall.locsysadmin@firewall.loc}notification_email_from Alexandre.Cassen@firewall.locsmtp_server 192.168.200.1smtp_connect_timeout 30router_id LVS_DEVELvrrp_skip_check_adv_addrvrrp_strictvrrp_garp_interval 0vrrp_gna_interval 0
}vrrp_instance VI_1 {# master 结点配置MASTER slave 结点配置BACKUPstate MASTER# 绑定虚机生效的网卡interface ens33virtual_router_id 51priority 100advert_int 1authentication {auth_type PASSauth_pass 1111}# 配置HA集群虚拟IPvirtual_ipaddress {192.168.0.200}#配置生胶的检测脚本track_script {  chk_http_port}
}
vrrp_script chk_http_port {#心跳执行的脚本，检测nginx是否启动script "/usr/local/src/check_nginx_pid.sh" #（检测脚本执行的间隔）interval 2 #权重                        weight 2                           
}
# 上述配置的虚拟ip
virtual_server 192.168.0.200 80 {delay_loop 6lb_algo rrlb_kind NAT#   persistence_timeout 50protocol TCP# 集群中虚机配置的静态ipreal_server 192.168.0.107 80 {weight 1TCP_CHECK {connect_timeout 3delay_before_retry 3}}real_server 192.168.0.111 80 {weight 1TCP_CHECK {connect_timeout 3delay_before_retry 3}}

虚机健康度检测脚本

#!/bin/bash
#检测nginx是否启动了
if [[ ! -n $(docker ps -q -f name=^nginx-web) ]];thendocker start nginx-web# echo '重启ng'if [[ ! -n $(docker ps -q -f name=^nginx-web) ]];then   #nginx重启失败，则停掉keepalived服务，进行VIP转移systemctl stop keepalived                    fiexit 1
elseexit 0
fi

# 给脚本配置执行权限chmod 755 /usr/local/src/check_nginx_pid.sh

测试工作

删除容器，观察是否被重启

docker fm -rf nginx-web

查看keepalived 状态，确认已经被stop, active： dead

[root@klv-master src]# systemctl status keepalived.service -l                                                                                                                                                                                                               
● keepalived.service - LVS and VRRP High Availability Monitor                                                                                                                                                                                                               Loaded: loaded (/usr/lib/systemd/system/keepalived.service; enabled; vendor preset: disabled)                                                                                                                                                                            Active: inactive (dead) since Thu 2022-12-15 03:36:43 EST; 1s ago                                                                                                                                                                                                        Process: 2090 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)                                                                                                                                                                          Main PID: 2091 (code=exited, status=0/SUCCESS)                                                                                                                                                                                                                             Tasks: 0                                                                                                                                                                                                                                                                Memory: 5.2M                                                                                                                                                                                                                                                             CGroup: /system.slice/keepalived.service                                                                                                                                                                                                                                 Dec 15 03:36:34 klv-master Keepalived_vrrp[2093]: /usr/local/src/check_nginx_pid.sh exited with status 1                                                                                                                                                                    
Dec 15 03:36:36 klv-master Keepalived_vrrp[2093]: /usr/local/src/check_nginx_pid.sh exited with status 1                                                                                                                                                                    
Dec 15 03:36:38 klv-master Keepalived_vrrp[2093]: /usr/local/src/check_nginx_pid.sh exited with status 1                                                                                                                                                                    
Dec 15 03:36:40 klv-master Keepalived_vrrp[2093]: /usr/local/src/check_nginx_pid.sh exited with status 1                                                                                                                                                                    
Dec 15 03:36:42 klv-master systemd[1]: Stopping LVS and VRRP High Availability Monitor...                                                                                                                                                                                   
Dec 15 03:36:42 klv-master Keepalived[2091]: Stopping                                                                                                                                                                                                                       
Dec 15 03:36:42 klv-master Keepalived_vrrp[2093]: VRRP_Instance(VI_1) sent 0 priority                                                                                                                                                                                       
Dec 15 03:36:42 klv-master Keepalived_vrrp[2093]: VRRP_Instance(VI_1) removing protocol VIPs.                                                                                                                                                                               
Dec 15 03:36:42 klv-master Keepalived_vrrp[2093]: VRRP_Instance(VI_1) removing protocol iptable drop rule                                                                                                                                                                   
Dec 15 03:36:43 klv-master systemd[1]: Stopped LVS and VRRP High Availability Monitor.  

访问虚拟ip

192.168.0.200

恢复容器后再次访问