前面我們聊了聊高可用集群corosync+pacemaker的相關概念以及相關工具的使用和說明，回顧請參考https://www.cnblogs.com/qiuhom-1874/category/1838133.html；今天我們說一下高可用服務keepalived；
keepalived相對corosync+pacemaker這種高可用集群，它要輕量很多；它的工作原理就是vrrp的實現；vrrp（Virtual Router Redundancy Protocol，虛擬路由冗餘協議 ），設計之初它主要用於對LVS集群的高可用，同時它也能夠對LVS後端real server做健康狀態檢測；它主要功能有基於vrrp協議完成地址流動，從而實現服務的故障轉移；為VIP位址所在的節點生成ipvs規則；為ipvs集群的各RS做健康狀態檢測；基於腳本調用接口通過執行腳本完成腳本中定義的功能，進而影響集群事務；
keepalved架構
提示：keepalived的主要由vrrp stack、checkers、ipvs wrapper以及控制組件配置文件分析器，IO復用器，內存管理這些組件組成，其中vrrp stack 是用來實現vip的高可用；checkers用於基於不同協議對後端服務做檢測，它兩都是基於系統調用和SMTP協議來完成對vip的轉移，以及故障轉移後的郵件通知，以及vip和後端服務的檢測；ipvs wrapper主要用於生成ipvs規則；而對於keepalved的核心組件vrrp stack 和checkers是由watchdog進程一直監控著，一旦vrrp stack 或者checkers宕掉，watchdog會立即啟動一個新的vrrp stack或checkers，從而保證了keepalived自身的組件的高可用；
keepalived實現
環境說明
準備兩臺keepalived伺服器，各server必須滿足時間同步，確保iptables及selinux都是關閉著；如果有必要可以配置各節點通過hosts文件解析以及各節點的ssh互信，後面的主機名解析和ssh互信不是必須的；
提示：有關ssh互信，可以參考本人博客https://www.cnblogs.com/qiuhom-1874/p/11783371.html；除了確保以上幾條外，還需要確保我們的網卡支持多播功能；
提示：如果網卡沒有啟動多播功能需要用ip link set multicast on dev 網卡名稱即可；
安裝keepalived程序包
yum install keepalived -y
提示：兩節點都要安裝；
查看keepalived的程序環境
[root@node01 ~]# rpm -ql keepalived/etc/keepalived/etc/keepalived/keepalived.conf/etc/sysconfig/keepalived/usr/bin/genhash/usr/lib/systemd/system/keepalived.service/usr/libexec/keepalived/usr/sbin/keepalived/usr/share/doc/keepalived-1.3.5/usr/share/doc/keepalived-1.3.5/AUTHOR/usr/share/doc/keepalived-1.3.5/CONTRIBUTORS/usr/share/doc/keepalived-1.3.5/COPYING/usr/share/doc/keepalived-1.3.5/ChangeLog/usr/share/doc/keepalived-1.3.5/NOTE_vrrp_vmac.txt/usr/share/doc/keepalived-1.3.5/README/usr/share/doc/keepalived-1.3.5/TODO/usr/share/doc/keepalived-1.3.5/keepalived.conf.SYNOPSIS/usr/share/doc/keepalived-1.3.5/samples/usr/share/doc/keepalived-1.3.5/samples/keepalived.conf.HTTP_GET.port/usr/share/doc/keepalived-1.3.5/samples/keepalived.conf.IPv6/usr/share/doc/keepalived-1.3.5/samples/keepalived.conf.SMTP_CHECK/usr/share/doc/keepalived-1.3.5/samples/keepalived.conf.SSL_GET/usr/share/doc/keepalived-1.3.5/samples/keepalived.conf.fwmark/usr/share/doc/keepalived-1.3.5/samples/keepalived.conf.inhibit/usr/share/doc/keepalived-1.3.5/samples/keepalived.conf.misc_check/usr/share/doc/keepalived-1.3.5/samples/keepalived.conf.misc_check_arg/usr/share/doc/keepalived-1.3.5/samples/keepalived.conf.quorum/usr/share/doc/keepalived-1.3.5/samples/keepalived.conf.sample/usr/share/doc/keepalived-1.3.5/samples/keepalived.conf.status_code/usr/share/doc/keepalived-1.3.5/samples/keepalived.conf.track_interface/usr/share/doc/keepalived-1.3.5/samples/keepalived.conf.virtual_server_group/usr/share/doc/keepalived-1.3.5/samples/keepalived.conf.virtualhost/usr/share/doc/keepalived-1.3.5/samples/keepalived.conf.vrrp/usr/share/doc/keepalived-1.3.5/samples/keepalived.conf.vrrp.localcheck/usr/share/doc/keepalived-1.3.5/samples/keepalived.conf.vrrp.lvs_syncd/usr/share/doc/keepalived-1.3.5/samples/keepalived.conf.vrrp.routes/usr/share/doc/keepalived-1.3.5/samples/keepalived.conf.vrrp.rules/usr/share/doc/keepalived-1.3.5/samples/keepalived.conf.vrrp.scripts/usr/share/doc/keepalived-1.3.5/samples/keepalived.conf.vrrp.static_ipaddress/usr/share/doc/keepalived-1.3.5/samples/keepalived.conf.vrrp.sync/usr/share/doc/keepalived-1.3.5/samples/sample.misccheck.smbcheck.sh/usr/share/man/man1/genhash.1.gz/usr/share/man/man5/keepalived.conf.5.gz/usr/share/man/man8/keepalived.8.gz/usr/share/snmp/mibs/KEEPALIVED-MIB.txt/usr/share/snmp/mibs/VRRP-MIB.txt/usr/share/snmp/mibs/VRRPv3-MIB.txt[root@node01 ~]#
提示：主配置文件是/etc/keepalived/keepalived.conf；主程序文件/usr/sbin/keepalived；unit file是/usr/lib/systemd/system/keepalived.service；unit file的環境配置文件是/etc/sysconfig/keepalived；
keepalived默認配置
[root@node01 ~]# cat /etc/keepalived/keepalived.conf! Configuration File for keepalivedglobal_defs { notification_email { acassen@firewall.loc failover@firewall.loc sysadmin@firewall.loc } notification_email_from Alexandre.Cassen@firewall.loc smtp_server 192.168.200.1 smtp_connect_timeout 30 router_id LVS_DEVEL vrrp_skip_check_adv_addr vrrp_strict vrrp_garp_interval 0 vrrp_gna_interval 0}vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.200.16 192.168.200.17 192.168.200.18 }}virtual_server 192.168.200.100 443 { delay_loop 6 lb_algo rr lb_kind NAT persistence_timeout 50 protocol TCP real_server 192.168.201.100 443 { weight 1 SSL_GET { url { path / digest ff20ad2481f97b1754ef3e12ecd3a9cc } url { path /mrtg/ digest 9b3a0c85a887a256d6939da88aabd8cd } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } }}virtual_server 10.10.10.2 1358 { delay_loop 6 lb_algo rr lb_kind NAT persistence_timeout 50 protocol TCP sorry_server 192.168.200.200 1358 real_server 192.168.200.2 1358 { weight 1 HTTP_GET { url { path /testurl/test.jsp digest 640205b7b0fc66c1ea91c463fac6334d } url { path /testurl2/test.jsp digest 640205b7b0fc66c1ea91c463fac6334d } url { path /testurl3/test.jsp digest 640205b7b0fc66c1ea91c463fac6334d } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } real_server 192.168.200.3 1358 { weight 1 HTTP_GET { url { path /testurl/test.jsp digest 640205b7b0fc66c1ea91c463fac6334c } url { path /testurl2/test.jsp digest 640205b7b0fc66c1ea91c463fac6334c } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } }}virtual_server 10.10.10.3 1358 { delay_loop 3 lb_algo rr lb_kind NAT persistence_timeout 50 protocol TCP real_server 192.168.200.4 1358 { weight 1 HTTP_GET { url { path /testurl/test.jsp digest 640205b7b0fc66c1ea91c463fac6334d } url { path /testurl2/test.jsp digest 640205b7b0fc66c1ea91c463fac6334d } url { path /testurl3/test.jsp digest 640205b7b0fc66c1ea91c463fac6334d } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } } real_server 192.168.200.5 1358 { weight 1 HTTP_GET { url { path /testurl/test.jsp digest 640205b7b0fc66c1ea91c463fac6334d } url { path /testurl2/test.jsp digest 640205b7b0fc66c1ea91c463fac6334d } url { path /testurl3/test.jsp digest 640205b7b0fc66c1ea91c463fac6334d } connect_timeout 3 nb_get_retry 3 delay_before_retry 3 } }}[root@node01 ~]#
提示：keepalived的配置文件主要由global configuration、vrrpdconfiguration、LVS configuration這三部分配置段組成；其中global配置段主要定義全局屬性以及靜態路由和地址相關配置；vrrp配置段主要定義VRRP實例或vrrp同步組相關配置；LVS配置段主要定義IPVS集群和LVS後端各real server相關的配置；
keepalived配置說明
全局配置常用指令說明
global_defs {...}：用於定義全局配置段，在這個配置段裡可以配置全局屬性，以及郵件通知相關配置；
notification_email {...}:該配置段是globald_defs配置段的一個子配置段用於配置當集群發生狀態變化時，接受通知的郵箱；
notification_email_from：用於指定發送郵件的發件人郵箱地址；
smtp_server：用於指定郵件伺服器地址；
smtp_connect_timeout：用於指定郵件伺服器連接超時時間；
router_id：集群節點ID，通常這個ID是唯一的，不和其他節點相同；
vrrp_skip_check_adv_addr：忽略檢查通告vrrp通告和上一次接收的vrrp是同master地址的通告；
vrrp_strict：嚴格遵守VRRP協議；
vrrp_garp_interval：設定同一接口的兩次arp廣播的延遲時長，默認為0表示不延遲；
vrrp_gna_interval：設定同一接口的兩次na消息延遲時長，默認為0表示不延遲；
vrrp_mcast_group4：設定組播ip地址，默認是224.0.0.18；組播地址是一個D類地址，它的範圍是224.0.0.0-239.255.255.255；
vrrp_iptables：關閉生成iptables規則；
vrrp實例常用指令
vrrp_instance：指定一個vrrp示例名稱，並引用一個配置實例上下文配置段用大括號括起來；
state：用於定義該vrrp實例的角色，常用的有MASTER和BACKUP兩個角色，並且多個節點上同虛擬路由id的實例，只能有一個MASTER角色且優先級是最高的，其他的都為BACKUP優先級都要略小於MASTER角色的優先級；
interface：用於指定vrrp實例的網卡名稱，就是把vip配置在那個接口上；
virtual_router_id：虛擬路由ID取值範圍是0-255；
advert_int：指定發送心跳間隔時長，默認是1秒；
priority：指定該實例的優先級；
authentication {...}：用於定義認證信息；
auth_type：指定認證類型，常用認證類型有PASS和AH，PASS指簡單的密碼認證，AH指IPSEC認證；如果使用PASS類型，默認只會取前8個字符作為認證密碼；
auth_pass：指定認證密碼；
virtual_ipaddress {..}：用於設定虛擬ip地址的配置，用大括號括起來；定義虛擬ip的語法格式為：<IPADDR>/<MASK> brd <IPADDR> dev <STRING> scope <SCOPE> label <LABEL>；其中brd用於指定廣播地址，dev用於指定接口名稱，scope用於指定作用域，label用於指定別名；可以配置多個虛擬ip，通常一個實例中只配置一個虛擬ip；
示例：在node01和node02利用keepalived配置vip192.168.0.33
node01上的配置
! Configuration File for keepalivedglobal_defs { notification_email { root@localhost } notification_email_from node01_keepalived@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id node01 vrrp_skip_check_adv_addr vrrp_strict vrrp_garp_interval 0 vrrp_gna_interval 0 vrrp_mcast_group4 224.0.12.132}vrrp_instance VI_1 { state MASTER interface ens33 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 12345678 } virtual_ipaddress { 192.168.0.33/24 brd 192.168.0.255 dev ens33 label ens33:1 }}
View Code
node02上的配置
! Configuration File for keepalivedglobal_defs { notification_email { root@localhost } notification_email_from node02_keepalived@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id node02 vrrp_skip_check_adv_addr vrrp_strict vrrp_garp_interval 0 vrrp_gna_interval 0 vrrp_mcast_group4 224.0.12.132}vrrp_instance VI_1 { state BACKUP interface ens33 virtual_router_id 51 priority 90 advert_int 1 authentication { auth_type PASS auth_pass 12345678 } virtual_ipaddress { 192.168.0.33/24 brd 192.168.0.255 dev ens33 label ens33:1 }}
View Code
啟動node01和node02上的keepalived
提示：可以看到把node01上的keepalived啟動起來以後，vip就配置在外面指定的ens33接口上；
提示：可以看到node02上的keepalived啟動起來以後，vip並沒有從node01上搶過來，並且在node02上看keepalived的狀態信息，清楚的看到node02以backup角色運行著，這意味著只有當master宕機以後，它才會有可能把vip搶過來；
在node02上抓包，看看心跳信息是否是我們指定1秒一個呢？是否是在我們指定的組播域？
提示：可以看到node01(MASTER節點)一秒一個心跳報文給指定的組播域發送通告信息，只要在組播域內地主機能夠收到MASTER的通告，它們都認為MASTER還活著，一旦master沒有發通告，那麼backup節點就會觸發重新爭奪VIP；
驗證：把master keepalived停掉，看看VIP是否飄到node02上呢？
提示：可以看到當把node01上的keepalived停掉以後，對應vip會飄到node02上，並且node02會向組播域一直通告自己的vrrid 優先級 等等信息；
驗證：把node01的keepalived啟動起來，vip是否會被node01搶過去呢？
提示：默認我們沒有指定是否工作在搶佔模式，默認就為搶佔模式，意思是只要對應的組播域有比當前VIP所在節點上的優先級高的通告，擁有VIP的節點會自動把vip讓出來，讓其優先級高的節點應用；
在node02上查看keepalived的狀態以及ip地址信息
提示：從node02的keepalived的狀態信息可以看到，它接收到更高優先級的通告，然後自己自動移除了VIP ，iptables規則，並工作為BACKUP角色；
示例：配置keepalived的雙主模型
node01上的配置
! Configuration File for keepalivedglobal_defs { notification_email { root@localhost } notification_email_from node01_keepalived@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id node01 vrrp_skip_check_adv_addr vrrp_strict vrrp_garp_interval 0 vrrp_gna_interval 0 vrrp_mcast_group4 224.0.12.132}vrrp_instance VI_1 { state MASTER interface ens33 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 12345678 } virtual_ipaddress { 192.168.0.33/24 brd 192.168.0.255 dev ens33 label ens33:1 }}vrrp_instance VI_2 { state BACKUP interface ens33 virtual_router_id 52 priority 90 advert_int 1 authentication { auth_type PASS auth_pass 87654321 } virtual_ipaddress { 192.168.0.34/24 brd 192.168.0.255 dev ens33 label ens33:2 }}
View Code
node02上的配置
! Configuration File for keepalivedglobal_defs { notification_email { root@localhost } notification_email_from node02_keepalived@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id node02 vrrp_skip_check_adv_addr vrrp_strict vrrp_garp_interval 0 vrrp_gna_interval 0 vrrp_mcast_group4 224.0.12.132}vrrp_instance VI_1 { state BACKUP interface ens33 virtual_router_id 51 priority 90 advert_int 1 authentication { auth_type PASS auth_pass 12345678 } virtual_ipaddress { 192.168.0.33/24 brd 192.168.0.255 dev ens33 label ens33:1 }}vrrp_instance VI_2 { state MASTER interface ens33 virtual_router_id 52 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 87654321 } virtual_ipaddress { 192.168.0.34/24 brd 192.168.0.255 dev ens33 label ens33:2 }}
View Code
提示：定義雙主模型，通常我們會利用兩個vrrp實例來配置，中心思想就是利用兩個節點的兩個vrrp實例，把兩個實例分別在node01和node02上各配置一個實例為MASTER，對應剩下節點就為BACKUP；這樣配置以後，重啟keepalived，如果node01和node02都正常在線，那麼對於兩個vip他們會各自佔一個，如果其中一臺server宕機，他們都會把自身為MASTER角色的vip轉移到另外的節點；
驗證：重啟node01和node02上的keepalived，看看對應vip是否都會在兩個節點各自一個呢？
提示：可以看到重啟兩個節點上的keepalived後，根據我們配置的初始化角色各自都佔用了一個vip；這樣我們只需在把對位的域名（如果是web服務）的A記錄解析分別解析到這兩個vip後，這兩個vip就可以各自承擔一部分請求，從而實現兩個keepalived都在工作；
驗證：把node01宕機以後，看看192.168.0.33這個地址是否會飄到node02上呢？
提示：可以看到當node01宕機以後，node02就把原來在node01上的vip搶過來應用在自身節點上；這樣一來就實現了把原來訪問192.168.0.33的流量轉移到node02上了；同樣的道理我們把node02宕機，在node02上的VIP也會轉移到node01上；