首页 > 系统运维 > Drbd 8.4+keepavled+Moosefs分布式存儲双机搭建
您的足迹
  • 你没有浏览过任何文章或者你没有开启cookies。

Drbd 8.4+keepavled+Moosefs分布式存儲双机搭建

moosefs前一篇已经讲过,这篇主要讲drbd+keepalived搭配使用。

软件版本:

drbd 8.4.2

keepalived-1.2.7.tar.gz

测试环境:

主:192.168.1.73

备:192.168.1.74

VIP:192.168.1.198

测试方便,直接添加了一块硬盘并分区sdb1

一、drbd安装

./configure --prefix=/usr --localstatedir=/var --sysconfdir=/etc --with-km
make && make install

主节点配置如下:

# cat /etc/drbd.d/global_common.conf
global {
        usage-count no;
}

common {
        handlers {
                pri-on-incon-degr "/usr/lib/drbd/notify-pri-on-incon-degr.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";
                pri-lost-after-sb "/usr/lib/drbd/notify-pri-lost-after-sb.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";
                local-io-error "/usr/lib/drbd/notify-io-error.sh; /usr/lib/drbd/notify-emergency-shutdown.sh; echo o > /proc/sysrq-trigger ; halt -f";
        }

        startup {
                 wfc-timeout 30;
                 degr-wfc-timeout 30;
                 outdated-wfc-timeout 30; 
        }

        options {
                # cpu-mask on-no-data-accessible
        }

        disk {
                on-io-error detach;
                fencing resource-and-stonith;
                resync-rate 100M;
        }

        net {
                protocol C;
                cram-hmac-alg sha1;
                shared-secret "linuxfun.me";
        }
}

cat /etc/drbd.d/mfs.res  
resource mfs{
        device /dev/drbd0;
        disk /dev/sdb1;
        meta-disk internal;
        on master{
            address 192.168.1.73:9876;
        }
        on slave{
            address 192.168.1.74:9876;
        }
}

复制配置文件到备用节点:

scp /etc/drbd.d root@192.168.1.74:/etc/drbd.d/

设置主节点的角色为primary:

drbdadm primary mfs

在主节点上为drbd分区创建文件系统,并挂载drbd分区:

yum -y install xfsprogs xfsdump 
mkfs.xfs /dev/drbd0 
mkdir /data 
mount /dev/drbd0 /data

注意:1) 是为drbd分区创建文件系统,就是在mfs.res中定义的/dev/drbd0,不是底层分区
2) 为drbd分区创建文件系统的工作,只在主节点上进行操作,备用节点上不用创建。
测试主备节点之间的数据同步
1) 在主节点上:
创建测试文件:echo -e “$(date +$F%T): hell0″ >/data/test.txt
卸载drbd设备:umount /data
主节点角色降级:drbdadm secondary mfs
2) 然后,在备用节点上:
把备用节点设置为primary角色:drbdadm primary mfs
挂载drbd设备:mount /dev/drbd0 /data

二、keepalived安装及整合

主备节点安装一样,如下:

./configure --prefix=/usr/local/keepalived
make && make install

主节点配置如下:


! Configuration File for keepalived

global_defs {
   router_id LVS_DEVEL
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.1.198
    }
notify_master "/usr/local/sbin/to-master"
notify_backup "/usr/local/sbin/to-backup"
notify_fault "/usr/local/sbin/to-backup"
}

备节点配置如下:

! Configuration File for keepalived

global_defs {
   router_id LVS_DEVEL
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 99
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.1.198
    }
notify_master "/usr/local/sbin/to-master"
notify_backup "/usr/local/sbin/to-backup"
notify_fault "/usr/local/sbin/to-backup"
}

主备节点分别启动:
/usr/local/keepalived/sbin/keepalived -f /usr/local/keepalived/etc/keepalived/keepalived.conf

to-master的脚本如下:

 
#!/bin/bash
while :
do
        drbd_status=`/etc/init.d/drbd status |grep -i connected |awk '{print $2}'`
        if [ -n "$drbd_status" ]; then
                echo "drbd's status is inconsisten,waiting..."
                sleep 1
        else
                echo "drbd's status is ok"
                break
        fi
done

while :
do
if [ -d /usr/local/mfsm/ ]; then
        #service mysqld stop
        /usr/local/mfsm/sbin/mfsmaster start
        break
else
        drbdadm primary mfs
        mount /dev/drbd0 /usr/local/mfsm/
        sleep 1
        /usr/local/mfsm/sbin/mfsmaster start
fi
done


drbd_status=`/etc/init.d/drbd status |grep -i connected |awk '{print $2}'`
if [ -n "$drbd_status" ]; then
vip_status=`ip addr |grep eth0|grep "192.168.1.198"`
   if [[ ! -z ${vip_status} ]];then
     for i in $(seq 3)
       do 
        drbdadm connect all
        sleep 2
     done
   fi
fi

to-backup脚本如下:

#!/bin/bash
if [ -d /usr/local/mfsm/ ];then
        /usr/local/mfsm/sbin/mfsmaster stop
        #pkill -9 mysql
        sleep 1
        fuser -ck /usr/local/mfsm/
        umount -l /dev/drbd0
        drbdadm secondary mfs
else
        drbd_status=`/etc/init.d/drbd status |grep -i connected |awk '{print $2}'`
        if [ -n "$drbd_status" ]; then
        drbdadm secondary mfs
fi
fi

参考:http://share.blog.51cto.com/278008/845357

http://wdj01.blog.51cto.com/1059856/621097

  1. 还没有评论
评论提交中, 请稍候...

留言

可以使用的标签: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>
Trackbacks & Pingbacks ( 0 )
  1. 还没有 trackbacks
Feed