使用lvs实现负载均衡原理以及配置详解(fullNAT模式) - 好文

lvs中nat和fullnat的区别：
nat模式下报文变化发送接收 cip ---> vip cip ---> rip ( DNAT ) rip ---> cip vip ---> cip (
SNAT ) fullnat模式下报文变化发送接收 cip ---> vip lip ---> rip ( SNAT + DNAT ) rip --->
lip vip ---> cip ( SNAT + DNAT ) 注释： cip为客户端的地址 vip为虚拟地址 rip为真实的服务器 lip为本地地址
SNAT为来源地址转换 DNAT为目的地址转换
粗略的说一下自己的理解：（以我自己的实验为例子）
首先保证server1调度器端和真机可以进行数据的传输，当真机数据包过来之后到达的是server1的目的地址也就是VIP，即就是cip --->
vip。server1要到达后端server2和server3，网段也得一致，进行了lip ---> rip，由于fullNAT 模式进行了两次SNAT+
DNAT转换，数据包回来也是如此

fullNAT模式的优势：
LVS 当前应用主要采用 DR 和 NAT 模式,但这 2 种模式要求 RealServer 和 LVS 在同一个 vlan
中,导致部署成本过高;TUNNEL 模式虽然可以跨 vlan,但 RealServer 上需要部署 ipip
模块等,网络拓扑上需要连通外网,较复杂,不易运维。为了解决上述问题,我们在 LVS 上添加了一种新的转发模式:FULLNAT,该模式和 NAT
模式的区别是:PacketIN 时,除了做 DNAT,还做 SNAT(用户 ip->内网 ip),从而实现 LVS-RealServer 间可以跨 vlan
通讯,RealServer 只需要连接到内网;
首先使用ipvsadm –help查看并没有fullNAT模块：
[root@server1 ~]# ipvsadm --help

1.添加fullNAT模块：
一开始更改虚拟机的内存：

在虚拟机查看空闲内存至少为大于10G：

在真机操作打开阿帕奇确保server虚拟机镜像正常：
[kiosk@foundation38 Desktop]$ systemctl start httpd.service 开启阿帕其服务
在虚拟机安装软件搭建服务：
[root@server1 ~]# ls keepalived-2.0.6 ldirectord-3.9.5-3.1.x86_64.rpm
keepalived-2.0.6.tar.gz libnfnetlink-devel-1.0.0-1.el6.x86_64.rpm kernel-2.6.32-
220.23.1.el6.src.rpm Lvs-fullnat-synproxy.tar.gz [root@server1 ~]# yum install
-y rpm-build 解压软件

[root@server1 ~]# ls keepalived-2.0.6 ldirectord-3.9.5-3.1.x86_64.rpm
keepalived-2.0.6.tar.gz libnfnetlink-devel-1.0.0-1.el6.x86_64.rpm kernel-2.6.32-
220.23.1.el6.src.rpm Lvs-fullnat-synproxy.tar.gz [root@server1 ~]# rpm -ivh
kernel-2.6.32-220.23.1.el6.src.rpm 安装源码包

[root@server1 ~]# ls keepalived-2.0.6 libnfnetlink-devel-1.0.0-1.el6.x86_64.rpm
keepalived-2.0.6.tar.gz Lvs-fullnat-synproxy.tar.gz kernel-2.6.32-220.23.1.el6
.src.rpm rpmbuild ldirectord-3.9.5-3.1.x86_64.rpm [root@server1 ~]# cd rpmbuild/
[root@server1 rpmbuild]# cd SPECS/ [root@server1 SPECS]# ls Kernel.spec
[root@server1 SPECS]# rpmbuild -bp kernel.spec 安装有依赖性 error: Failed build
dependencies: redhat-rpm-config is needed by kernel-2.6.32-220.23.1.el6.x86_64
patchutils is needed by kernel-2.6.32-220.23.1.el6.x86_64 xmlto is needed by
kernel-2.6.32-220.23.1.el6.x86_64 asciidoc is needed by kernel-2.6.32-220.23.1
.el6.x86_64 elfutils-libelf-devel is needed by kernel-2.6.32-220.23.1.el6.x
86_64 binutils-devel is needed by kernel-2.6.32-220.23.1.el6.x86_64 newt-devel
is needed by kernel-2.6.32-220.23.1.el6.x86_64 python-devel is needed by kernel-
2.6.32-220.23.1.el6.x86_64 perl(ExtUtils::Embed) is needed by kernel-2.6.32-
220.23.1.el6.x86_64 hmaccalc is needed by kernel-2.6.32-220.23.1.el6.x86_64
[root@server1 SPECS]# yum install redhat-rpm-config patchutils xmlto asciidoc
elfutils-libelf-devel binutils-devel newt-devel python-devel hmaccalc
perl-ExtUtils-Embed -y 依次安装解决依赖性

[root@server1 ~]#yum install -y asciidoc-8.4.5-4.1.el6.noarch.rpm
安装自己下载的包为了解决依赖性 [root@server1 ~]# yum install -y
slang-devel-2.2.1-1.el6.x86_64.rpm [root@server1 ~]# yum install
newt-devel-0.52.11-3.el6.x86_64.rpm

[root@server1 ~]# cd rpmbuild/SPECS/ [root@server1 SPECS]# ls kernel.spec [root
@server1 SPECS]# rpmbuild -bp kernel.spec 会卡着重新打开一个终端连接server1

[root@server1 ~]# yum provides */rngd 寻找所需要的安装包 Loaded plugins: product-id,
subscription-manager This systemis not registered to Red Hat Subscription
Management. You can use subscription-managerto register. rng-tools-2-13.
el6_2.x86_64 : Randomnumber generator related utilities Repo : rhel-source
Matchedfrom: Filename : /etc/sysconfig/rngd Filename : /sbin/rngd Filename :
/etc/rc.d/init.d/rngd [root@server1 ~]# yum install -y rng-tools 安装工具

[root@server1 ~]# rngd -r /dev/urandom 生成随机数工具会不卡 [root@server1 SPECS]# cd

在原来的终端操作: [root@server1 ~]# ls asciidoc-8.4.5-4.1.el6.noarch.rpm
lvs-fullnat-synproxy keepalived-2.0.6 Lvs-fullnat-synproxy.tar.gz keepalived-2.0
.6.tar.gz newt-devel-0.52.11-3.el6.x86_64.rpm kernel-2.6.32-220.23.1.el6.src.rpm
rpmbuild ldirectord-3.9.5-3.1.x86_64.rpm slang-devel-2.2.1-1.el6.x86_64.rpm
libnfnetlink-devel-1.0.0-1.el6.x86_64.rpm [root@server1 ~]# cd rpmbuild/BUILD

[root@server1 BUILD]# ls kernel-2.6.32-220.23.1.el6 [root@server1 BUILD]# cd
kernel-2.6.32-220.23.1.el6/ [root@server1 kernel-2.6.32-220.23.1.el6]# ls linux-
2.6.32-220.23.1.el6.x86_64 vanilla-2.6.32-220.23.1.el6 [root@server1 kernel-2.6
.32-220.23.1.el6]# cd linux-2.6.32-220.23.1.el6.x86_64/

[root@server1 linux-2.6.32-220.23.1.el6.x86_64]# ls [root@server1 linux-2.6.32-
220.23.1.el6.x86_64]# pwd /root/rpmbuild/BUILD/kernel-2.6.32-220.23.1.el6/linux-
2.6.32-220.23.1.el6.x86_64 [root@server1 linux-2.6.32-220.23.1.el6.x86_64]# cp
/root/lvs-fullnat-synproxy/lvs-2.6.32-220.23.1.el6.patch . 将补丁复制到当前

[root@server1 linux-2.6.32-220.23.1.el6.x86_64]# ls [root@server1 linux-2.6.32-
220.23.1.el6.x86_64]# ll lvs-2.6.32-220.23.1.el6.patch -rw-r--r-- 1 root root
475082 Jul 31 10:44 lvs-2.6.32-220.23.1.el6.patch [root@server1 linux-2.6.32-
220.23.1.el6.x86_64]# patch -p1 < lvs-2.6.32-220.23.1.el6.patch 进行打补丁操作

[root@server1 linux-2.6.32-220.23.1.el6.x86_64]# make 进行源码编译

root@server1 linux-2.6.32-220.23.1.el6.x86_64]# cd /boot/ [root@server1 boot]#
ls [root@server1 boot]# cd grub/

[root@server1 grub]# vim grub.conf 更改default ## 更改defult=0

[root@server1 grub]# reboot ##重启 ##再次连接 [root@server1 ~]# uname -r 查看更改的内核版本
2.6.32 [root@server1 ~]# yum remove ipvsadm 卸载ipvsadm [root@server1 ~]# cd
lvs-fullnat-synproxy/ [root@server1 lvs-fullnat-synproxy]# tar zxf
lvs-tools.tar.gz [root@server1 lvs-fullnat-synproxy]# cd tools/ [root@server1
tools]# ls ipvsadm keepalived quagga rpm [root@server1 tools]# cd keepalived/

[root@server1 keepalived]# ls [root@server1 keepalived]# ./configure
--with-kernel-dir="/lib/modules/`uname -r`/build" [root@server1 keepalived]#
yum install -y popt-devel 安装依赖性

[root@server1 keepalived]# ./configure --with-kernel-dir="/lib/modules/`uname
-r`/build" 进行源码编译三步

[root@server1 keepalived]#make

[root@server1 keepalived]#make install

[root@server1 ~]# cd lvs-fullnat-synproxy/ [root@server1 lvs-fullnat-synproxy]
# ls [root@server1 lvs-fullnat-synproxy]# cd tools/ [root@server1 tools]# ls
ipvsadm keepalived quagga rpm [root@server1 tools]# cd ipvsadm/ [root@server1
ipvsadm]# ls [root@server1 ipvsadm]# make 进行编译

[root@server1 ipvsadm]# make install [root@server1 ipvsadm]# cd

[root@server1 ~]# ipvsadm -l 进入查看大小已经变成了2的22次方 IP Virtual Server version 1.2.1
(size=4194304) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port
Forward Weight ActiveConn InActConn TCP 172.25.254.100:http rr

2.fullNAT实验测试：

server1依旧作为调度器：
[root@server1 ~]# ip addr 添加一个虚拟IP 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436
qdisc noqueuestate UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever
preferred_lft forever2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
pfifo_faststate UP qlen 1000 link/ether 52:54:00:8a:d4:d6 brd ff:ff:ff:ff:ff:ff
inet172.25.84.4/24 brd 172.25.84.255 scope global eth0 inet6 fe80::5054
:ff:fe8a:d4d6/64 scope link valid_lft forever preferred_lft forever 3: eth1:
<BROADCAST,MULTICAST> mtu1500 qdisc noop state DOWN qlen 1000 link/ether 52:54:
00:1b:c4:71 brd ff:ff:ff:ff:ff:ff inet 172.25.254.100/24 scope global eth1

server2和server3作为后端服务器，打开阿帕其，网关指向server1的虚拟IP即可：

在server1写入策略：(fullNAT模式无法在虚拟机测试)
[root@server1 ~]# ipvsadm -C [root@server1 ~]# ipvsadm -A -t 172.25.254.100:80
-s wrr [root@server1 ~]# ipvsadm -a -t 172.25.254.100:80 -r 172.25.84.2:80 -b
-b表示fullNAT模式 [root@server1 ~]# ipvsadm -a -t 172.25.254.100:80 -r
172.25.84.3:80 -b [root@server1 ~]# ipvsadm -P -t 172.25.254.100:80 -z
127.0.0.1:80 [root@server1 ~]# ipvsadm -G -t 172.25.254.100:80 VIP:VPORT TOTAL
SNAT_IP CONFLICTS CONNS 172.25.254.100:80 1 127.0.0.1 0 0 [root@server1 ~]#
ipvsadm -ln 查看策略 IP Virtual Server version 1.2.1 (size=4194304) Prot
LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight
ActiveConn InActConn TCP 172.25.254.100:80 wrr -> 172.25.84.2:80 FullNat 1 0 0
->172.25.84.3:80 FullNat 1 0 0 [root@server1 ~]# ipvsadm -lnc 查看策略信息 IPVS
connection entries pro expire state source virtual destination [root@server1 ~]#

热门工具换一换