Piranha | High Availability Server in Red Hat
Clustering is by no means having any similarity usually. I read somewhere that if you’ll ask ten persons the definition of clustering, you’re likely to get at least nine unique answers – which is what I’ve found true! There are many different kinds of clustering and a lot of ways to do it specially when it comes to Linux. As for as what I’ve learnt in Windows, there are two main categories of purposes where clustering is usually used for. i.e. Static content and non-static content. Network load balancing (NLB) is what used to fulfill the first one and clustering within any shared media for the later. Red Hat does it using the same LVS kernel modules, the one used in Debian but with different set of tools bundled in Piranha rpm (if you’ll Google it out you’ll only find how its done in Debian and that’s the reason why I am writing this tutorial) where its done with heartbeat and ldirectord both of which are available on Red Hat distributions as well but I’ve not been able to fully deploy the same solution on CentOS with LVS, heartbeat and ldirector no more further to a stable point with a mechanism of direct routing of LVS and without no errors. So I went ahead and tried configuring Piranha with an LVS-DR approach where load balanced requests are sent back to end users directly from real (web) servers.
The beauty of Piranha is that it comes with a web based administration panel to configure load balancer configuration which I prefer to configure my first load balancer and then simply copy over the configuration file under /etc/sysconfig/ha/lvs.cf to any further backup load balancer, though there does exist a few documentation files under /usr/share/doc/piranha-*/docs/ if you would like to check over built in sample configs. In this example we would be having these IPs for the machines.
Load Balancer: 12.12.12.40
Real or Web Servers:
VM1: 12.12.12.41
VM2: 12.12.12.41
Virtual IP (VIP): 12.12.12.50 (this is the IP where a FQDN should point to)
Lets move a head and follow the steps below.
1. Downloading and installing the packages
yum install piranha ipvsadm -y
2. Configuration
touch /etc/sysconfig/ipvsadm sed -i 's/net.ipv4.ip_forward = 0/net.ipv4.ip_forward = 1/' /etc/sysctl.conf sysctl -p piranha-passwd service piranha-gui start
You’ll then browse to http://12.12.12.40:3636/ or http://localhost:3636/ and will be presented a login screen.
Log on in and adjust the settings as below.
A. Configure the main load balancer settings
B. Create a Virtual Server
C. Create two Real Servers within it
D. Leave monitoring segment to its default
About VIP’s settings, of course you can customize; like that of scheduling alogrithims etc.
3. Configuration on Real Servers
Since we configured the webservers to cluster with direct routing, there’s need to configure real servers to prevent reverse ARP havoc. On both of real servers, run:
#Make sure that apache is running on both real servers service httpd restart && chkconfig httpd on #Install arptables yum install arptables_jf -y chkconfig arptables_jf on arptables -I IN -d 12.1.2.12.50 -j REJECT arptables -A OUT -d 12.12.12.50 -j mangle --mangle-ip-s <real-server-ip> service arptables_jf save && service arptables_jf restart #Add an alias of VIP on both real servers ip addr add 12.12.12.50 eth0 echo "ip addr add 12.12.12.50 eth0" >> /etc/rc.local
This can be done with iptables as well but Red Hat recommends using arptables as iptables could be aggressive in ARP filtering. About network alias some people will tell you to create a secondary loopback alias, either way works so it totally depends upon you!
4. Starting the service on load balancer
After configuring from web panel this is how a fully configured LVS config file looks like:
# cat /etc/sysconfig/ha/lvs.cf
serial_no = 34
primary = 12.12.12.40
service = lvs
backup_active = 0
backup = 0.0.0.0
heartbeat = 1
heartbeat_port = 539
keepalive = 3
deadtime = 25
network = direct
debug_level = NONE
virtual HTTP {
active = 1
address = 12.12.12.50 eth0:1
vip_nmask = 255.255.255.0
port = 80
send = "GET / HTTP/1.1\r\n\r\n"
expect = "HTTP"
use_regex = 0
load_monitor = /etc/sysconfig/ha/ruptime-piranha
scheduler = rr
protocol = tcp
timeout = 4
reentry = 4
quiesce_server = 1
server VM1 {
address = 12.12.12.41
active = 1
weight = 1
}
server VM2 {
address = 12.12.12.42
active = 1
weight = 1
}
}
Make sure to click on ‘ACCEPT’ and ‘ACTIVATE/DEACTIVATE’ to make the VIP and Real Servers’ status “up” in the configuration panel – this would save the configuration and now its time to restart pulse service which would use the lvs.cf config file, configure the VIP and start nanny, lvsadm daemons.
chkconfig pulse on service pulse start
After that it’ll take a few seconds to a minute, start up pulse and logs activity which can be seen from /var/log/messages.
5. Time to test out
Add some different content in both web servers’ virtual host document root. Go to http://12.12.12.50 on any other machine in your network and try reloading the page a couple of times. And you’ll see it’ll be loaded from both of web servers depending upon scheduling algorithm. Watch the output of ipvsadm on load balancer, stop apache on one of web servers and all of requests would get start serving from the other web server with a noticable change on weight in ipvsadm output falling back to zero for a dead apache server.
# ipvsadm -L -n IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 12.12.12.50:80 rr -> 12.12.12.42:80 Route 1 0 0 -> 12.12.12.41:80 Route 1 0 0
Troubleshooting / Issues:
1. When using weighted scheduling algorithms with nanny’s load monitor, behaviour of load balancer can change than to what is expected.
2. Even load_monitor w/ nanny seems buggy itself as I got a lot of errors like these, after trying a fix found on Piranha’s mailing list which was to use to a customized ruptime in a bash script with rhowd turned on, on all of cluster nodes. The only workaround as for now is to disable load_monitor.
"Nov 29 14:57:03 Methane nanny[7719]: The following exited abnormally: Nov 29 14:57:03 Methane nanny[7719]: failed to read remote load"
3. And if you get an error like this one; that means you’re having SELinux on. Use ’sestatus’ to check and /etc/selinux/config to disable it or you better should how to change security context if you don’t want to!
# service piranha-gui start Starting piranha-gui: (13)Permission denied: make_sock: could not bind to address [::]:3636 no listening sockets available, shutting down Unable to open logs [FAILED]
Recent Comments