DRBD with PCS: Make production at HIGH availability!

 

DRBD Configuration with PCS (Pacemaker/Corosync)
----------------------------------------------------------

High Availability DRBD on RHEL 8. DRBD®– software is a distributed… | by Ach.Chusnul Chikam | Medium

1. Introduction

DRBD (Distributed Replicated Block Device) is a block-level replication tool that allows real-time data replication between two servers for high availability. When combined with PCS (Pacemaker/Corosync), it provides automatic failover in case of primary node failure.

2. Understanding DRBD with PCS

  • DRBD replicates data between two servers at the block level.

  • Both servers have an HDD connected via a dedicated cross cable to ensure fast and direct replication.

  • Only one server is Primary at a time, while the other remains Secondary.

  • The Primary node handles read/write operations, while the Secondary node keeps data in sync.

  • In case of failover, the Secondary node becomes Primary, ensuring high availability.

  • The IP address of the cross cable connection is mentioned in /etc/drbd.d/drbd0.res, allowing ping or telnet to verify connectivity.

3. Key Verification Commands

 

Command

Purpose

#pcs status :

Check if the Primary and Secondary hosts are online

#systemctl status drbd :

Check the status of the DRBD service

#drbd-overview :

Verify if data synchronization is up-to-date

#cat /etc/drbd.d/drbd0.res :

Check DRBD resource configuration details

#telnet <secondary_IP>  <DRBD_Port> :

 Verify if the cross cable is connected

#ping <secondary_IP> :

Check network connectivity between nodes

4. DRBD Configuration Steps

Step 1: Install Required Packages

#yum install -y drbd kmod-drbd pacemaker pcs
#systemctl enable pcsd
#systemctl start pcsd

Step 2: Authenticate PCS Cluster Nodes

#echo "password" | passwd --stdin hacluster
#pcs cluster auth <node1> <node2> -u hacluster -p password

Step 3: Configure DRBD Resource

Create DRBD Configuration File

#vi /etc/drbd.d/drbd0.res

Example:

resource drbd0 {
  device    /dev/drbd0;
  disk      /dev/sdb;
  meta-disk internal;
  on node1 {
    address 192.168.1.1:7789;
  }
  on node2 {
    address 192.168.1.2:7789;
  }
}

Initialize DRBD

#drbdadm create-md drbd0
#systemctl enable drbd
#systemctl start drbd
#drbdadm up drbd0

Step 4: Set Primary Node and Format DRBD Disk

#drbdadm primary --force drbd0
#mkfs.ext4 /dev/drbd0

Step 5: Configure DRBD in PCS Cluster

#pcs cluster setup drbd-cluster <node1> <node2>
#pcs cluster start --all
#pcs property set stonith-enabled=false
#pcs property set no-quorum-policy=ignore

Create DRBD Resource in PCS

#pcs resource create drbd0 ocf:linbit:drbd drbd_resource=drbd0 op monitor interval=30s
#pcs resource master drbd0-master drbd0 master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 notify=true

Create Filesystem Resource

#pcs resource create drbd_fs Filesystem device=/dev/drbd0 directory=/mnt/data fstype=ext4
#pcs constraint colocation add drbd_fs with master drbd0-master INFINITY
#pcs constraint order promote drbd0-master then start drbd_fs

5. Troubleshooting & Best Practices

Common Issues & Fixes:

 

Issue

Cause

Fix

DRBD stuck in Secondary mode

Primary node not set

Run #drbdadm primary --force drbd0

Split-brain situation

 Network failure

Run #drbdadm disconnect drbd0; drbdadm secondary drbd0; drbdadm connect drbd0

Service not starting

 Config error

Check logs: #journalctl -xe -u drbd

DRBD sync issue

Network connectivity

Verify using #ping and #telnet commands

Failover not happening

PCS misconfiguration

Run #pcs resource cleanup drbd0 and #pcs status

Testing Failover Manually

#pcs resource move drbd_fs <other_node>

Verifying DRBD Synchronization

#drbd-overview
#drbdadm status
 
Related Config Files

/etc/drbd.conf – Config file
/etc/drbd.d/global_common.conf - contains the global and common sections of the DRBD configuration.
/etc/drbd.d/*.res – Each resource configuration file

Port Number : 7788
Service Name : drbd
 

6. Conclusion

This document provides a complete guide to configuring DRBD with PCS for high availability. Regular monitoring using pcs status, drbd-overview, and network checks (ping, telnet) is crucial to ensure smooth operation. For production setups, enable STONITH (Shoot The Other Node In The Head) to prevent split-brain scenarios.




======================================================================================
Thank you, check out more blogs for such information..!
For any queries,feel free to reach out me on shubhammore07007@gmail.com

Comments

Post a Comment

Popular posts from this blog

Nginx : Setting Up Nginx with Load Balancing

Image
This guide explains how to configure Nginx as a load balancer for multiple backend applications running on different ports. The setup includes three application servers running on ports 8001, 8002, and 8003, with Nginx balancing requests among them.   Nginx Configuration Details Individual Server Blocks Each backend application is configured as an independent Nginx server block: Server 1 Configuration ( server1.conf ) [root@server1 conf.d]# cat /etc/nginx/conf.d/server1.conf | grep -v "#"   server { listen 8001; server_tokens off; server_name _; client_max_body_size 35M; charset utf-8; large_client_header_buffers 4 16k; root /usr/share/nginx/server1; index index.html index.php index.cgi; access_log /var/log/nginx/access.log; error_log /var/log/nginx/error.log; } Server 2 Configuration ( server2.conf ) [root@server1 conf.d]# cat /etc/nginx/conf.d/server2.conf | grep -v "#"     server { listen 8002; server_tokens off;...
Read more

Nginx : Virtual hosting using Location directives

Image
This setup demonstrates a virtual container-based environment in Nginx where multiple locations are configured under a single server block, rather than a load-balancing mechanism. Each location serves a different application based on its respective directory structure. Nginx Configuration Server Configuration server { listen 80; server_tokens off; server_name _; client_max_body_size 35M; charset utf-8; large_client_header_buffers 4 16k; root /usr/share/nginx/server1; index index.html index.php index.cgi; access_log /var/log/nginx/access.log; error_log /var/log/nginx/error.log; location /server2 { root /usr/share/nginx/; index index.html index.php index.cgi; access_log /var/log/nginx/access.log; error_log /var/log/nginx/error.log; } location /server3 { root /usr/share/nginx/; index index.html index.php index.cgi; access_log /var/log/nginx/access.log; error_log /...
Read more

Terraform project with AWS to build infra within seconds! (Web Ec2 with eIP)

Image
1. Introduction: What is Terraform? Terraform is an open-source Infrastructure as Code (IaC) tool created by HashiCorp. It lets you define, provision, and manage cloud infrastructure using declarative configuration files . Instead of manually creating AWS resources in the console, you write code that describes what you want, and Terraform takes care of creating or updating it. Why use it with AWS? a) Infrastructure as Code (IaC) You can define AWS resources (EC2, VPC, S3, IAM, etc.) in .tf files. These configurations can be version-controlled in Git. b) Multi-Cloud Support Although AWS has its own tool (CloudFormation), Terraform works with AWS + Azure + GCP + on-prem at the same time. c) Reusability & Automation You can reuse Terraform modules to deploy the same AWS setup in different environments (dev, test, prod). d) State Management Terraform keeps a state file to know which resources it manages. Makes it easy to track and update changes in AWS without accidentally del...
Read more