DEV Community

Hari Bantwal
Hari Bantwal

Posted on

Oracle Cluster File System(OCFS2) setup for shared Block Volume

#oci #ocfs2 #cluster #blockvolume

There are moments when simple tasks can slow down our progress, often because they're not part of our regular routine. This post captured set of steps to enable Shared Block Volume for multiple OCI Compute Instances. Such a setup also needs Oracle Cluster File System(OCFS2) to be enabled.
I so wish this enablement is possible by a click of Button, API call ...

Lets make our hands dirty.

Starting point.

  1. Two OCI Instance with the provision to access through SSH keys

Image description

  1. Block Volume in the same Availability Domain

Image description

Attach the Block Volume to Instance

  1. Navigate to the Unattached Block Volume. Attached Instances >> Attach Instance

Image description

Attach the First Instance.
Repeat the same for Second Instance

Image description

Setup the Oracle Cluster File System(OCFS2)

1) Add stateful Ingress in Security List of the Instance VCN

Source: Subnet/VCN CIDR
IP Prototocol: TCP and UDP
Source Port: All
Target Port: 3260 and 7777

Image description

2) Open the Target ports on Local OS Firewall on instance-1 and instance-2

 sudo firewall-cmd --zone=public --permanent --add-port=7777/tcp sudo firewall-cmd --zone=public --permanent --add-port=7777/udp sudo firewall-cmd --zone=public --permanent --add-port=3260/tcp sudo firewall-cmd --zone=public --permanent --add-port=3260/udp sudo firewall-cmd --complete-reload
Enter fullscreen mode Exit fullscreen mode

3) Install OCFS2 Packages on instance-1 and instance-2

 sudo dnf install -y ocfs2-tools
Enter fullscreen mode Exit fullscreen mode

4) Create Cluster definition on instance-1

 sudo o2cb add-cluster ocifs2
Enter fullscreen mode Exit fullscreen mode

5) Add the nodes to Cluster on instance-1

 sudo o2cb add-node ocifs2 instance-1 --ip <PRIVATE IP OF instance 1> sudo o2cb add-node ocifs2 instance-2 --ip <PRIVATE IP OF instance 2>
Enter fullscreen mode Exit fullscreen mode

6) Check the Cluster Config file on instance-1

 [opc@instance-1 ~]$ sudo cat /etc/ocfs2/cluster.conf cluster: heartbeat_mode = local node_count = 2 name = ocifs2 node: number = 0 cluster = ocifs2 ip_port = 7777 ip_address = 172.17.0.173 name = instance-1 node: number = 1 cluster = ocifs2 ip_port = 7777 ip_address = 172.17.0.154 name = instance-2
Enter fullscreen mode Exit fullscreen mode

7) Copy the content of Cluster Config file to instance-2 at the /etc/ocfs2/cluster.conf

8) Configure the Cluster Stack in instance-1 and instance-2 sequentially

 [opc@instance-1 ~]$ sudo /sbin/o2cb.init configure Configuring the O2CB driver. ... ... Cluster to start on boot (Enter "none" to clear) [ocfs2]: ocifs2 ...
Enter fullscreen mode Exit fullscreen mode

9) Verify the settings for Cluster Stack and note the values are loaded, mounted and online. But the heart beat is not active

 [opc@instance-1 ~]$ sudo /sbin/o2cb.init status Driver for "configfs": Loaded Filesystem "configfs": Mounted Stack glue driver: Loaded Stack plugin "o2cb": Loaded Driver for "ocfs2_dlmfs": Loaded Filesystem "ocfs2_dlmfs": Mounted Checking O2CB cluster "ocifs2": Online Heartbeat dead threshold: 31 Network idle timeout: 30000 Network keepalive delay: 2000 Network reconnect delay: 2000 Heartbeat mode: Local Checking O2CB heartbeat: Not active Debug file system at /sys/kernel/debug: mounted
Enter fullscreen mode Exit fullscreen mode

Repeat above steps in instance-2

10) Setup the Boot time startup of o2cb and ocfs2 services on instance-1 and instance-2

 sudo systemctl enable o2cb sudo systemctl enable ocfs2
Enter fullscreen mode Exit fullscreen mode

11) Configure Kernel for Cluster Operation on instance-1 and instance-2 using SSH

 sudo sysctl kernel.panic=30 sudo sysctl kernel.panic_on_oops=1
Enter fullscreen mode Exit fullscreen mode

Add the entries to sysctl configuration file for persistence

 sudo vi /etc/sysctl.conf kernel.panic=30 kernel.panic_on_oops=1
Enter fullscreen mode Exit fullscreen mode

12) Create OCFS2 Volume on only instance-1

 [opc@instance-1 ~]$ sudo mkfs.ocfs2 -L "ocfs2" /dev/sdb mkfs.ocfs2 1.8.6 Cluster stack: classic o2cb Label: ocfs2 ... Formatting Journals: ... Writing lost+found: done mkfs.ocfs2 successful
Enter fullscreen mode Exit fullscreen mode

Create a mount directory

 [opc@instance-1 ~]$ sudo mkdir /ocfs2
Enter fullscreen mode Exit fullscreen mode

Specify the netdev option in /etc/fstab to allow the system to mount ocfs2 volume during boot time

 sudo vi /etc/fstab /dev/sdb /ocfs2 ocfs2 _netdev,defaults 0 0
Enter fullscreen mode Exit fullscreen mode

Reload the sysctl and mount

 sudo systemctl daemon-reload sudo mount -a
Enter fullscreen mode Exit fullscreen mode

Check the mounted block volume

 [opc@instance-1 ~]$ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT ... sdb 8:16 0 1T 0 disk /ocfs2
Enter fullscreen mode Exit fullscreen mode

13) OCFS2 Volume setup and mount on instance-2

 [opc@instance-2 ~]$ sudo mkdir /ocfs2 [opc@instance-2 ~]$ sudo vi /etc/fstab [opc@instance-2 ~]$ sudo systemctl daemon-reload [opc@instance-2 ~]$ sudo mount -a [opc@instance-2 ~]$ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT ... sdb 8:16 0 1T 0 disk /ocfs2
Enter fullscreen mode Exit fullscreen mode

14) Test: Create a file on the mounted volume for eg. on instance-2

 [opc@instance-2 ~]$ cd /ocfs2/ [opc@instance-2 ocfs2]$ sudo touch shared-cluster-file.txt [opc@instance-2 ocfs2]$ ls -lart total 8 drwxr-xr-x. 2 root root 3896 Apr 29 12:03 lost+found dr-xr-xr-x. 19 root root 265 Apr 29 12:15 .. -rw-r--r--. 1 root root 0 Apr 29 12:19 shared-cluster-file.txt drwxr-xr-x. 3 root root 3896 Apr 29 12:19 .
Enter fullscreen mode Exit fullscreen mode

15) Test: Access the file on the mounted volume of other instance eg. on instance-1

 [opc@instance-1 ~]$ cd /ocfs2/ [opc@instance-1 ocfs2]$ ls -lart total 8 drwxr-xr-x. 2 root root 3896 Apr 29 12:03 lost+found dr-xr-xr-x. 19 root root 265 Apr 29 12:04 .. -rw-r--r--. 1 root root 0 Apr 29 12:19 shared-cluster-file.txt drwxr-xr-x. 3 root root 3896 Apr 29 12:19 .
Enter fullscreen mode Exit fullscreen mode

Top comments (0)