How to Add a Subnet to Grid

Servers that have multiple network interfaces may offload the traffic by adding another subnet to grid infra. In this post, I demonstrate how I add another subnet (i.e. 10.0.8/24) from OS to grid infrastructure.

Please note that, network resource object is a synonym of subnet from grid infrastructure's perspective.

First of all, we need to configure the network interface that we want it to be added on both nodes.
[root@primary01 ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth2
[root@primary02 ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth2
Restart the network on both nodes.
[root@primary01 ~]# service network restart
[root@primary02 ~]# service network restart
Add the second set of hostname for the new network on both nodes, which includes public IP and VIP.
[root@primary01 ~]# vi /etc/hosts
...
10.0.8.11 primary01-2 primary01-2.example.com
10.0.8.12 primary02-2 primary02-2.example.com

10.0.8.111 primary01-vip2 primary01-vip2.example.com
10.0.8.112 primary02-vip2 primary02-vip2.example.com
[root@primary02 ~]# vi /etc/hosts
...
10.0.8.11 primary01-2 primary01-2.example.com
10.0.8.12 primary02-2 primary02-2.example.com

10.0.8.111 primary01-vip2 primary01-vip2.example.com
10.0.8.112 primary02-vip2 primary02-vip2.example.com

Adding a network resource object

Using root privilege to complete our job. From now on, you should perform the following operations on only one node of RAC.
[root@primary01 ~]# export PATH=$PATH:/u01/app/12.1.0/grid/bin
Check the current network resource objects.
[root@primary01 ~]# srvctl config network
Network 1 exists
Subnet IPv4: 10.0.0.0/255.255.255.0/eth0, static
Subnet IPv6:
Ping Targets:
Network is enabled
Network is individually enabled on nodes:
Network is individually disabled on nodes:
The current network is assigned as number 1 using eth0 with static configuration.
Add another network resource for the new subnet. In which, the new network number would be 2.
[root@primary01 ~]# srvctl add network -k 2 -S 10.0.8.0/255.255.255.0/eth2
[root@primary01 ~]# srvctl config network
Network 1 exists
Subnet IPv4: 10.0.0.0/255.255.255.0/eth0, static
Subnet IPv6:
Ping Targets:
Network is enabled
Network is individually enabled on nodes:
Network is individually disabled on nodes:
Network 2 exists
Subnet IPv4: 10.0.8.0/255.255.255.0/eth2, static
Subnet IPv6:
Ping Targets:
Network is enabled
Network is individually enabled on nodes:
Network is individually disabled on nodes:

Adding one VIP resource object for each node

Check current VIP settings.
[root@primary01 ~]# srvctl config vip -n primary01
VIP exists: network number 1, hosting node primary01
VIP Name: primary01-vip
VIP IPv4 Address: 10.0.0.111
VIP IPv6 Address:
VIP is enabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:
[root@primary01 ~]# srvctl config vip -n primary02
VIP exists: network number 1, hosting node primary02
VIP Name: primary02-vip
VIP IPv4 Address: 10.0.0.112
VIP IPv6 Address:
VIP is enabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:
Add the new set of VIP to network #2
[root@primary01 ~]# srvctl add vip -n primary01 -k 2 -A primary01-vip2/255.255.255.0/eth2
[root@primary01 ~]# srvctl add vip -n primary02 -k 2 -A primary02-vip2/255.255.255.0/eth2
Check current VIP settings again.
[root@primary01 ~]# srvctl config vip -n primary01
VIP exists: network number 1, hosting node primary01
VIP Name: primary01-vip
VIP IPv4 Address: 10.0.0.111
VIP IPv6 Address:
VIP is enabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:
VIP exists: network number 2, hosting node primary01
VIP Name: primary01-vip2
VIP IPv4 Address: 10.0.8.111
VIP IPv6 Address:
VIP is enabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:
[root@primary01 ~]# srvctl config vip -n primary02
VIP exists: network number 1, hosting node primary02
VIP Name: primary02-vip
VIP IPv4 Address: 10.0.0.112
VIP IPv6 Address:
VIP is enabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:
VIP exists: network number 2, hosting node primary02
VIP Name: primary02-vip2
VIP IPv4 Address: 10.0.8.112
VIP IPv6 Address:
VIP is enabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:
Now, we can start to deal with SCAN related configuration of the second network.

Adding a SCAN resource object

Add related entries to DNS.
[root@dns ~]# cd /var/named/
[root@dns named]# vi example.com.zone
[root@dns named]# vi 0.10.zone
[root@dns named]# systemctl restart named
Verify the hostname for primary-cluster-scan2.
[root@primary01 ~]$ nslookup primary-cluster-scan2
Server:         10.0.0.199
Address:        10.0.0.199#53

Name:   primary-cluster-scan2.example.com
Address: 10.0.8.82
Name:   primary-cluster-scan2.example.com
Address: 10.0.8.83
Name:   primary-cluster-scan2.example.com
Address: 10.0.8.84
Name:   primary-cluster-scan2.example.com
Address: 10.0.8.81
Add the new SCAN.
[root@primary01 ~]# srvctl add scan -k 2 -n primary-cluster-scan2
Check the new SCAN configuration.
[root@primary01 ~]# srvctl config scan -k 2
SCAN name: primary-cluster-scan2, Network: 2
Subnet IPv4: 10.0.8.0/255.255.255.0/eth2, static
Subnet IPv6:
SCAN 0 IPv4 VIP: 10.0.8.83
SCAN VIP is enabled.
SCAN VIP is individually enabled on nodes:
SCAN VIP is individually disabled on nodes:
SCAN 1 IPv4 VIP: 10.0.8.84
SCAN VIP is enabled.
SCAN VIP is individually enabled on nodes:
SCAN VIP is individually disabled on nodes:
SCAN 2 IPv4 VIP: 10.0.8.81
SCAN VIP is enabled.
SCAN VIP is individually enabled on nodes:
SCAN VIP is individually disabled on nodes:
SCAN 3 IPv4 VIP: 10.0.8.82
SCAN VIP is enabled.
SCAN VIP is individually enabled on nodes:
SCAN VIP is individually disabled on nodes:
[root@primary01 ~]# srvctl status scan -k 2
SCAN VIP scan1_net2 is enabled
SCAN VIP scan1_net2 is not running
SCAN VIP scan2_net2 is enabled
SCAN VIP scan2_net2 is not running
SCAN VIP scan3_net2 is enabled
SCAN VIP scan3_net2 is not running
SCAN VIP scan4_net2 is enabled
SCAN VIP scan4_net2 is not running
As you can see the new SCAN is enabled, but is not running. Let's start the SCAN.
[root@primary01 ~]# srvctl start scan -k 2
[root@primary01 ~]# srvctl status scan -k 2
SCAN VIP scan1_net2 is enabled
SCAN VIP scan1_net2 is running on node primary01
SCAN VIP scan2_net2 is enabled
SCAN VIP scan2_net2 is running on node primary02
SCAN VIP scan3_net2 is enabled
SCAN VIP scan3_net2 is running on node primary01
SCAN VIP scan4_net2 is enabled
SCAN VIP scan4_net2 is running on node primary02
We have added another subnet to RAC. Still, so far it's no function for us. Next, we will make the additional subnet useful and workable.
How to Add a Cluster Listener on Second Network to Grid