How to Change Subnet of Grid

  • by
Oracle 12c uses 4 types of IP addresses to configure network of RAC:
  1. Public IP
  2. Private IP
  3. Virtual IP
  4. SCAN IP
In which, Public IP, Virtual IP and SCAN IP are in the same subnet. Up to 4 pairs of Private IP can be set in RAC, which should be in another subnet and used for interconnect redundancy to secure high availability of RAC.

Before you can really modify configuration in RAC, you should make sure that all of the network settings other than RAC are all changed and ready.
  1. Network interfaces: IP Address, Gateway and DNS for this NIC.
  2. Local resolution files: /etc/hosts, /etc/resolv.conf, or /etc/sysconfig/network, etc.
  3. Central resolution: DNS Server
  4. Appliances: Routers, Firewalls, IDS or IPS.
Our goal in this post is to notify RAC that we have changed the public subnet from 192.168.15.0/24 to 192.168.21.0/24. Let’s see the configuration in the grid infrastructure.

If you are working with Oracle 12c, please be aware of the following things:
  • You just need to do changes on only one node. The result will take effect automatically on all other nodes.
  • You don’t have to stop the resources before changing IP Addresses. That is to say, you can do the changes online.
  • The order of changes should be followed in most cases:
    1. Subnet (of Public IP)
    2. Virtual IP
    3. SCAN IP
  • In this post, we do all the changes by root for convenience because some actions require higher privilege to be performed. This may not meet your situation.

A. Changing the Subnet of RAC

It may seem to be no problem at all in RAC.
[root@primary01 ~]# oifcfg iflist
eth0  192.168.21.0
eth1  192.168.24.0
eth1  169.254.0.0

This is because the above information is retrieved from the current setting of network interfaces. From RAC point of view, it still hold stale information about the subnet.
[root@primary01 ~]# oifcfg getif -global
eth0  192.168.15.0  global  public
eth1  192.168.24.0  global  cluster_interconnect

You have to delete the old configuration then set the new one on RAC server
[root@primary01 ~]# oifcfg delif -global eth0/192.168.15.0
[root@primary01 ~]# oifcfg setif -global eth0/192.168.21.0:public

[root@primary01 ~]# oifcfg getif -global
eth1  192.168.24.0  global  cluster_interconnect
eth0  192.168.21.0  global  public

Note: If you want to change the subnet of private IP as well, you should setif first, then delif:
[root@primary01 ~]# oifcfg setif -global eth1/192.168.83.0:cluster_interconnect
[root@primary01 ~]# oifcfg delif -global eth1/192.168.24.0

[root@primary01 ~]# oifcfg getif -global
eth0 192.168.21.0 global public
eth1 192.168.83.0 global cluster_interconnect

B. Changing Virtual IP Addresses

Let’s check VIP on all nodes before change.
[root@primary01 ~]# srvctl config nodeapps -a
Network 1 exists
Subnet IPv4: 192.168.15.0/255.255.255.0/eth0, static
Subnet IPv6:
Ping Targets:
Network is enabled
Network is individually enabled on nodes:
Network is individually disabled on nodes:
VIP exists: network number 1, hosting node primary01
VIP Name: primary01-vip
VIP IPv4 Address: 192.168.21.111
VIP IPv6 Address:
VIP is enabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:
VIP exists: network number 1, hosting node primary02
VIP Name: primary02-vip
VIP IPv4 Address: 192.168.21.112
VIP IPv6 Address:
VIP is enabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:

We can use the above information to modify VIP for every node, one node one command.
[root@primary01 ~]# srvctl modify nodeapps -n primary01 -A primary01-vip/255.255.255.0/eth0
[root@primary01 ~]# srvctl modify nodeapps -n primary02 -A primary02-vip/255.255.255.0/eth0

Actually, we are not modifying VIP addresses, we are notifying RAC to flush the old data and refresh them. That’s why we can do it online.

Let’s check the status after modifying VIP.
[root@primary01 ~]# srvctl config nodeapps -a
Network 1 exists
Subnet IPv4: 192.168.21.0/255.255.255.0/eth0, static
Subnet IPv6:
Ping Targets:
Network is enabled
Network is individually enabled on nodes:
Network is individually disabled on nodes:
VIP exists: network number 1, hosting node primary01
VIP Name: primary01-vip
VIP IPv4 Address: 192.168.21.111
VIP IPv6 Address:
VIP is enabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:
VIP exists: network number 1, hosting node primary02
VIP Name: primary02-vip
VIP IPv4 Address: 192.168.21.112
VIP IPv6 Address:
VIP is enabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:
[root@primary01 ~]# srvctl config vip -n primary01
VIP exists: network number 1, hosting node primary01
VIP Name: primary01-vip
VIP IPv4 Address: 192.168.21.111
VIP IPv6 Address:
VIP is enabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:
[root@primary01 ~]# srvctl config vip -n primary02
VIP exists: network number 1, hosting node primary02
VIP Name: primary02-vip
VIP IPv4 Address: 192.168.21.112
VIP IPv6 Address:
VIP is enabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:

As you can see, VIP are all changed on all nodes.

C. Changing SCAN IP Addresses

Check the SCAN name before changing SCAN IP
[root@primary01 ~]# srvctl config scan
SCAN name: primary-cluster-scan, Network: 1
Subnet IPv4: 192.168.21.0/255.255.255.0/eth0, static
Subnet IPv6:
SCAN 0 IPv4 VIP: 192.168.15.81
SCAN VIP is enabled.
SCAN VIP is individually enabled on nodes:
SCAN VIP is individually disabled on nodes:
SCAN 1 IPv4 VIP: 192.168.15.82
SCAN VIP is enabled.
SCAN VIP is individually enabled on nodes:
SCAN VIP is individually disabled on nodes:
SCAN 2 IPv4 VIP: 192.168.15.83
SCAN VIP is enabled.
SCAN VIP is individually enabled on nodes:
SCAN VIP is individually disabled on nodes:

Use the SCAN name to refresh itself.
[root@primary01 ~]# srvctl modify scan -n primary-cluster-scan
Let’s see the final result.
[root@primary01 ~]# srvctl config scan
SCAN name: primary-cluster-scan, Network: 1
Subnet IPv4: 192.168.21.0/255.255.255.0/eth0, static
Subnet IPv6:
SCAN 0 IPv4 VIP: 192.168.21.83
SCAN VIP is enabled.
SCAN VIP is individually enabled on nodes:
SCAN VIP is individually disabled on nodes:
SCAN 1 IPv4 VIP: 192.168.21.82
SCAN VIP is enabled.
SCAN VIP is individually enabled on nodes:
SCAN VIP is individually disabled on nodes:
SCAN 2 IPv4 VIP: 192.168.21.81
SCAN VIP is enabled.
SCAN VIP is individually enabled on nodes:
SCAN VIP is individually disabled on nodes:
[root@primary01 ~]# srvctl status scan
SCAN VIP scan1 is enabled
SCAN VIP scan1 is running on node primary02
SCAN VIP scan2 is enabled
SCAN VIP scan2 is running on node primary01
SCAN VIP scan3 is enabled
SCAN VIP scan3 is running on node primary01
[root@primary01 ~]# srvctl status scan_listener
SCAN Listener LISTENER_SCAN1 is enabled
SCAN listener LISTENER_SCAN1 is running on node primary02
SCAN Listener LISTENER_SCAN2 is enabled
SCAN listener LISTENER_SCAN2 is running on node primary01
SCAN Listener LISTENER_SCAN3 is enabled
SCAN listener LISTENER_SCAN3 is running on node primary01

[root@primary01 ~]# crs_stat -t
Name           Type           Target    State     Host
------------------------------------------------------------
ora.DATA.dg    ora....up.type ONLINE    ONLINE    primary01
ora....ER.lsnr ora....er.type ONLINE    ONLINE    primary01
ora....N1.lsnr ora....er.type ONLINE    ONLINE    primary02
ora....N2.lsnr ora....er.type ONLINE    ONLINE    primary01
ora....N3.lsnr ora....er.type ONLINE    ONLINE    primary01
ora.MGMTLSNR   ora....nr.type ONLINE    ONLINE    primary01
ora.asm        ora.asm.type   ONLINE    ONLINE    primary01
ora.compdb.db  ora....se.type ONLINE    ONLINE    primary01
ora.cvu        ora.cvu.type   ONLINE    ONLINE    primary01
ora.mgmtdb     ora....db.type ONLINE    ONLINE    primary01
ora....network ora....rk.type ONLINE    ONLINE    primary01
ora.oc4j       ora.oc4j.type  ONLINE    ONLINE    primary01
ora.ons        ora.ons.type   ONLINE    ONLINE    primary01
ora....SM1.asm application    ONLINE    ONLINE    primary01
ora....01.lsnr application    ONLINE    ONLINE    primary01
ora....y01.ons application    ONLINE    ONLINE    primary01
ora....y01.vip ora....t1.type ONLINE    ONLINE    primary01
ora....SM2.asm application    ONLINE    ONLINE    primary02
ora....02.lsnr application    ONLINE    ONLINE    primary02
ora....y02.ons application    ONLINE    ONLINE    primary02
ora....y02.vip ora....t1.type ONLINE    ONLINE    primary02
ora.scan1.vip  ora....ip.type ONLINE    ONLINE    primary02
ora.scan2.vip  ora....ip.type ONLINE    ONLINE    primary01
ora.scan3.vip  ora....ip.type ONLINE    ONLINE    primary01

There’re could be some troubles …

If necessary, please modify the endpoint listener on all nodes by yourself.
[root@primary01 ~]# vi $GRID_HOME/network/admin/endpoints_listener.ora
[root@primary02 ~]# vi $GRID_HOME/network/admin/endpoints_listener.ora

If SCAN is enabled, but not running, you may have to restart SCAN.
[root@primary01 ~]# srvctl status scan
SCAN VIP scan1 is enabled
SCAN VIP scan1 is not running
SCAN VIP scan2 is enabled
SCAN VIP scan2 is not running
SCAN VIP scan3 is enabled
SCAN VIP scan3 is not running
[root@primary01 admin]# srvctl stop scan
[root@primary01 admin]# srvctl start scan

If you suspect the cluster is somewhat dysfunctional, you may have to restart the cluster.
[root@primary01 ~]# crsctl stop cluster -all
[root@primary01 ~]# crsctl start cluster -all

Leave a Reply

Your email address will not be published. Required fields are marked *