ASMLib Scanned Nothing on Boot-Time
ASMLib seems to be started too soon to scan iSCSI disks which may not be ready on boot in Enterprise Linux 7.
[root@primary01 ~]# oracleasm listdisks
[root@primary01 ~]# oracleasm-discover
Using ASMLib from /opt/oracle/extapi/64/asm/orcl/1/libasm.so
[ASM Library - Generic Linux, version 2.0.12 (KABI_V2)]
Nothing was found there when we tried to list registered disks. Let's see some background information of ASMLib
[root@primary01 ~]# oracleasm configure
ORACLEASM_ENABLED=true
ORACLEASM_UID=grid
ORACLEASM_GID=asmadmin
ORACLEASM_SCANBOOT=true
ORACLEASM_SCANORDER=""
ORACLEASM_SCANEXCLUDE=""
ORACLEASM_SCAN_DIRECTORIES=""
ORACLEASM_USE_LOGICAL_BLOCK_SIZE="false"
[root@primary01 ~]# cat /etc/sysconfig/oracleasm
#
# This is a configuration file for automatic loading of the Oracle
# Automatic Storage Management library kernel driver. It is generated
# By running /etc/init.d/oracleasm configure. Please use that method
# to modify this file
#
# ORACLEASM_ENABLED: 'true' means to load the driver on boot.
ORACLEASM_ENABLED=true
# ORACLEASM_UID: Default user owning the /dev/oracleasm mount point.
ORACLEASM_UID=grid
# ORACLEASM_GID: Default group owning the /dev/oracleasm mount point.
ORACLEASM_GID=asmadmin
# ORACLEASM_SCANBOOT: 'true' means scan for ASM disks on boot.
ORACLEASM_SCANBOOT=true
# ORACLEASM_SCANORDER: Matching patterns to order disk scanning
ORACLEASM_SCANORDER=""
# ORACLEASM_SCANEXCLUDE: Matching patterns to exclude disks from scan
ORACLEASM_SCANEXCLUDE=""
# ORACLEASM_SCAN_DIRECTORIES: Scan disks under these directories
ORACLEASM_SCAN_DIRECTORIES=""
# ORACLEASM_USE_LOGICAL_BLOCK_SIZE: 'true' means use the logical block size
# reported by the underlying disk instead of the physical. The default
# is 'false'
ORACLEASM_USE_LOGICAL_BLOCK_SIZE=false
How about the status of oracleasm?
[root@primary01 ~]# oracleasm status
Checking if ASM is loaded: yes
Checking if /dev/oracleasm is mounted: yes
[root@primary01 ~]# systemctl status oracleasm
● oracleasm.service - Load oracleasm Modules
Loaded: loaded (/usr/lib/systemd/system/oracleasm.service; enabled; vendor preset: disabled)
Active: active (exited) since Sat 2018-09-29 12:55:06 CST; 21min ago
Process: 669 ExecStart=/usr/sbin/oracleasm.init start_sysctl (code=exited, status=0/SUCCESS)
Main PID: 669 (code=exited, status=0/SUCCESS)
CGroup: /system.slice/oracleasm.service
Sep 29 12:55:05 primary01.example.com systemd[1]: Starting Load o...
Sep 29 12:55:06 primary01.example.com oracleasm.init[669]: Initia...
Sep 29 12:55:06 primary01.example.com oracleasm.init[669]: Scanni...
Sep 29 12:55:06 primary01.example.com systemd[1]: Started Load or...
Hint: Some lines were ellipsized, use -l to show in full.
Solution
It seems that oracleasm is running well, but it scanned nothing on boot according to the above fact. This is because oracleasm started before the iSCSI storage is fully started by the server. So all we need to do is to re-arrange the initialization order.
We re-arrange the boot order by adding the following two lines in red.
[root@primary01 ~]# vi /usr/lib/systemd/system/oracleasm.service
...
[Unit]
Description=Load oracleasm Modules
Requires=multipathd.service iscsi.service
After=multipathd.service iscsi.service
...
Try to reboot the database server so as to verify the result.
[root@primary01 ~]# init 6
I don't see the same symptoms in Enterprise Linux 6. So I think this problem is dedicated to Enterprise Linux 7.
Please note that, Enterprise Linux 7 means Red Hat Enterprise Linux 7 and its variants including Oracle Linux 7 and CentOS 7.
If the above solution cannot resolve your problem, you may check my post for basic concepts of ASM disk: How to Resolve Grid Installation OUI Discovers No Disk
A more complicated problem is that ASM disks can be listed by ASMLib, but they cannot be discovered during the installation of grid infrastructure. In such case, you have to do some tricks to workaround the wrong sector size of ASM disks.