Skip to content
Home » Oracle » How to Resolve ASMLib Fails to Open Disk Candidates

How to Resolve ASMLib Fails to Open Disk Candidates

ASMLib Seems Good

In my grid installation plan for 18c database, I use ASMLib to make device path persistent for ASM candidate disks. As a result, oracleasm should be able to recognize those disks as expected.

[root@primary01 ~]# oracleasm listdisks
ASMDISK01
ASMDISK02
[root@primary01 ~]# oracleasm-discover
Using ASMLib from /opt/oracle/extapi/64/asm/orcl/1/libasm.so
[ASM Library - Generic Linux, version 2.0.12 (KABI_V2)]
Discovered disk: ORCL:ASMDISK01 [1048575 blocks (17179852800 bytes), maxio 16, integrity none]
Discovered disk: ORCL:ASMDISK02 [1048575 blocks (17179852800 bytes), maxio 16, integrity none]

A deeper understanding about the above result is that 1048575 blocks come from 17179852800 bytes, so the block size is calculated as 16KB (16384 bytes).

The persistent device path can be seen at /dev/oracleasm/disks.

[root@primary01 ~]# ll /dev/oracleasm/disks
total 0
brw-rw---- 1 grid asmadmin 8, 33 Oct  4 18:42 ASMDISK01
brw-rw---- 1 grid asmadmin 8, 17 Oct  4 18:42 ASMDISK02

The leading b indicates that the device type is block and the permission looked no problem at all.

OUI Fails to Discover Disks

In our case, OUI did not recognize disks that we prepared even though we provided several possible paths of candidates in Change Discovery Path.

There're two possible device paths for ASM to discover. One is /dev/oracleasm/disks, the persistent device path made by ASMLib. The other is ORCL:*, a virtual path emulated by ASMLib. Both disappointed us just like the following screenshots.

  1. /dev/oracleasm/disks

    2. Nothing is found in /dev/oracleasm/disks.

    Oracle 18c Grid Infrastructure Installation OUI - Change Discovery Path - /dev/oracleasm/disks
    Oracle 18c Grid Infrastructure Installation OUI - Change Discovery Path - /dev/oracleasm/disks
  2. ORCL:*

    3. Nothing is found in ORCL:*, either.

    Oracle 18c Grid Infrastructure Installation OUI - Change Discovery Path - ORCL:*
    Oracle 18c Grid Infrastructure Installation OUI - Change Discovery Path - ORCL:*

Inspect Installation Log

Since there's no sign, no error or no any hints in OUI, we should turn to the installation log for more clues.

[grid@primary01 ~]$ vi /tmp/GridSetupActions2018-10-04_19-47-59PM/gridSetupActions2018-10-04_19-47-59PM.log
...
INFO:  [Oct 4, 2018 19:49:10 PM] ... discoveryString = /dev/oracleasm/disks
INFO:  [Oct 4, 2018 19:49:10 PM] Determining the number of disks...
INFO:  [Oct 4, 2018 19:49:10 PM] Executing [/u01/app/18.0.0/grid/bin/kfod.bin, nohdr=true, verbose=true, disks=all, op=disks, shallow=true, asm_diskstring='/dev/oracleasm/disks']
 with environment variables {PATH=/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/grid/.local/bin:/home/grid/bin:/u01/app/grid/bin:/u01/app/18.0.0/grid/bin:/home/grid/.local/bin:/home/grid/bin:/u01/app/grid/bin:/u01/app/18.0.0/grid/bin, ORACLE_SID=+ASM1, HISTCONTROL=ignoredups, HISTSIZE=1000, TERM=xterm, LANG=en_US.UTF-8, XDG_SESSION_ID=3, DISPLAY=localhost:11.0, MAIL=/var/spool/mail/grid, LD_LIBRARY_PATH=/u01/app/18.0.0/grid/bin/../lib, LOGNAME=grid, SHLIB_PATH=/u01/app/18.0.0/grid/bin/../lib, ORACLE_HOME=/u01/app/18.0.0/grid, PWD=/u01/app/18.0.0/grid, LD_LIBRARY_PATH_64=/u01/app/18.0.0/grid/bin/../lib, _=/u01/app/18.0.0/grid/perl/bin/perl, LESSOPEN=||/usr/bin/lesspipe.sh %s, SHELL=/bin/bash, ORACLE_BASE=/tmp/GridSetupActions2018-10-04_10-47-59AM/temp_ob, SSH_TTY=/dev/pts/1, SSH_CLIENT=192.168.15.100 50986 22, OLDPWD=/u01/app/18.0.0/grid, USER=grid, TMPDIR=/tmp, SSH_AUTH_SOCK=/tmp/ssh-et8c795ixu/agent.1628, SSH_CONNECTION=192.168.15.100 50986 192.168.15.11 22, HOSTNAME=primary01.example.com, TMP=/tmp, LIBPATH=/u01/app/18.0.0/grid/bin/../lib, XDG_RUNTIME_DIR=/run/user/54322, LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:, SHLVL=2, HOME=/home/grid}
INFO:  [Oct 4, 2018 19:49:10 PM] Starting Output Reader Threads for process /u01/app/18.0.0/grid/bin/kfod.bin
INFO:  [Oct 4, 2018 19:49:10 PM] Parsing Shallow discovery returned 2 devices
INFO:  [Oct 4, 2018 19:49:10 PM] The process /u01/app/18.0.0/grid/bin/kfod.bin exited with code 0
INFO:  [Oct 4, 2018 19:49:10 PM] Waiting for output processor threads to exit.
INFO:  [Oct 4, 2018 19:49:10 PM] Output processor threads exited.
INFO:  [Oct 4, 2018 19:49:10 PM] Discovering the disks...
INFO:  [Oct 4, 2018 19:49:10 PM] Executing [/u01/app/18.0.0/grid/bin/kfod.bin, nohdr=true, verbose=true, disks=all, status=true, op=disks, asm_diskstring='/dev/oracleasm/disks']
 with environment variables {PATH=/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/grid/.local/bin:/home/grid/bin:/u01/app/grid/bin:/u01/app/18.0.0/grid/bin:/home/grid/.local/bin:/home/grid/bin:/u01/app/grid/bin:/u01/app/18.0.0/grid/bin, ORACLE_SID=+ASM1, HISTCONTROL=ignoredups, HISTSIZE=1000, TERM=xterm, LANG=en_US.UTF-8, XDG_SESSION_ID=3, DISPLAY=localhost:11.0, MAIL=/var/spool/mail/grid, LD_LIBRARY_PATH=/u01/app/18.0.0/grid/bin/../lib, LOGNAME=grid, SHLIB_PATH=/u01/app/18.0.0/grid/bin/../lib, ORACLE_HOME=/u01/app/18.0.0/grid, PWD=/u01/app/18.0.0/grid, LD_LIBRARY_PATH_64=/u01/app/18.0.0/grid/bin/../lib, _=/u01/app/18.0.0/grid/perl/bin/perl, LESSOPEN=||/usr/bin/lesspipe.sh %s, SHELL=/bin/bash, ORACLE_BASE=/tmp/GridSetupActions2018-10-04_10-47-59AM/temp_ob, SSH_TTY=/dev/pts/1, SSH_CLIENT=192.168.15.100 50986 22, OLDPWD=/u01/app/18.0.0/grid, USER=grid, TMPDIR=/tmp, SSH_AUTH_SOCK=/tmp/ssh-et8c795ixu/agent.1628, SSH_CONNECTION=192.168.15.100 50986 192.168.15.11 22, HOSTNAME=primary01.example.com, TMP=/tmp, LIBPATH=/u01/app/18.0.0/grid/bin/../lib, XDG_RUNTIME_DIR=/run/user/54322, LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:, SHLVL=2, HOME=/home/grid}
INFO:  [Oct 4, 2018 19:49:10 PM] Starting Output Reader Threads for process /u01/app/18.0.0/grid/bin/kfod.bin
INFO:  [Oct 4, 2018 19:49:10 PM] The process /u01/app/18.0.0/grid/bin/kfod.bin exited with code 0
INFO:  [Oct 4, 2018 19:49:10 PM] Waiting for output processor threads to exit.
INFO:  [Oct 4, 2018 19:49:10 PM] Output processor threads exited.
INFO:  [Oct 4, 2018 19:49:12 PM] ... discoveryString = ORCL:*
INFO:  [Oct 4, 2018 19:49:12 PM] Determining the number of disks...
INFO:  [Oct 4, 2018 19:49:12 PM] Executing [/u01/app/18.0.0/grid/bin/kfod.bin, nohdr=true, verbose=true, disks=all, op=disks, shallow=true, asm_diskstring='ORCL:*']
 with environment variables {PATH=/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/grid/.local/bin:/home/grid/bin:/u01/app/grid/bin:/u01/app/18.0.0/grid/bin:/home/grid/.local/bin:/home/grid/bin:/u01/app/grid/bin:/u01/app/18.0.0/grid/bin, ORACLE_SID=+ASM1, HISTCONTROL=ignoredups, HISTSIZE=1000, TERM=xterm, LANG=en_US.UTF-8, XDG_SESSION_ID=3, DISPLAY=localhost:11.0, MAIL=/var/spool/mail/grid, LD_LIBRARY_PATH=/u01/app/18.0.0/grid/bin/../lib, LOGNAME=grid, SHLIB_PATH=/u01/app/18.0.0/grid/bin/../lib, ORACLE_HOME=/u01/app/18.0.0/grid, PWD=/u01/app/18.0.0/grid, LD_LIBRARY_PATH_64=/u01/app/18.0.0/grid/bin/../lib, _=/u01/app/18.0.0/grid/perl/bin/perl, LESSOPEN=||/usr/bin/lesspipe.sh %s, SHELL=/bin/bash, ORACLE_BASE=/tmp/GridSetupActions2018-10-04_10-47-59AM/temp_ob, SSH_TTY=/dev/pts/1, SSH_CLIENT=192.168.15.100 50986 22, OLDPWD=/u01/app/18.0.0/grid, USER=grid, TMPDIR=/tmp, SSH_AUTH_SOCK=/tmp/ssh-et8c795ixu/agent.1628, SSH_CONNECTION=192.168.15.100 50986 192.168.15.11 22, HOSTNAME=primary01.example.com, TMP=/tmp, LIBPATH=/u01/app/18.0.0/grid/bin/../lib, XDG_RUNTIME_DIR=/run/user/54322, LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:, SHLVL=2, HOME=/home/grid}
INFO:  [Oct 4, 2018 19:49:12 PM] Starting Output Reader Threads for process /u01/app/18.0.0/grid/bin/kfod.bin
INFO:  [Oct 4, 2018 19:49:12 PM] Parsing Shallow discovery returned 0 devices
INFO:  [Oct 4, 2018 19:49:12 PM] Parsing KFOD-00302: Error encountered in device access layer: asmlib ASM:ASM Library - Generic Linux, version 2.0.12 (KABI_V2) op asm_open error Operation not permitted
INFO:  [Oct 4, 2018 19:49:12 PM] Parsing
INFO:  [Oct 4, 2018 19:49:12 PM] Parsing KFOD-00302: Error encountered in device access layer: asmlib ASM:ASM Library - Generic Linux, version 2.0.12 (KABI_V2) op asm_open error Operation not permitted
INFO:  [Oct 4, 2018 19:49:12 PM] Parsing
INFO:  [Oct 4, 2018 19:49:12 PM] The process /u01/app/18.0.0/grid/bin/kfod.bin exited with code 0
INFO:  [Oct 4, 2018 19:49:12 PM] Waiting for output processor threads to exit.
INFO:  [Oct 4, 2018 19:49:12 PM] Output processor threads exited.
INFO:  [Oct 4, 2018 19:49:12 PM] Discovering the disks...
INFO:  [Oct 4, 2018 19:49:12 PM] Executing [/u01/app/18.0.0/grid/bin/kfod.bin, nohdr=true, verbose=true, disks=all, status=true, op=disks, asm_diskstring='ORCL:*']
 with environment variables {PATH=/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/grid/.local/bin:/home/grid/bin:/u01/app/grid/bin:/u01/app/18.0.0/grid/bin:/home/grid/.local/bin:/home/grid/bin:/u01/app/grid/bin:/u01/app/18.0.0/grid/bin, ORACLE_SID=+ASM1, HISTCONTROL=ignoredups, HISTSIZE=1000, TERM=xterm, LANG=en_US.UTF-8, XDG_SESSION_ID=3, DISPLAY=localhost:11.0, MAIL=/var/spool/mail/grid, LD_LIBRARY_PATH=/u01/app/18.0.0/grid/bin/../lib, LOGNAME=grid, SHLIB_PATH=/u01/app/18.0.0/grid/bin/../lib, ORACLE_HOME=/u01/app/18.0.0/grid, PWD=/u01/app/18.0.0/grid, LD_LIBRARY_PATH_64=/u01/app/18.0.0/grid/bin/../lib, _=/u01/app/18.0.0/grid/perl/bin/perl, LESSOPEN=||/usr/bin/lesspipe.sh %s, SHELL=/bin/bash, ORACLE_BASE=/tmp/GridSetupActions2018-10-04_10-47-59AM/temp_ob, SSH_TTY=/dev/pts/1, SSH_CLIENT=192.168.15.100 50986 22, OLDPWD=/u01/app/18.0.0/grid, USER=grid, TMPDIR=/tmp, SSH_AUTH_SOCK=/tmp/ssh-et8c795ixu/agent.1628, SSH_CONNECTION=192.168.15.100 50986 192.168.15.11 22, HOSTNAME=primary01.example.com, TMP=/tmp, LIBPATH=/u01/app/18.0.0/grid/bin/../lib, XDG_RUNTIME_DIR=/run/user/54322, LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:, SHLVL=2, HOME=/home/grid}
INFO:  [Oct 4, 2018 19:49:12 PM] Starting Output Reader Threads for process /u01/app/18.0.0/grid/bin/kfod.bin
INFO:  [Oct 4, 2018 19:49:12 PM] Parsing KFOD-00302: Error encountered in device access layer: asmlib ASM:ASM Library - Generic Linux, version 2.0.12 (KABI_V2) op asm_open error Operation not permitted
INFO:  [Oct 4, 2018 19:49:12 PM] Parsing
INFO:  [Oct 4, 2018 19:49:12 PM] Parsing KFOD-00302: Error encountered in device access layer: asmlib ASM:ASM Library - Generic Linux, version 2.0.12 (KABI_V2) op asm_open error Operation not permitted
INFO:  [Oct 4, 2018 19:49:12 PM] Parsing
INFO:  [Oct 4, 2018 19:49:12 PM] The process /u01/app/18.0.0/grid/bin/kfod.bin exited with code 0
INFO:  [Oct 4, 2018 19:49:12 PM] Waiting for output processor threads to exit.
INFO:  [Oct 4, 2018 19:49:12 PM] Output processor threads exited.
...

A lot of uninterpretable error messages were there, and the key error repeated itself. So I took one of them for example:

KFOD-00302: Error encountered in device access layer: asmlib ASM:ASM Library - Generic Linux, version 2.0.12 (KABI_V2) op asm_open error Operation not permitted

It seemed that ASMLib knows the device but it cannot open the device properly.

ASM Disk Requirement

According to Oracle Automatic Storage Management Administrator's Guide for Release 18c, I found something very useful: Specifying the Logical and Physical Sector Size. I highlighted some text as below for you.

The SECTOR_SIZE disk group attribute can be set to 512, 4096, or 4K if the disks support those values.
...
A 512 native sector size disk has sector size equal to logical sector size equal to 512. The 512 native and 512 emulated heading describes the combination of 512 native disks and 512 emulated disk. The 512 emulated disk has a 4 k physical sector size and a 512 logical sector size. The values in each of the table cells represents the final value of sector sizes of the disk group. Error indicates that the disk group creation fails and an error message is displayed.

Which means, 512 or 4K is supported, otherwise is NOT supported by ASM. Now let's see the disk attributes of ASM disk candidates which are all iSCSI LUN from NAS.

[root@primary01 ~]# fdisk -l /dev/sdb1

Disk /dev/sdb1: 17.2 GB, 17178820608 bytes, 33552384 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 16384 bytes
I/O size (minimum/optimal): 16384 bytes / 1048576 bytes

As you can see, the logical sector size of this disk is 512 bytes but the physical sector size is 16384 bytes (16KB). By default, ASMLib use the physical sector size which is 16K in this case. Therefore, ASMLib cannot support the block device.

Solutions

Now the solutions are obvious to us. Either to change the physical sector size of the block device from the ground, or to make ASMLib use the logical sector size instead.

The first solution is left for you. Here in this post, I take the second solution for instance. Which means, we are going to switch from physical sector size to logical sector size in ASMLib.

[root@primary01 ~]# oracleasm configure -b
Writing Oracle ASM library driver configuration: done
[root@primary01 ~]# oracleasm configure | grep ORACLEASM_USE_LOGICAL_BLOCK_SIZE
ORACLEASM_USE_LOGICAL_BLOCK_SIZE="true"

Or you can modify the configuration file of oracleasm directly.

[root@primary01 ~]# vi /etc/sysconfig/oracleasm
...
ORACLEASM_USE_LOGICAL_BLOCK_SIZE=true

A subsequent restart of ASMLib should be done before you enter grid OUI again.

[root@primary01 ~]# oracleasm exit
Unmounting ASMlib driver filesystem: /dev/oracleasm
Unloading module "oracleasm": oracleasm
[root@primary01 ~]# oracleasm init
Loading module "oracleasm": oracleasm
Configuring "oracleasm" to use device logical block size
Mounting ASMlib driver filesystem: /dev/oracleasm
[root@primary01 ~]# oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
Instantiating disk "ASMDISK02"
Instantiating disk "ASMDISK01"

Now the question is, how do we know ASMLib start to use 512 bytes, the logical sector size to operate the device? Let's see the discovering result.

[root@primary01 ~]# oracleasm-discover
Using ASMLib from /opt/oracle/extapi/64/asm/orcl/1/libasm.so
[ASM Library - Generic Linux, version 2.0.12 (KABI_V2)]
Discovered disk: ORCL:ASMDISK01 [33552384 blocks (17178820608 bytes), maxio 1024, integrity none]
Discovered disk: ORCL:ASMDISK02 [33552384 blocks (17178820608 bytes), maxio 1024, integrity none]

According to the above, the block size can be calculated as:

17178820608 bytes / 33552384 blocks = 512 bytes

OUI seems a little different now.

Oracle 18c Grid Infrastructure Installation - Found ORCL Disk Candidates After Changing ORACLEASM_USE_LOGICAL_BLOCK_SIZE to true
Oracle 18c Grid Infrastructure Installation - Found ORCL Disk Candidates After Changing ORACLEASM_USE_LOGICAL_BLOCK_SIZE to true

That's what we want! But it looks like a workaround and not a real solution.

Tags:

Leave a Reply

Your email address will not be published. Required fields are marked *