How to Build SSH Equivalency Between Two Servers

  • by

In this post, we will introduce two kinds of ways to build user’s SSH equivalency.

Build SSH Equivalency Manually

Before 11g, Oracle Universal Installer (OUI) does not setup the passwordless SSH connectivity during the installation time, we have to build the SSH equivalency between nodes by ourselves.

Here are the steps that you can build it manually and probably can prevent the error ORA-06006 during 12c installation.

Note: If you had ever build the equivalency but failed, I suggest that you remove all related files and start it over.

A. Create RSA and DSA public and private keys on both nodes.

On BOTH nodes, execute the following commands:
mkdir ~/.ssh
chmod 700 ~/.ssh
cd ~/.ssh
ssh-keygen -t rsa
ssh-keygen -t dsa

If there’s already one ~/.ssh, you may consider to remove it and create one for this operation.

The lab result on Node 1 is as followings: (This is also an example for Node 2.)

[oracle@primary01 ~]$ mkdir ~/.ssh
[oracle@primary01 ~]$ chmod 700 ~/.ssh
[oracle@primary01 ~]$ cd ~/.ssh
[oracle@primary01 .ssh]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_rsa.
Your public key has been saved in /home/oracle/.ssh/id_rsa.pub.
The key fingerprint is:
... oracle@primary01.example.com
The key's randomart image is:
+--[ RSA 2048]----+
...
+-----------------+
[oracle@primary01 .ssh]$ ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/home/oracle/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/oracle/.ssh/id_dsa.
Your public key has been saved in /home/oracle/.ssh/id_dsa.pub.
The key fingerprint is:
... oracle@primary01.example.com
The key's randomart image is:
+--[ DSA 1024]----+
...
+-----------------+

B. Pass the public keys to each other.

On Node 1, execute the following commands:

cat *.pub >> authorized_keys
scp authorized_keys primary02:/home/oracle/.ssh/

The lab result on Node 1 is as followings:

[oracle@primary01 .ssh]$ cat *.pub >> authorized_keys
[oracle@primary01 .ssh]$ scp authorized_keys primary02:/home/oracle/.ssh/
The authenticity of host 'primary02 (10.10.10.12)' can't be established.
RSA key fingerprint is ...
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'primary02,10.10.10.12' (RSA) to the list of known hosts.
oracle@primary02's password:
authorized_keys                                100% 1028     1.0KB/s   00:00

On Node 2, execute the following commands:

cat *.pub >> authorized_keys
scp authorized_keys primary01:/home/oracle/.ssh/

The lab result on Node 2 is as followings:

[oracle@primary02 .ssh]$ cat *.pub >> authorized_keys
[oracle@primary02 .ssh]$ scp authorized_keys primary01:/home/oracle/.ssh/
The authenticity of host 'primary01 (10.10.10.11)' can't be established.
RSA key fingerprint is ....
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'primary01,10.10.10.11' (RSA) to the list of known hosts.
oracle@primary01's password:
authorized_keys                                100% 2056     2.0KB/s   00:00

So far, we have established the connectivity, but we should verify the functions before proceeding installation.

C. Verify the connectivity on both nodes.

On Node 1, we check the hostname of Node 2.

[oracle@primary01 .ssh]$ ssh primary02 hostname
primary02.example.com

On Node 2, we check the hostname of Node 1.

[oracle@primary02 .ssh]$ ssh primary01 hostname
primary01.example.com

The SSH equivalency is established successfully. Pretty easy, right? But please replace the hostname for your requirements and follow the sequence closely.

If there’s anything wrong, please remove all ~/.ssh and start it over.

Build SSH Equivalency by Oracle Scripts

Since Oracle 11g, we have scripts that can build equivalency of SSH for us. It is usually in the installation software of grid infrastructure.

For 11.2 or 12.1, the file is at:

<PATH_TO_SOURCE>/grid/sshsetup/sshUserSetup.sh

For 12.2 or later, the file is at:

<PATH_TO_SOURCE>/oui/prov/resources/scripts/sshUserSetup.sh

In this case, we’re using Oracle database 18c grid infrastructure, the path of scripts will be:

/u01/app/18.0.0/grid/oui/prov/resources/scripts/sshUserSetup.sh

Build SSH Equivalency for User grid

Let’s see how we build SSH equivalency by this script. First of all, user grid:

[grid@primary01 ~]$ cd /u01/app/18.0.0/grid/oui/prov/resources/scripts
[grid@primary01 scripts]$ ./sshUserSetup.sh -user grid -hosts "primary01 primary02" -noPromptPassphrase
The output of this script is also logged into /tmp/sshUserSetup_2019-05-09-11-29-57.log
Hosts are primary01 primary02
user is grid
Platform:- Linux
Checking if the remote hosts are reachable
PING primary01 (10.10.10.11) 56(84) bytes of data.
64 bytes from primary01 (10.10.10.11): icmp_seq=1 ttl=64 time=0.014 ms
64 bytes from primary01 (10.10.10.11): icmp_seq=2 ttl=64 time=0.055 ms
64 bytes from primary01 (10.10.10.11): icmp_seq=3 ttl=64 time=0.052 ms
64 bytes from primary01 (10.10.10.11): icmp_seq=4 ttl=64 time=0.053 ms
64 bytes from primary01 (10.10.10.11): icmp_seq=5 ttl=64 time=0.034 ms

--- primary01 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4018ms
rtt min/avg/max/mdev = 0.014/0.041/0.055/0.017 ms
PING primary02 (10.10.10.12) 56(84) bytes of data.
64 bytes from primary02 (10.10.10.12): icmp_seq=1 ttl=64 time=0.201 ms
64 bytes from primary02 (10.10.10.12): icmp_seq=2 ttl=64 time=0.810 ms
64 bytes from primary02 (10.10.10.12): icmp_seq=3 ttl=64 time=0.337 ms
64 bytes from primary02 (10.10.10.12): icmp_seq=4 ttl=64 time=0.989 ms
64 bytes from primary02 (10.10.10.12): icmp_seq=5 ttl=64 time=0.584 ms

--- primary02 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4006ms
rtt min/avg/max/mdev = 0.201/0.584/0.989/0.291 ms
Remote host reachability check succeeded.
The following hosts are reachable: primary01 primary02.
The following hosts are not reachable: .
All hosts are reachable. Proceeding further...
firsthost primary01
numhosts 2
The script will setup SSH connectivity from the host primary01.example.com to all
the remote hosts. After the script is executed, the user can use SSH to run
commands on the remote hosts or copy files between this host primary01.example.com
and the remote hosts without being prompted for passwords or confirmations.

NOTE 1:
As part of the setup procedure, this script will use ssh and scp to copy
files between the local host and the remote hosts. Since the script does not
store passwords, you may be prompted for the passwords during the execution of
the script whenever ssh or scp is invoked.

NOTE 2:
AS PER SSH REQUIREMENTS, THIS SCRIPT WILL SECURE THE USER HOME DIRECTORY
AND THE .ssh DIRECTORY BY REVOKING GROUP AND WORLD WRITE PRIVILEGES TO THESE
directories.

Do you want to continue and let the script make the above mentioned changes (yes/no)?
yes

The user chose yes
User chose to skip passphrase related questions.
Creating .ssh directory on local host, if not present already
Creating authorized_keys file on local host
Changing permissions on authorized_keys to 644 on local host
Creating known_hosts file on local host
Changing permissions on known_hosts to 644 on local host
Creating config file on local host
If a config file exists already at /home/grid/.ssh/config, it would be backed up to /home/grid/.ssh/config.backup.
Removing old private/public keys on local host
Running SSH keygen on local host with empty passphrase
Generating public/private rsa key pair.
Your identification has been saved in /home/grid/.ssh/id_rsa.
Your public key has been saved in /home/grid/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:X0Fpr00DeZWGVVCLKvPAM4Ent/bjUxe7fOKZAfv0ouY grid@primary01.example.com
The key's randomart image is:
+---[RSA 1024]----+
|            .o+=*|
|        .  .=.oo.|
|       o + ..=.. |
|        = o ..+. |
|        SX .o+ .o|
|        ..O..+.o |
|          .+o = .|
|          ..oooB.|
|           +Eo=oo|
+----[SHA256]-----+
Creating .ssh directory and setting permissions on remote host primary01
THE SCRIPT WOULD ALSO BE REVOKING WRITE PERMISSIONS FOR group AND others ON THE HOME DIRECTORY FOR grid. THIS IS AN SSH REQUIREMENT.
The script would create ~grid/.ssh/config file on remote host primary01. If a config file exists already at ~grid/.ssh/config, it would be backed up to ~grid/.ssh/config.backup.
The user may be prompted for a password here since the script would be running SSH on host primary01.
Warning: Permanently added 'primary01,10.10.10.11' (ECDSA) to the list of known hosts.
grid@primary01's password:
Done with creating .ssh directory and setting permissions on remote host primary01.
Creating .ssh directory and setting permissions on remote host primary02
THE SCRIPT WOULD ALSO BE REVOKING WRITE PERMISSIONS FOR group AND others ON THE HOME DIRECTORY FOR grid. THIS IS AN SSH REQUIREMENT.
The script would create ~grid/.ssh/config file on remote host primary02. If a config file exists already at ~grid/.ssh/config, it would be backed up to ~grid/.ssh/config.backup.
The user may be prompted for a password here since the script would be running SSH on host primary02.
Warning: Permanently added 'primary02,10.10.10.12' (ECDSA) to the list of known hosts.
grid@primary02's password:
Done with creating .ssh directory and setting permissions on remote host primary02.
Copying local host public key to the remote host primary01
The user may be prompted for a password or passphrase here since the script would be using SCP for host primary01.
grid@primary01's password:
Done copying local host public key to the remote host primary01
Copying local host public key to the remote host primary02
The user may be prompted for a password or passphrase here since the script would be using SCP for host primary02.
grid@primary02's password:
Done copying local host public key to the remote host primary02
cat: /home/grid/.ssh/known_hosts.tmp: No such file or directory
cat: /home/grid/.ssh/authorized_keys.tmp: No such file or directory
SSH setup is complete.

------------------------------------------------------------------------
Verifying SSH setup
===================
The script will now run the date command on the remote nodes using ssh
to verify if ssh is setup correctly. IF THE SETUP IS CORRECTLY SETUP,
THERE SHOULD BE NO OUTPUT OTHER THAN THE DATE AND SSH SHOULD NOT ASK FOR
PASSWORDS. If you see any output other than date or are prompted for the
password, ssh is not setup correctly and you will need to resolve the
issue and set up ssh again.
The possible causes for failure could be:
1. The server settings in /etc/ssh/sshd_config file do not allow ssh
for user grid.
2. The server may have disabled public key based authentication.
3. The client public key on the server may be outdated.
4. ~grid or ~grid/.ssh on the remote host may not be owned by grid.
5. User may not have passed -shared option for shared remote users or
may be passing the -shared option for non-shared remote users.
6. If there is output in addition to the date, but no password is asked,
it may be a security alert shown as part of company policy. Append the
additional text to the <OMS HOME>/sysman/prov/resources/ignoreMessages.txt file.
------------------------------------------------------------------------
--primary01:--
Running /usr/bin/ssh -x -l grid primary01 date to verify SSH connectivity has been setup from local host to primary01.
IF YOU SEE ANY OTHER OUTPUT BESIDES THE OUTPUT OF THE DATE COMMAND OR IF YOU ARE PROMPTED FOR A PASSWORD HERE, IT MEANS SSH SETUP HAS NOT BEEN SUCCESSFUL. Please note that being prompted for a passphrase may be OK but being prompted for a password is ERROR.
Thu May  9 11:30:21 CST 2019
------------------------------------------------------------------------
--primary02:--
Running /usr/bin/ssh -x -l grid primary02 date to verify SSH connectivity has been setup from local host to primary02.
IF YOU SEE ANY OTHER OUTPUT BESIDES THE OUTPUT OF THE DATE COMMAND OR IF YOU ARE PROMPTED FOR A PASSWORD HERE, IT MEANS SSH SETUP HAS NOT BEEN SUCCESSFUL. Please note that being prompted for a passphrase may be OK but being prompted for a password is ERROR.
Thu May  9 11:30:21 CST 2019
------------------------------------------------------------------------
SSH verification complete.

Then we can test the connectivity by this:

[grid@primary01 ~]$ ssh primary02 "hostname -s; id"
primary02
uid=54322(grid) gid=54321(oinstall) groups=54321(oinstall),54327(asmdba),54328(asmoper),54329(asmadmin),54330(racdba)

As we can see, no password is needed.

Build SSH Equivalency for User oracle

Next, let’s do user oracle:

[oracle@primary01 ~]$ cd /u01/app/18.0.0/grid/oui/prov/resources/scripts
[oracle@primary01 scripts]$ ./sshUserSetup.sh -user oracle -hosts "primary01 primary02" -noPromptPassphrase
The output of this script is also logged into /tmp/sshUserSetup_2019-05-09-11-31-56.log
Hosts are primary01 primary02
user is oracle
Platform:- Linux
Checking if the remote hosts are reachable
PING primary01 (10.10.10.11) 56(84) bytes of data.
64 bytes from primary01 (10.10.10.11): icmp_seq=1 ttl=64 time=0.014 ms
64 bytes from primary01 (10.10.10.11): icmp_seq=2 ttl=64 time=0.046 ms
64 bytes from primary01 (10.10.10.11): icmp_seq=3 ttl=64 time=0.042 ms
64 bytes from primary01 (10.10.10.11): icmp_seq=4 ttl=64 time=0.037 ms
64 bytes from primary01 (10.10.10.11): icmp_seq=5 ttl=64 time=0.046 ms

--- primary01 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 3999ms
rtt min/avg/max/mdev = 0.014/0.037/0.046/0.012 ms
PING primary02 (10.10.10.12) 56(84) bytes of data.
64 bytes from primary02 (10.10.10.12): icmp_seq=1 ttl=64 time=0.651 ms
64 bytes from primary02 (10.10.10.12): icmp_seq=2 ttl=64 time=0.787 ms
64 bytes from primary02 (10.10.10.12): icmp_seq=3 ttl=64 time=0.581 ms
64 bytes from primary02 (10.10.10.12): icmp_seq=4 ttl=64 time=0.572 ms
64 bytes from primary02 (10.10.10.12): icmp_seq=5 ttl=64 time=0.840 ms

--- primary02 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4004ms
rtt min/avg/max/mdev = 0.572/0.686/0.840/0.110 ms
Remote host reachability check succeeded.
The following hosts are reachable: primary01 primary02.
The following hosts are not reachable: .
All hosts are reachable. Proceeding further...
firsthost primary01
numhosts 2
The script will setup SSH connectivity from the host primary01.example.com to all
the remote hosts. After the script is executed, the user can use SSH to run
commands on the remote hosts or copy files between this host primary01.example.com
and the remote hosts without being prompted for passwords or confirmations.

NOTE 1:
As part of the setup procedure, this script will use ssh and scp to copy
files between the local host and the remote hosts. Since the script does not
store passwords, you may be prompted for the passwords during the execution of
the script whenever ssh or scp is invoked.

NOTE 2:
AS PER SSH REQUIREMENTS, THIS SCRIPT WILL SECURE THE USER HOME DIRECTORY
AND THE .ssh DIRECTORY BY REVOKING GROUP AND WORLD WRITE PRIVILEGES TO THESE
directories.

Do you want to continue and let the script make the above mentioned changes (yes/no)?
yes

The user chose yes
User chose to skip passphrase related questions.
Creating .ssh directory on local host, if not present already
Creating authorized_keys file on local host
Changing permissions on authorized_keys to 644 on local host
Creating known_hosts file on local host
Changing permissions on known_hosts to 644 on local host
Creating config file on local host
If a config file exists already at /home/oracle/.ssh/config, it would be backed up to /home/oracle/.ssh/config.backup.
Removing old private/public keys on local host
Running SSH keygen on local host with empty passphrase
Generating public/private rsa key pair.
Your identification has been saved in /home/oracle/.ssh/id_rsa.
Your public key has been saved in /home/oracle/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:vpD5baAdgEBXS7/J3wuoYnM5fbQCDw08sL2zjI03ZQk oracle@primary01.example.com
The key's randomart image is:
+---[RSA 1024]----+
| .. ..o          |
|  .... o         |
|   . *. .        |
|    o E. o       |
|       BS.       |
|      =+Bo..     |
|     =+&o+o..    |
|    * @+=o+. .   |
|   . =.oo+. .    |
+----[SHA256]-----+
Creating .ssh directory and setting permissions on remote host primary01
THE SCRIPT WOULD ALSO BE REVOKING WRITE PERMISSIONS FOR group AND others ON THE HOME DIRECTORY FOR oracle. THIS IS AN SSH REQUIREMENT.
The script would create ~oracle/.ssh/config file on remote host primary01. If a config file exists already at ~oracle/.ssh/config, it would be backed up to ~oracle/.ssh/config.backup.
The user may be prompted for a password here since the script would be running SSH on host primary01.
Warning: Permanently added 'primary01,10.10.10.11' (ECDSA) to the list of known hosts.
oracle@primary01's password:
Done with creating .ssh directory and setting permissions on remote host primary01.
Creating .ssh directory and setting permissions on remote host primary02
THE SCRIPT WOULD ALSO BE REVOKING WRITE PERMISSIONS FOR group AND others ON THE HOME DIRECTORY FOR oracle. THIS IS AN SSH REQUIREMENT.
The script would create ~oracle/.ssh/config file on remote host primary02. If a config file exists already at ~oracle/.ssh/config, it would be backed up to ~oracle/.ssh/config.backup.
The user may be prompted for a password here since the script would be running SSH on host primary02.
Warning: Permanently added 'primary02,10.10.10.12' (ECDSA) to the list of known hosts.
oracle@primary02's password:
Done with creating .ssh directory and setting permissions on remote host primary02.
Copying local host public key to the remote host primary01
The user may be prompted for a password or passphrase here since the script would be using SCP for host primary01.
oracle@primary01's password:
Done copying local host public key to the remote host primary01
Copying local host public key to the remote host primary02
The user may be prompted for a password or passphrase here since the script would be using SCP for host primary02.
oracle@primary02's password:
Done copying local host public key to the remote host primary02
cat: /home/oracle/.ssh/known_hosts.tmp: No such file or directory
cat: /home/oracle/.ssh/authorized_keys.tmp: No such file or directory
SSH setup is complete.

------------------------------------------------------------------------
Verifying SSH setup
===================
The script will now run the date command on the remote nodes using ssh
to verify if ssh is setup correctly. IF THE SETUP IS CORRECTLY SETUP,
THERE SHOULD BE NO OUTPUT OTHER THAN THE DATE AND SSH SHOULD NOT ASK FOR
PASSWORDS. If you see any output other than date or are prompted for the
password, ssh is not setup correctly and you will need to resolve the
issue and set up ssh again.
The possible causes for failure could be:
1. The server settings in /etc/ssh/sshd_config file do not allow ssh
for user oracle.
2. The server may have disabled public key based authentication.
3. The client public key on the server may be outdated.
4. ~oracle or ~oracle/.ssh on the remote host may not be owned by oracle.
5. User may not have passed -shared option for shared remote users or
may be passing the -shared option for non-shared remote users.
6. If there is output in addition to the date, but no password is asked,
it may be a security alert shown as part of company policy. Append the
additional text to the <OMS HOME>/sysman/prov/resources/ignoreMessages.txt file.
------------------------------------------------------------------------
--primary01:--
Running /usr/bin/ssh -x -l oracle primary01 date to verify SSH connectivity has been setup from local host to primary01.
IF YOU SEE ANY OTHER OUTPUT BESIDES THE OUTPUT OF THE DATE COMMAND OR IF YOU ARE PROMPTED FOR A PASSWORD HERE, IT MEANS SSH SETUP HAS NOT BEEN SUCCESSFUL. Please note that being prompted for a passphrase may be OK but being prompted for a password is ERROR.
Thu May  9 11:32:17 CST 2019
------------------------------------------------------------------------
--primary02:--
Running /usr/bin/ssh -x -l oracle primary02 date to verify SSH connectivity has been setup from local host to primary02.
IF YOU SEE ANY OTHER OUTPUT BESIDES THE OUTPUT OF THE DATE COMMAND OR IF YOU ARE PROMPTED FOR A PASSWORD HERE, IT MEANS SSH SETUP HAS NOT BEEN SUCCESSFUL. Please note that being prompted for a passphrase may be OK but being prompted for a password is ERROR.
Thu May  9 11:32:17 CST 2019
------------------------------------------------------------------------
SSH verification complete.

Then we can test the connectivity by this:

[oracle@primary01 ~]$ ssh primary02 "hostname -s; id"
primary02
uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba),54323(oper),54324(backupdba),54325(dgdba),54326(kmdba),54327(asmdba),54330(racdba)

Good!

Leave a Reply

Your email address will not be published. Required fields are marked *