In this post, we will see step by step configuration of servers for installation of Oracle RAC for home system
Please click on the INDEX and browse for more interesting posts.
For more posts regarding Oracle 19c RAC, please click on the links below
Installation of 19c Grid and RAC database using Virtual Box
Steps to fix a corrupted SPFile in ASM with 2 node RAC
Change SCAN Name in Oracle 19c
19c Grid Patch from 19.3 to 19.5
Softwares required :
- Virtual Box 6.1 for Windows
- Oracle Enterprise Linux 7.3
- Oracle Database 19c Release 3 Linux x86-64
- kmod-20-21.el7.x86_64.rpm
- kmod-libs-20-21.el7.x86_64.rpm
- oracleasmlib-2.0.12-1.el7.x86_64.rpm
- kmod-oracleasm-2.0.8-17.el7.x86_64
- oracleasmlib-2.0.12-1.el7.x86_64.rpm
- oracleasm-support-2.1.8-3.el7.x86_64.rpm
Here we are assuming, that you have downloaded the required softwares, installed Virtual Box 6.1.
Please make sure your system is compatible for the upgrade, since a minimum of 8GB RAM is required per Node for installation of Oracle RAC. In this example, we will be using 6GB of RAM per node, because of which our cluster verification utility would error out.
Also, please note that you will receive some errors in your Cluster Verification utility related to NTP and Network, which needs to be ignored for your home setup. Please make sure they aren’t ignored if you are implementing this in your Work Environment
Configuration :
Nodes : 2 (Jack and Jill)
DNS Setup : Both these machines would also be configured as a DNS server
This is not realistic, but to save on RAM/CPU, the requirement of 3rd VM needs to be cut down
Shared Storage : In real life, for setting up RAC, you would need a shared storage which can be SAN or NAS.
However in this case, we will use Oracle Virtual Boxes feature to create shared LUNs
Disk Space for each Node :
Total : 80GB
SWAP : 16 GB
/tmp : 10 GB
/boot : 2 GB
/ : 10 GB
/u01 : 42 GB
RAM (Per Node) : 6 GB (Oracle recommends 8GB per Node, but due to resource constraints, we are proceeding with 6GB per node)
Processor (Per Node) : 2
Network Configuration :
Public Network : Host ONly Interface with IP Address : 172.168.190.x
Private Network : Internal Interface : 192.168.190.x
DHCP Server :
Server Address : 172.168.190.100
Server Mask : 255.255.0.0
Lower Address Bound :172.168.190.1
Upper Address Bound : 172.168.190.254
Shared Disks :
4 Shared RAW disks of 50GB each
2 Shared RAW Disks of 20GB each
STEPS :
1. Click on Host Network Manager in Virtual Box :
2. Click Create and after successful creation of the Ethernet Adapter, enter the below IP details :
3. Click on DHCP Server and Click on Enable Server. Provide the details as mentioned below and close :
Server Address : 172.168.190.100
Server Mask :255.255.0.0
Lower Address Bound :172.168.190.1
Upper Address Bound : 172.168.190.254
4. Click on New and click Next
5. Provide the Machine Name, Machine Type and Folder where your *.vdi will be saved and click Next
6. Provide the Desired Memory and click Next. In here we are using 6GB per Node due to resource constraints
7. Select Create a Virtual hard disk now and click Next
8. Select VDI (VirtualBox Disk Image) for the Hard Disk file type and click Next
9. Click Dynamically Allocated for Storage on Physical Disk, which would consume space as and when your VDI consumes space and wont consumed the full mentioned space for your VDI
10. Provide the File Location and Size of your VDI and click Create. Here we are allotting 80GB of Dynamically Allocated space
11. Perform the same steps (1-10) to create your Node 2
12. Now click on Settings of your Node 1 – here Jack
13. In General Tab, change Shared Clipboard and DragnDrop to Bidirectional
14. In System tab, Uncheck Floppy and keep Hard Disk at top, so that it boots from Hard Disk. Select 2 Processors
Also change the pointing devices to USB Tablet for smooth operation of your mouse inside the Virtual Machine
15. In Storage, use Host I/O Cache
16. Disable Audio
17. In Network
Adapter 1 :
Name : Host Only Adapter you had created in Steps 1 and 2
Promiscous Mode : Allow All
Adapter 2 :
Attached to : Internal Network
Promiscuous Mode : Allow All
18. Create a shared folder to keep your softwares and from where you can copy files from your Physical Disk to the Virtual Disk
19. Repeat the same steps (12-18) for your Node 2
20. Now we need to configure our Shared Disks for ASM.
Go to Storage under Settings of Node 1 and click on Add Hard Disks
21. Click on Create and follow the steps as mentioned in the below pictures :
Click on Fixed Size for these ASM Disks
22. Repeat Step 21 to create a total of 4 RAW Disks of 50 GB each and 2 RAW Disks of 20GB each
23. Click OK
24. Currently these RAW disks arent shared and allotted to only Node 1. We need to share the disks and allot to Node 2
25. From Files, click Virtual Media Manager
26. Click on each RAW Disks and change the type to Shareable and hit Apply
27. Go to Node 2 Settings –> Storage –> Add the Shared RAW Disks
28. Load the OEL 7.3 ISO image in both the nodes
29. Start Node 1
30. Click on Install Oracle Linux 7.3 and hit Enter
31. Select English and English (United States)
32. Select your localization.
Here we are selecting Asia/Kolkata as Date & Time
33. Click on Installation Destination
Select the 80 GB Hard Disk , click I will configure partitioning and click Done
34. Click on Standard Partition and + symbol to add each mount points as mentioned in the configuration section above and click Done and Accept Changes
35. Click on Software Selection and select following packages or Select all packages, if you are hesitating to install any packages later
Server with GUI
Hardware Monitoring Utilities
Large Systems Performance
Network file system client
Performance Tools
Compatibility Libraries
Development Tools
36. Disable Kdump and Security Policy
37. Repeat the steps 29-36 for Node 2
38. Click on Network & Hostname of Node 1
Change Hostname to jack.infraxpertzz.com and click on configure enp0s3
39. Make the following Changes :
General : Select Automatically connect to this network when it is available
IPv6 Settings : Ignore
Ipv4 settings :
- Method : Manual
- Address: 172.168.190.1
- Netmask : 255.255.0.0
- Gateway : 172.168.190.100
- DNS Servers : 172.168.190.1 , 172.168.190.2
40. Click on enp0s8 and make the following changes :
General : Select Automatically connect to this network when it is available
IPv6 Settings : Ignore
Ipv4 settings :
- Method : Manual
- Address: 192.168.190.1
- Netmask : 255.255.255.0
- Gateway : 172.168.190.100
- DNS Servers : 172.168.190.1 , 172.168.190.2
41. Click Done and Begin Installation
42. Provide the Root Password
43. Go to Node 2 and click on Network & Hostname
Change Hostname to jill.infraxpertzz.com and click on configure enp0s3
44. Make the following Changes :
General : Select Automatically connect to this network when it is available
IPv6 Settings : Ignore
Ipv4 settings :
- Method : Manual
- Address: 172.168.190.2
- Netmask : 255.255.0.0
- Gateway : 172.168.190.100
- DNS Servers : 172.168.190.2 , 172.168.190.1
45. Click on enp0s8 and make the following changes :
General : Select Automatically connect to this network when it is available
IPv6 Settings : Ignore
Ipv4 settings :
- Method : Manual
- Address: 192.168.190.2
- Netmask : 255.255.255.0
- Gateway : 172.168.190.100
- DNS Servers : 172.168.190.2 , 172.168.190.1
46. Click Done and Begin Installation and provide the Root Password
47. Click on Reboot on each node once the installation completes
48. Accept the License on each node
49. Double check whether network is connected or not on each node and then Click on Finish Configuration
50. Select English –> Next–> Timezone –> Skip –> create an user
51. Once your both nodes are ready, configure Vmbox tools
Click Run
Provide the password and once VMbox Tools are successfully, reboot the systems again
52. Disable firewall in both nodes
systemctl stop firewalld
systemctl disable firewalld
53. Disable SELINUX in /etc/selinux/config
54. If you havent selected all packages while installation of your server, go to your Packages folder in your media and install the below packages :
rpm -ivh libaio-0.3.109-13.el7.x86_64.rpm
rpm -ivh libaio-0.3.109-13.el7.i686.rpm
rpm -ivh libaio-devel-0.3.109-13.el7.i686.rpm
rpm -ivh libaio-devel-0.3.109-13.el7.x86_64.rpm
rpm -ivh ksh-20120801-26.el7.x86_64.rpm
rpm -ivh glibc-devel-2.17-157.el7.x86_64.rpm
rpm -ivh glibc-2.17-157.el7.x86_64.rpm
rpm -ivh glibc-common-2.17-157.el7.x86_64.rpm
rpm -ivh bind-9.9.4-37.el7.x86_64.rpm
rpm -ivh bind-chroot-9.9.4-37.el7.x86_64.rpm
rpm -ivh bc
rpm -ivh binutils
rpm -ivh compat-libcap1
rpm -ivh compat-libstdc++-33
rpm -ivh elfutils-libelf
rpm -ivh elfutils-libelf-devel
rpm -ivh fontconfig-devel
rpm -ivh glibc
rpm -ivh glibc-devel
rpm -ivh ksh
rpm -ivh libaio
rpm -ivh libaio-devel
rpm -ivh libXrender
rpm -ivh libXrender-devel
rpm -ivh libX11
rpm -ivh libXau
rpm -ivh libXi
rpm -ivh libXtst
rpm -ivh libgcc
rpm -ivh librdmacm-devel
rpm -ivh libstdc++
rpm -ivh libstdc++-devel
rpm -ivh libxcb
rpm -ivh make
rpm -ivh net-tools # Clusterware
rpm -ivh nfs-utils # ACFS
rpm -ivh python # ACFS
rpm -ivh python-configshell # ACFS
rpm -ivh python-rtslib # ACFS
rpm -ivh python-six # ACFS
rpm -ivh targetcli # ACFS
rpm -ivh smartmontools
rpm -ivh sysstat
rpm -ivh unixODBC
55. Download the updated RPM’s and RPM’s not available with your iso and install them in both nodes
- kmod-20-21.el7.x86_64.rpm
- kmod-libs-20-21.el7.x86_64.rpm
- oracleasmlib-2.0.12-1.el7.x86_64.rpm
- kmod-oracleasm-2.0.8-17.el7.x86_64
- oracleasmlib-2.0.12-1.el7.x86_64.rpm
- oracleasm-support-2.1.8-3.el7.x86_64.rpm
56. If your updated version doesnt install, due to lower version being installed try any of the below 2 commands to uninstall the package and install the new updated one
yum remove <package_name>
or
rpm -e <package_name> – -nodeps
57. Create the groups, oracle user, modify oracle user with primary and secondary groups in both nodes :
groupadd asmoper
groupadd asmadmin
groupadd asmdba
groupadd oper
groupadd oinstall
groupadd dba
useradd oracle
passwd oracle
usermod -g oinstall oracle
usermod -G dba,asmoper,asmadmin,asmdba,oper,dba oracle
58. Provide details in /etc/hosts
vi /etc/hosts
# Public
172.168.190.1 jack.infraxpertzz.com jack
172.168.190.2 jill.infraxpertzz.com jill
# Private
192.168.190.1 jack-priv.infraxpertzz.com jack-priv
192.168.190.2 jill-priv.infraxpertzz.com jill-priv
# Virtual
172.168.190.3 jack-vip.infraxpertzz.com jack-vip
172.168.190.4 jill-vip.infraxpertzz.com jill-vip
# SCAN
172.168.190.10 crsprod-scan.infraxpertzz.com crsprod-scan
172.168.190.20 crsprod-scan.infraxpertzz.com crsprod-scan
172.168.190.30 crsprod-scan.infraxpertzz.com crsprod-scan
59. Check whether your Node 2 IPs are pingable from Node 1 and vice versa
60. Configure DNS in both Nodes.
I am posting the DNS entries of Node 1. Please make sure to change the entries wherever those are highlighted in Bold and RED with Node 2 IP address and DNS address
a. Edit the Configuration of Name Server. Please take a backup of your current Name server before proceeding
vi /etc/named.conf
//
// named.conf
//
// Provided by Red Hat bind package to configure the ISC BIND named(8) DNS
// server as a caching only nameserver (as a localhost DNS resolver only).
//
// See /usr/share/doc/bind*/sample/ for example named configuration files.
//
// See the BIND Administrator’s Reference Manual (ARM) for details about the
// configuration located in /usr/share/doc/bind-{version}/Bv9ARM.html
options {
listen-on port 53 { 127.0.0.1;172.168.190.1; };
listen-on-v6 port 53 { ::1; };
directory “/var/named”;
dump-file “/var/named/data/cache_dump.db”;
statistics-file “/var/named/data/named_stats.txt”;
memstatistics-file “/var/named/data/named_mem_stats.txt”;
allow-query { localhost;172.168.190.0/16; };
/*
– If you are building an AUTHORITATIVE DNS server, do NOT enable recursion.
– If you are building a RECURSIVE (caching) DNS server, you need to enable recursion.
– If your recursive DNS server has a public IP address, you MUST enable access control to limit queries to your legitimate users. Failing to do so will cause your server to become part of large scale DNS amplification attacks. Implementing BCP38 within your network would greatly reduce such attack surface
*/
recursion yes;
dnssec-enable yes;
dnssec-validation yes;
/* Path to ISC DLV key */
bindkeys-file “/etc/named.iscdlv.key”;
managed-keys-directory “/var/named/dynamic”;
pid-file “/run/named/named.pid”;
session-keyfile “/run/named/session.key”;
};
logging {
channel default_debug {
file “data/named.run”;
severity dynamic;
};
};
zone “.” IN {
type hint;
file “/dev/null”;
};
zone “infraxpertzz.com” IN {
type master;
file “infraxpertzz.com.fwd.zone”;
allow-update { none; };
};
zone “localhost” IN {
type master;
file “localhost.fwd.zone”;
allow-update { none; };
};
zone “190.168.172.in-addr.arpa.” IN {
type master;
file “172.168.190.rev.zone”;
allow-update { none; };
};
zone “0.0.127.in-addr.arpa” IN {
type master;
file “localhost.rev.zone”;
allow-update { none; };
};
include “/etc/named.rfc1912.zones”;
include “/etc/named.root.key”;
*****************
b. Hash out the local host in “/etc/named.rfc1912.zones”
*****************
c. Configure the Local Host Reverse Zone
vi /var/named/localhost.rev.zone
$TTL 1D
@ IN SOA localhost. root.localhost (
2011071500 ; serial
8H ; refresh
4H ; retry
1W ; expiry
1D ) ; minimum
IN NS localhost.
1 IN PTR localhost.
***********************
d. Configure Local Host Forward Zone
vi /var/named/localhost.fwd.zone
$TTL 1D
@ IN SOA @ root (
2011071500 ; serial
8H ; refresh
4H ; retry
1W ; expiry
1D ) ; minimum
IN NS @
IN A 127.0.0.1
**************************
e. Configure Reverse Zone File
vi /var/named/172.168.190.rev.zone
$TTL 1D
@ IN SOA jack.infraxpertzz.com. root.localhost (
2011071500 ; serial
8H ; refresh
4H ; retry
1W ; expiry
1D ) ; minimum
@ IN NS jack.infraxpertzz.com.
1 IN PTR jack.infraxpertzz.com.
2 IN PTR jill.infraxpertzz.com.
3 IN PTR jack-vip.infraxpertzz.com.
4 IN PTR jill-vip.infraxpertzz.com.
**************************************
f. Configure Forward Zone file
vi /var/named/infraxpertzz.com.fwd.zone
$TTL 1D
@ IN SOA jack.infraxpertzz.com. root.localhost (
2011071500 ; serial
8H ; refresh
4H ; retry
1W ; expiry
1D ) ; minimum
@ IN NS jack.infraxpertzz.com.
localhost IN A 127.0.0.1
jack IN A 172.168.190.1
jill IN A 172.168.190.2
jack-vip IN A 172.168.190.3
jill-vip IN A 172.168.190.4
crsprod-scan IN A 172.168.190.10
IN A 172.168.190.20
IN A 172.168.190.30
*************************************
61. Deploy iptables Rules to allow DNS service in both nodes
Add the iptables rules
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -m state –state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -p tcp -m state –state NEW -m tcp –dport 22 -j ACCEPT
iptables -A INPUT -p tcp -m state –state NEW -m tcp –dport 53 -j ACCEPT
iptables -A INPUT -p udp -m state –state NEW -m udp –dport 53 -j ACCEPT
iptables -A INPUT -j DROP
62. Edit the entries of /etc/resolv.conf keeping the Primary Name server at top for respective Server. The below example is from Node 1
# Generated by NetworkManager
search infraxpertzz.com
nameserver 172.168.190.1
nameserver 172.168.190.2
options attempts: 3
options timeout: 3
63. Enable and start Named service in both nodes
systemctl enable named
systemctl start named
64. Chrony NTP Configuration in both nodes
systemctl enable chronyd.service
systemctl restart chronyd.service
chronyc -a ‘burst 4/4’
chronyc -a makestep
65. Append the entry in /etc/sysconfig/network in both nodes :
NOZEROCONF=yes
66. Now nslookup your scan name from both nodes and check whether it works fine or not
67. Add or amend the following lines to the “/etc/sysctl.conf” file:
fs.file-max = 6815744
kernel.sem = 250 32000 100 128
kernel.shmmni = 4096
kernel.shmall = 1073741824
kernel.shmmax = 4398046511104
kernel.panic_on_oops = 1
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
net.ipv4.conf.all.rp_filter = 2
net.ipv4.conf.default.rp_filter = 2
fs.aio-max-nr = 1048576
net.ipv4.ip_local_port_range = 9000 65500
Run the following command to change the current kernel parameters.
/sbin/sysctl -p
68. Add the following lines to the /etc/security/limits.conf file:
oracle soft nofile 1024
oracle hard nofile 65536
oracle soft nproc 16384
oracle hard nproc 16384
oracle soft stack 10240
oracle hard stack 32768
oracle hard memlock 134217728
oracle soft memlock 134217728
This marks the end of our article on creation and configuration of servers for 2 Node RAC.
Stay tuned for the next part, where we will configure the Grid Infrastructure and Create RAC Database