Oracle RAC VIP and SCAN IPs - oracle

I've read the Oracle RAC documentation a couple of times but SCAN and VIP are still confusing me. Can someone help me understand how this needs to be configured technically so that I can explain it my network admin.
VIP in Oracle RAC, should each VIP bind to the node or just require a DNS A record without allocating it to node1 or node2 and an entry in the host file?
I know while performing Grid cluster installation Oracle will bind the VIP automatically, but should this be part of DNS assigned to one of the nodes or should it be free and unassigned?
Oracle SCAN IPs need to be created in DNS record; is this an A record to 3 IPs with reverse lookup or round robin way and this should not be part of hosts file?
I need to explain this to my network admin to add it on the DNS server.

First, VIPs:
A VIP is a Virtual IP address, and should be defined in DNS and not assigned to any host or interface. When you install GRID/ASM home, you'll specify the VIP names that were assigned in DNS. When Oracle Clusterware starts up, it will assign a VIP to each node in the cluster. The idea is, if a node goes down (crashes), clusterware can immediately re-assign that VIP to a new (surviving) node. This way, you avoid TCP timeout issues.
Next, SCAN:
A SCAN (Single Client Access Name) is a special case of VIP. The SCAN should also be defined in DNS, and not assigned to any host or interface. There should be three IPs associated with the SCAN name in DNS, and the DNS entry should be defined so that one of the three IPs is returned each time DNS is queried, in a round robin fashion.
At clusterware startup time, each of the three VIPs that make up the SCAN will be assigned to a different node in the cluster. (Except in the special case of a two node cluster, one of the nodes wil have a 2 SCAN VIPs assigned to it.) The point of the SCAN, is that no matter how many nodes are added to or removed from the cluster, all the Net Service Name definitions in your tnsnames.ora (or LDAP equivalent) will not need to ever change, because they all refer to the SCAN, which doesn't change, regardless of how many node additions or drops are made to the cluster.
For example, in the three node cluster, you may have:
Physical and virtual hostnames/IPs assigned as follows:
Hostname Physical IP Virtual hostnmae Virtual IP
rac1 10.1.1.1 rac1-vip 10.1.1.4
rac2 10.1.1.2 rac2-vip 10.1.1.5
rac3 10.1.1.3 rac3-vip 10.1.1.6
Additionally, you may have the SCAN defined as:
rac-scan with three IPs, 10.1.1.7, 10.1.1.8, 10.1.1.9. Again, the DNS definition would be defined so those IPs are served up in a round robin order.
Note that the SCAN VIPs, Host VIPs, and the Physical IPs are all in the same subnet.
Finally, though you didn't ask about it, to complete the picture, you'd also need one private, non-routable IP assigned per host, and that IP would be associated with the private interconnect. So, you may have something like:
rac1-priv 172.16.1.1
rac2-priv 172.16.1.2
rac3-priv 172.16.1.3
Note that the '-priv' addresses should not be in DNS, only in the /etc/hosts file of each host in the RAC cluster. (They are private, non-routable, and only clusterware will ever know about or use those addresses, so adding to DNS doesn't make sense.)
Note also, that '-priv' and physical IP/hostname definitions should go in /etc/hosts, and the physical IPs and VIPs should be in DNS. So, physical IPs in both DNS and /etc/hosts, VIPs only in DNS, '-priv' addresses only in /etc/hosts.

not entirely sure what you mean for this, i have each VIP address created in DNS as A records assigned to the hosts, and also record them in the hosts file as well.
in answer to 2, you are correct, the SCAN IPs should not be in the hosts file. And yes 3 "A" records with reserve lookup will be enough (at least that's what has worked for me).
these are my iptables entries
Oracle ports
Allow access from other oracle RAC hosts
-A INPUT -m state --state NEW -p tcp -m iprange --src-range 172.28.1.90-172.28.1.97 -j ACCEPT
-A INPUT -m state --state NEW -p tcp -m iprange --src-range 172.28.97.91-172.28.97.93 -j ACCEPT
-A INPUT -m state --state NEW -p tcp -m iprange --src-range 192.168.28.91-192.168.28.93 -j ACCEPT
Allow Multicast
-A INPUT -m state --state NEW -p udp -m iprange --src-range 172.28.1.90-172.28.1.97 -j ACCEPT
-A INPUT -m state --state NEW -p udp -m iprange --src-range 172.28.97.91-172.28.97.93 -j ACCEPT
-A INPUT -m state --state NEW -p udp -m iprange --src-range 192.168.28.91-192.168.28.93 -j ACCEPT
Allow multicast
-A INPUT -m pkttype --pkt-type multicast -j ACCEPT
-A INPUT -s 224.0.0.0/24 -j ACCEPT
-A INPUT -s 230.0.1.0/24 -j ACCEPT
I also needed to get our systems admin to give permissions a the firewall level to allow my nodes, their vips and the scan ips to connect via port 1521
hope this helps

Related

What is this command prompt code in MacOS?

I know this code works on Windows, but how do I get workable code executable on MacOS?
netsh -c interface ipv4 add neighbors "(connection name)" "(router
address)" "(mac address)" store=persistent
Any suggestions please...
try typing "man arp" into the terminal. Something like this might work :
arp -s hostname ether_addr
Create an ARP entry for the host called hostname with the Ethernet address ether_addr. The Ethernet address is
given as six hex bytes separated by colons. The entry will be permanent unless the word temp is given in the
command. If the word pub is given, the entry will be ``published''; i.e., this system will act as an ARP server,
responding to requests for hostname even though the host address is not its own. In this case the ether_addr can
be given as auto in which case the interfaces on this host will be examined, and if one of them is found to
occupy the same subnet, its Ethernet address will be used. If the only keyword is also specified, this will cre-
ate a ``published (proxy only)'' entry. This type of entry is created automatically if arp detects that a rout-
ing table entry for hostname already exists.

How do I pick a random unassigned port on localhost

I am building an application and want to pick a random unassigned port above 1024 to host that application. How can I do that?
"Above 1024" you are indirectly referencing to know what ports can be used by server for listening.
Approx way :
if yes netstat -pant | grep "portno" if there is no output there is no server listening to this port and can be used.
If you try to bind to a used port you will get notable to bind error.

amazon ec2, cannot ping internal host

In amazon ec2, I have 2 instances in a placement group. First node is 172.31.12.76/20, second, 172.31.12.77/20 I can ssh both nodes from my pc. They share the same security group that has got these 2 rules:
Inbound rules:
Type Protocol Port Range Source
SSH TCP 22 0.0.0.0/0
All IMCP All N/A 0.0.0.0/0
(no outbound rules)
Both nodes see to each other in L2:
root#ip-172-31-12-76:~# arp
[...]
ip-172-31-12-77.eu-west ether 0a:ad:5e:e4:12:de C eth0
[...]
root#ip-172-31-12-77:~# arp
[...]
ip-172-31-12-76.eu-west ether 0a:34:a1:17:57:28 C eth0
[...]
iptables are empty on both nodes.
But ping does not work between each other
I have already checked a previous post:
EC2 instances not responding to internal ping
but it does not address the issue. It looks like there are no other similar posts.
Any idea? Thank you very much!
I got the answer; I need to also allow outbound icmp on each host in order to be able to ping both external and internal IPs.

parse /etc/hosts for ip and hostname with puppet, and transpose into /etc/sysconfig/network-scripts/ifcfg-eth0

I am kicking machines with cobbler to install redhat 6 and also put in hostname.
I am managing my machines with /etc/hosts file right now (may switch to dns in the future) and would like to have puppet (or anything that will work) parse through /etc/hosts and find my ip address and hostname (based on the hostname supplied by cobbler at kickstart). The trick is that some machines have multiple IP's and hostnames in the /etc/hosts file, like such:
# Maintenance Network
192.168.80.192 testsrv01-maint
192.168.80.193 testsrv02-maint
192.168.80.194 testsrv03-maint
# Lights Out Network
192.168.120.192 testsrv01-ilo
192.168.120.193 testsrv02-ilo
192.168.120.194 testsrv03-ilo
# Primary Data Network
192.168.150.192 testsrv01-pri
192.168.150.193 testsrv02-pri
192.168.150.194 testsrv03-pri
# Secondary Data Network
192.168.200.192 testsrv01-sec
192.168.200.193 testsrv02-sec
192.168.200.194 testsrv03-sec
I need to capture each ip and hostname pair (in a line) and transpose into /etc/sysconfig/network-scripts/ifcfg-eth* (eth1, eth2, eth3, ...). Puppet will need to create as many ifcfg-eth* files as there are matches in /etc/hosts for the hostname.
I just need puppet to append the $IP and $hostname to the ifcfg-eth file, the rest of the content is common.
So how would I get 4x ifcfg-eth files for 'testsrv01', with puppet?
Puppet is a very ill fit for this task. This calls for a script, which Puppet does not support. Puppet allows you to declare a piece of machine state that can portably enforced on different platforms.
The task is simple enough for a Shell or Perl Script.
With Puppet, it would entail the following scripts
writing custom facts to retrieve each address / hostname pair
devising a defined type to render such a pair into an ethX file
You'd possibly even need to generate the respective manifest, so that the appropriate interface index is chosen for each address.
All things considered, you would not make use of Puppet's strengths and suffer some of its weaknesses.

Remove EC2's entry from resolv.conf

I have private DNS servers and I want to write them to resolv.conf with resolvconf on Debian on AWS/EC2.
There is a problem in the order of nameserver entries.
In my resolv.conf, EC2's default nameserver is always written at first line like so:
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 172.16.0.23
nameserver 10.0.1.185
nameserver 10.100.0.130
search ap-northeast-1.compute.internal
172.16.0.23 is EC2's default nameserver and others are mine.
How to remove EC2 entry? Or, how to move EC2 entry to third?
Here I have an interface file:
% ls -l /etc/resolvconf/run/interface/
-rw-r--r-- 1 root root 62 Jun 7 23:35 eth0
It seems that the file eth0 is automatically generated by dhcp so can't remove it permanently.
% cat /etc/resolvconf/run/interface/eth0
search ap-northeast-1.compute.internal
nameserver 172.16.0.23
My private DNS entry is here:
% cat /etc/resolvconf/resolv.conf.d/base
nameserver 10.0.1.185
nameserver 10.100.0.130
Please help.
I think I just solved a very similar problem. I was bothered by Amazon EC2's crappy internal DNS servers so I wanted to run a local caching dnsmasq daemon and use that in /etc/resolv.conf. At first I just did echo nameserver 127.0.0.1 > /etc/resolv.conf but then I realized that my change would eventually be overwritten by the DHCP client after a reboot or DHCP lease refresh.
What I've now done instead is to edit /etc/dhcp3/dhclient.conf and uncomment the line prepend domain-name-servers 127.0.0.1;. You should be able to use the prepend directive in a very similar way.
Update: These instructions are based on Ubuntu Linux but I imagine the general concept applies on other systems as well, even other DHCP clients must have similar configuration options.
I'm approaching this problem from the other direction (wanting the internal nameservers), much of what I've learned may be of interest.
There are several options to control name resolution in the VPC management console.
VPC -> DHCP option sets -> Create dhcp option set
You can specify your own name servers there.
http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_DHCP_Options.html
Be sure to attach this dhcp option set to your VPC to get it to take effect.
Alternatively (I found this out by mistake) local dns servers are not set if the following settings are disabled in VPC settings:
DnsHostnames
and
DnsSupport
http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-dns.html
Settings can also be overridden locally (which you'll notice if you move instances between vpcs). /etc/dhcp/dhclient.conf
The following line might be of interest:
prepend domain-name-servers
Changes, of course, take effect on dhclient start.
How do I assign a static DNS server to a private Amazon EC2 instance running Ubuntu, RHEL, or Amazon Linux?
Short Description
Default behavior for an EC2 instance associated with a virtual private cloud (VPC) is to request a DNS server address at startup using the Dynamic Host Configuration Protocol (DHCP). The VPC responds to DHCP requests with the address of an internal DNS server. The DNS server addresses returned in the DHCP response are written to the local /etc/resolv.conf file and are used for DNS name resolution requests. Any manual modifications to the resolv.conf file are overwritten when the instance is restarted.
Resolution
To configure an EC2 instance running Linux to use static DNS server entries, use a text editor such as vim to edit the file /etc/dhcp/dhclient.conf and add the following line to the end of the file:
supersede domain-name-servers xxx.xxx.xxx.xxx, xxx.xxx.xxx.xxx;
Ubuntu - dhclient.conf - DHCP client configuration file 
The supersede statement
supersede [ option declaration ] ;
If for some option the client should always use a locally-configured value or values
rather than whatever is supplied by the server, these values can be defined in the
supersede statement.
The prepend statement
prepend [ option declaration ] ;
If for some set of options the client should use a value you supply, and then use the
values supplied by the server, if any, these values can be defined in the prepend
statement. The prepend statement can only be used for options which allow more than one
value to be given. This restriction is not enforced - if you ignore it, the behaviour
will be unpredictable.
The append statement
append [ option declaration ] ;
If for some set of options the client should first use the values supplied by the server,
if any, and then use values you supply, these values can be defined in the append
statement. The append statement can only be used for options which allow more than one
value to be given. This restriction is not enforced - if you ignore it, the behaviour
will be unpredictable.
In here someone come with solution that basically replaces the file on boot using rc.local
https://forums.aws.amazon.com/thread.jspa?threadID=74497
Edit /etc/sysconfig/network-scripts/ifcfg-eth0 to say PEERDNS=no
Create a file called /etc/resolv.backup with what you want
Add the following 2 lines to /etc/rc.local:
rm -f /etc/resolv.conf cp /etc/resolv.backup /etc/resolv.conf
This is what we are doing for our servers in the environment.
interface "eth0"
{
prepend domain-name-servers 10.x.x.x;
supersede host-name "{Hostname}";
append domain-search "domain";
supersede domain-name "DOMAIN";
}
Hope this helps.
The following worked in a Debian stretch on AWS EC2.
Just create /etc/dhcp/dhclient-enter-hooks.d/nodnsupdate:
#!/bin/sh
make_resolv_conf(){
:
}
Then you can modify /etc/resolv.conf and it will persist your changes across restarts.
Setup in crontab as
#reboot cp -r /home/.../resolv.conf /etc/resolv.conf

Resources