What gems do you recommend to use for this kind of automation? - ruby

I have to create a script to manage maintenance pages server for my hosting company.
I will need to do a CLI interface that would act like this (example scenario) :
(here, let's suppose that mcli is the name of the script, 1.1.1.1 the original server address (that host the website, www.exemple.com)
Here I just create the loopback interface on the maintenance server with the original ip address and create the nginx site-specific config file in sites-enabled
$ mcli register www.exemple.com 1.1.1.1
[DEBUG] Adding IP 1.1.1.1 to new loopback interface lo:001001001001
[WARNING] No root directory specified, setting default maintenance page.
[DEBUG] Registering www.exemple.com maintenance page and reloading Nginx: OK
Then when I want to enable the maintenance page and completely shutdown the website:
$ mcli maintenance www.exemple.com
[DEBUG] Connecting to router with SSH: OK
[DEBUG] Setting new route to 1.1.1.1 to maintenance server: OK
[DEBUG] Writing configuration: Ok
Then removing the maintenance page:
$ mcli nomaintenance www.exemple.com
[DEBUG] Connecting to router with SSH: OK
[DEBUG] Removing route to 1.1.1.1: Ok
[DEBUG] Writing configuration: Ok
And I would need a function to see the actual states of the websites
$ mcli list
+------------------+-----------------+------------------+
| Site Name | Server I.P | Maintenance mode |
+------------------+-----------------+------------------+
| www.example.com | 1.1.1.1 | Enabled |
| www.example.org | 1.1.1.2 | Disabled |
+------------------+-----------------+------------------+
$ mcli show www.example.org
Site Name: www.example.org
Server I.P: 1.1.1.1
Maintenance Mode: Enabled
Root Directory : /var/www/maintenance/default/
But I never did this kind of scripting with Ruby. What gems do you recommend for this kind of things ? For command line parsing ? Column/Colorized output ? SSH connection (needed to connect to cisco routers)
Do you recommend me to use a local database (sqlite) to store meta datas (Stages changes, actual states) or do you recommend me to compute on the fly by analyzing nginx/interfaces configuration files and using syslog for monitoring changes done with this script ?
This script will be used at first time for a massive datacenter physical migration, and next for standard usages for scheduled downtimes.
Thank you

First of all, I'd recommend you get a copy of Build awesome command-line applications in Ruby.
That said, you might want to check
GLI command line parsing like git
OptionParser command line parsing
Personally, I'd go for the SQLite approach for storing data, but I'm biased (having a strong SQL background).

Thor is a good gem for handling CLI options. It allows this type of organization in your script:
class Maintenance < Thor
desc "maintenance", "put up maintenance page"
method_option :switch, :aliases => '-s', :type => 'string'
#The method name is the name of the task that would be run => mcli maintenance
def maintenance
#do stuff
end
no_tasks do
#methods that you don't want cli tasks for go here
end
end
Maintenance.start
I don't really have any good suggestions for column/colorized output though.
I definitely recommend using some kind of a database to store states though. Maybe not sqlite, I would probably opt for maybe a redis database that stores key/value pairs with the information you are looking for.

We have similar task. I use next architecture
Small application (C) what generate config file
Add nginx init.d script new switch update_clusters. This script will restart nginx only if config file is changed
update_clusters() {
${CONF_GEN} --outfile=/tmp/nginx_clusters.conf
RETVAL=$?
if [[ "$RETVAL" != "0" ]]; then
return 5
fi
if ! diff ${CLUSTER_CONF_FILE} /tmp/nginx_clusters.conf > /dev/null; then
echo "Cluster configuration changed. Reload service"
mv -f /tmp/nginx_clusters.conf ${CLUSTER_CONF_FILE}
reload
fi
}
Set of bash scripts to add records to database.
Web console to add/modify/delete records in database (extjs+nginx module)

Related

How can I query in a bash script which of the haproxy load balancer hosts is currently the primary and which is the secondary?

I want to be able to determine which LB is primary and which is secondary from a bash script running on both load balancers.
Background is: For the renewal of a Letsencrypt certificate on a HAproxy load balancer pair, where the service IP is usually bound to the master, it would be necessary to determine which server is the master (has the service IP bound) and which is only a secondary backup (without web access via port :80 and port:443)
If you follow this guide by Sebastian Schrader https://serverfault.com/a/871783, the following procedure will help to determine the master and backup:
IFS="/"
# /org/keepalived/Vrrp1/Instance/ens192/151/IPv4
vrrpInstance=$(/usr/bin/busctl tree | grep keepalived | grep IPv4)
set $vrrpInstance
#151
vrrpRouterID=$7
# (us) 2 "Master" or "Backup"
vrrpProp=$(/usr/bin/busctl get-property org.keepalived.Vrrp1 /org/keepalived/Vrrp1/Instance/ens192/"${vrrpRouterID}"/IPv4 org.keepalived.Vrrp1.Instance State)
# Master or Backup
vrrpStatus=$(echo ${vrrpProp} | cut -c 9-14)
unset IFS

Perforce - how to prevent "p4 client" from creating a client when the template form is not saved?

The Perforce documentation for p4 client <no args> states:
The p4 client command puts the client spec into a temporary file and
invokes the editor configured by the environment variable P4EDITOR.
For new workspaces, the client name defaults to the P4CLIENT
environment variable, if set, or to the current host name. Saving the
file creates or modifies the client spec.
What I am seeing on our network is that the client is created no matter what, even when I exit without saving.
Ex.
[cad_test_user#sws-cab9-0 ~]$ pwd
/home/cad_test_user
[cad_test_user#sws-cab9-0 ~]$ env | grep P4
P4EDITOR=
P4PORT=tcp:p4p:1666
P4DIFF=tkdiff
P4CONFIG=.p4config
P4IGNORE=.ignore
P4USER=cad_test_user
[cad_test_user#sws-cab9-0 ~]$ p4 clients | grep sws-cab9-0
[cad_test_user#sws-cab9-0 ~]$ p4 client
Client: sws-cab9-0
Owner: cad_test_user
Host: sws-cab9-0.aus5.mythic-ai.com
Client sws-cab9-0 saved.
Root: /home/cad_test_user
Options: noallwrite noclobber nocompress unlocked nomodtime normdir
SubmitOptions: submitunchanged
LineEnd: local
View:
<quit without save>
Client sws-cab9-0 saved.
[cad_test_user#sws-cab9-0 ~]$ p4 clients | grep sws-cab9-0
Client sws-cab9-0 2021/04/06 root /home/cad_test_user 'Created by cad_test_user. '
Now as another user outside of a .p4config hierarhchy, I get an unexpected value for %clientroot%:
[cad_test_user#sws-cab9-0 /]$ p4 -F %clientRoot% -ztag info
/home/cad_test_user
I am wondering if there is something wrong with our default settings; why is the client created and saved even without a write? Ideally, I'd want to manage the default specification to some degree, like:
synthesize the client name so that it is never the hostname, like c:$USER:foo
Not have a "Host:"
define the "Root:" to be somewhere personal
Not create the client unless the user does a write-quit!
Thanks for your answers!
Set up a trigger (a form-save trigger on the client form) that rejects a client which doesn't meet your criteria. It's hard to enforce #4 directly, but as long as at least one of your other criteria is something that requires the form to be edited, it's handled well enough indirectly.
Note that you can pair your form-save trigger with a form-out trigger that modifies the default client form -- you could for example replace Root with an obviously invalid field like --ENTER SOMETHING PERSONALIZED HERE-- and then make sure your form-save trigger rejects it. The Perforce sys admin guide has some nice simple example triggers, one of which demonstrates customizing client spec defaults: https://www.perforce.com/manuals/p4sag/Content/P4SAG/scripting.triggers.forms.out.html
On your criteria #2, I would recommend against this unless you're in an environment where it's commonplace for multiple host machines to share a single filesystem. The default Host guardrails are there to keep you from confusing yourself (and possibly losing data) by reusing a client spec in ways that throw the workspace state out of whack.

parse /etc/hosts for ip and hostname with puppet, and transpose into /etc/sysconfig/network-scripts/ifcfg-eth0

I am kicking machines with cobbler to install redhat 6 and also put in hostname.
I am managing my machines with /etc/hosts file right now (may switch to dns in the future) and would like to have puppet (or anything that will work) parse through /etc/hosts and find my ip address and hostname (based on the hostname supplied by cobbler at kickstart). The trick is that some machines have multiple IP's and hostnames in the /etc/hosts file, like such:
# Maintenance Network
192.168.80.192 testsrv01-maint
192.168.80.193 testsrv02-maint
192.168.80.194 testsrv03-maint
# Lights Out Network
192.168.120.192 testsrv01-ilo
192.168.120.193 testsrv02-ilo
192.168.120.194 testsrv03-ilo
# Primary Data Network
192.168.150.192 testsrv01-pri
192.168.150.193 testsrv02-pri
192.168.150.194 testsrv03-pri
# Secondary Data Network
192.168.200.192 testsrv01-sec
192.168.200.193 testsrv02-sec
192.168.200.194 testsrv03-sec
I need to capture each ip and hostname pair (in a line) and transpose into /etc/sysconfig/network-scripts/ifcfg-eth* (eth1, eth2, eth3, ...). Puppet will need to create as many ifcfg-eth* files as there are matches in /etc/hosts for the hostname.
I just need puppet to append the $IP and $hostname to the ifcfg-eth file, the rest of the content is common.
So how would I get 4x ifcfg-eth files for 'testsrv01', with puppet?
Puppet is a very ill fit for this task. This calls for a script, which Puppet does not support. Puppet allows you to declare a piece of machine state that can portably enforced on different platforms.
The task is simple enough for a Shell or Perl Script.
With Puppet, it would entail the following scripts
writing custom facts to retrieve each address / hostname pair
devising a defined type to render such a pair into an ethX file
You'd possibly even need to generate the respective manifest, so that the appropriate interface index is chosen for each address.
All things considered, you would not make use of Puppet's strengths and suffer some of its weaknesses.

Remove EC2's entry from resolv.conf

I have private DNS servers and I want to write them to resolv.conf with resolvconf on Debian on AWS/EC2.
There is a problem in the order of nameserver entries.
In my resolv.conf, EC2's default nameserver is always written at first line like so:
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 172.16.0.23
nameserver 10.0.1.185
nameserver 10.100.0.130
search ap-northeast-1.compute.internal
172.16.0.23 is EC2's default nameserver and others are mine.
How to remove EC2 entry? Or, how to move EC2 entry to third?
Here I have an interface file:
% ls -l /etc/resolvconf/run/interface/
-rw-r--r-- 1 root root 62 Jun 7 23:35 eth0
It seems that the file eth0 is automatically generated by dhcp so can't remove it permanently.
% cat /etc/resolvconf/run/interface/eth0
search ap-northeast-1.compute.internal
nameserver 172.16.0.23
My private DNS entry is here:
% cat /etc/resolvconf/resolv.conf.d/base
nameserver 10.0.1.185
nameserver 10.100.0.130
Please help.
I think I just solved a very similar problem. I was bothered by Amazon EC2's crappy internal DNS servers so I wanted to run a local caching dnsmasq daemon and use that in /etc/resolv.conf. At first I just did echo nameserver 127.0.0.1 > /etc/resolv.conf but then I realized that my change would eventually be overwritten by the DHCP client after a reboot or DHCP lease refresh.
What I've now done instead is to edit /etc/dhcp3/dhclient.conf and uncomment the line prepend domain-name-servers 127.0.0.1;. You should be able to use the prepend directive in a very similar way.
Update: These instructions are based on Ubuntu Linux but I imagine the general concept applies on other systems as well, even other DHCP clients must have similar configuration options.
I'm approaching this problem from the other direction (wanting the internal nameservers), much of what I've learned may be of interest.
There are several options to control name resolution in the VPC management console.
VPC -> DHCP option sets -> Create dhcp option set
You can specify your own name servers there.
http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_DHCP_Options.html
Be sure to attach this dhcp option set to your VPC to get it to take effect.
Alternatively (I found this out by mistake) local dns servers are not set if the following settings are disabled in VPC settings:
DnsHostnames
and
DnsSupport
http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-dns.html
Settings can also be overridden locally (which you'll notice if you move instances between vpcs). /etc/dhcp/dhclient.conf
The following line might be of interest:
prepend domain-name-servers
Changes, of course, take effect on dhclient start.
How do I assign a static DNS server to a private Amazon EC2 instance running Ubuntu, RHEL, or Amazon Linux?
Short Description
Default behavior for an EC2 instance associated with a virtual private cloud (VPC) is to request a DNS server address at startup using the Dynamic Host Configuration Protocol (DHCP). The VPC responds to DHCP requests with the address of an internal DNS server. The DNS server addresses returned in the DHCP response are written to the local /etc/resolv.conf file and are used for DNS name resolution requests. Any manual modifications to the resolv.conf file are overwritten when the instance is restarted.
Resolution
To configure an EC2 instance running Linux to use static DNS server entries, use a text editor such as vim to edit the file /etc/dhcp/dhclient.conf and add the following line to the end of the file:
supersede domain-name-servers xxx.xxx.xxx.xxx, xxx.xxx.xxx.xxx;
Ubuntu - dhclient.conf - DHCP client configuration file 
The supersede statement
supersede [ option declaration ] ;
If for some option the client should always use a locally-configured value or values
rather than whatever is supplied by the server, these values can be defined in the
supersede statement.
The prepend statement
prepend [ option declaration ] ;
If for some set of options the client should use a value you supply, and then use the
values supplied by the server, if any, these values can be defined in the prepend
statement. The prepend statement can only be used for options which allow more than one
value to be given. This restriction is not enforced - if you ignore it, the behaviour
will be unpredictable.
The append statement
append [ option declaration ] ;
If for some set of options the client should first use the values supplied by the server,
if any, and then use values you supply, these values can be defined in the append
statement. The append statement can only be used for options which allow more than one
value to be given. This restriction is not enforced - if you ignore it, the behaviour
will be unpredictable.
In here someone come with solution that basically replaces the file on boot using rc.local
https://forums.aws.amazon.com/thread.jspa?threadID=74497
Edit /etc/sysconfig/network-scripts/ifcfg-eth0 to say PEERDNS=no
Create a file called /etc/resolv.backup with what you want
Add the following 2 lines to /etc/rc.local:
rm -f /etc/resolv.conf cp /etc/resolv.backup /etc/resolv.conf
This is what we are doing for our servers in the environment.
interface "eth0"
{
prepend domain-name-servers 10.x.x.x;
supersede host-name "{Hostname}";
append domain-search "domain";
supersede domain-name "DOMAIN";
}
Hope this helps.
The following worked in a Debian stretch on AWS EC2.
Just create /etc/dhcp/dhclient-enter-hooks.d/nodnsupdate:
#!/bin/sh
make_resolv_conf(){
:
}
Then you can modify /etc/resolv.conf and it will persist your changes across restarts.
Setup in crontab as
#reboot cp -r /home/.../resolv.conf /etc/resolv.conf

Bash case not properly evaluating value

The Problem
I have a script that has a case statement which I'm expecting to execute based on the value of a variable. The case statement appears to either ignore the value or not properly evaluate it instead dropping to the default.
The Scenario
I pull a specific character out of our server hostnames which indicates where in our environment the server resides. We have six different locations:
Management(m): servers that are part of the infrastructure such as monitoring, email, ticketing, etc
Development(d): servers that are for developing code and application functionality
Test(t): servers that are used for initial testing of the code and application functionality
Implementation(i): servers that the code is pushed to for pre-production evaluation
Production(p): self-explanatory
Services(s): servers that the customer needs to integrate that provide functionality across their project. These are separate from the Management servers in that these are customer servers while Management servers are owned and operated by us.
After pulling the character from the hostname I pass it to a case block. I expect the case block to evaluate the character and add a couple lines of text to our rsyslog.conf file. What is happening instead is that the case block returns the default which does nothing but tell the person building the server to manually configure the entry due to an unrecognized character.
I've tested this manually against a server I recently built and verified that the character I am pulling from the hostname (an 's') is expected and accounted for in the case block.
The Code
# Determine which environment our server resides in
host=$(hostname -s)
env=${host:(-8):1}
OLDFILE=/etc/rsyslog.conf
NEWFILE=/etc/rsyslog.conf.new
# This is the configuration we need on every server regardless of environment
read -d '' common <<- EOF
...
TEXT WHICH IS ADDED TO ALL CONFIG FILES REGARDLESS OF FURTHER CODE EXECUTION
SNIPPED
....
EOF
# If a server is in the Management, Dev or Test environments send logs to lg01
read -d '' lg01conf <<- EOF
# Relay messages to lg01
*.notice ##xxx.xxx.xxx.100
#### END FORWARDING RULE ####
EOF
# If a server is in the Imp, Prod or is a non-affiliated Services zone server send logs to lg02
read -d '' lg02conf <<- EOF
# Relay messages to lg02
*.notice ##xxx.xxx.xxx.101
#### END FORWARDING RULE ####
EOF
# The general rsyslog configuration remains the same; pull it out and write it to a new file
head -n 63 $OLDFILE > $NEWFILE
# Add the common language to our config file
echo "$common" >> $NEWFILE
# Depending on which environment ($env) our server is in, add the appropriate
# remote log server to the configuration with the $common settings.
case $env in
m) echo "$lg01conf" >> $NEWFILE;;
d) echo "$lg01conf" >> $NEWFILE;;
t) echo "$lg01conf" >> $NEWFILE;;
i) echo "$lg02conf" >> $NEWFILE;;
p) echo "$lg02conf" >> $NEWFILE;;
s) echo "$lg02conf" >> $NEWFILE;;
*) echo "Unknown environment; Manually configure"
esac
# Keep a dated backup of the original rsyslog.conf file
cp $OLDFILE $OLDFILE.$(date +%Y%m%d)
# Replace the original rsyslog.conf file with the new version
mv $NEWFILE $OLDFILE
An Aside
I've already determined that I can combine the different groups of code from the case block onto single lines (a total of two) using the | operator. I've listed it in the manner above since this is how it is coded while I'm having issues with it.
I can't see what's wrong with your code. Maybe add another ;; to the default clause. To find the problem add a set -vx as a first line. Will show you lots of debug information.

Resources