remove multiple line from /etc/hosts using augtool - bash

My /etc/hosts look like this:
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.13.11 idm.myhost.com
192.168.13.10 dnsf.myhost.com
192.168.13.11 idm.myhost.com
192.168.13.10 dnsf.myhost.com
192.168.13.11 idm.myhost.com
192.168.13.10 dnsf.myhost.com
192.168.13.10 temp1.myhost.com
I can query duplicated lines by query below:
print /files/etc/hosts/*/canonical["idm.myhost.com"]
or either use regex
defvar mypath /files/etc/hosts
print $mypath/*/*[.=~ regexp("dnsf.myhost.com")]
But I can't remove these matched lines with a simple rm
rm /files/etc/hosts/*/canonical["idm.myhost.com"]
The problem is, augtool just removes the canonical leaf from the node and ipaddr remains. it cause the save function to returns error.
I don't want to use bash script and I try to solve the problem with augeas itself.
Assist, please...
P.S. I couldn't find a complete tutorial or document for this useful tool. I would be thankful is someone can propose a good document. I've searched all the web and just found a few simple examples.

Your expression selects the canonical node instead of the host node. If you want to delete the host nodes, you need to use:
rm /files/etc/hosts/*[canonical="idm.myhost.com"]
This selects any entry under /files/etc/hosts, whose canonical subnode has value idm.myhost.com.
As for documentation, there's a tutorial on the Augeas website, as well as some more docs on the Augeas wiki.
There's also a video intro to Augeas on Youtube.
I started writing a book some 7 years ago, which can still be found online, but never tpok the time to finish it.

Related

Bash commands putting out extra information which results into issues with scripts

Okay, hopefully I can explain this correctly as I have no idea what's causing this or how to resolve this.
For some reason bash commands (on a CentOS 6.x server) are displaying more information than "normally" and that causes issues with certain scripts. I have no clue if there is a name for this, but hopefully someone knows a solution for this.
First example.
Correct / good server:
[root#goodserver ~]# vzctl enter 3567
entered into CT 3567
[root#example /]#
(this is the correct behaviour)
Incorrect / bad server:
[root#badserver /]# vzctl enter 3127
Entering CT
entered into CT 3127
Open /dev/pts/0
[root#example /]#
With the "bad" server it will display more information as usual, like:
Entering CT
Open /dev/pts/0
It's like it parsing extra information on what it's doing.
Ofcourse the above is purely something cosmetic, however with several bash scripts we use, these issues are really issues.
A part of the script we use, uses the following command (there are more, but this is mainly a example of what's wrong):
DOMAIN=`vzctl exec $VEID 'hostname -d'`
The result of the above information is parsed in /etc/named.conf.
On the GOOD server it would be added in the named.conf like this:
zone "example.com" {
type master;
file "example.com";
allow-transfer {
200.190.100.10;
200.190.101.10;
common-allow-transfer;
};
};
The above is correct.
On the BAD server it would be added in the named.conf like this:
zone "Executing command: hostname -d
example.com" {
type master;
file "Executing command: hostname -d
example.com";
allow-transfer {
200.190.100.10;
200.190.101.10;
common-allow-transfer;
};
};
So it's add stuff of the action it does, in this example "Executing command: hostname -d"
Another example here when I run the command on a good server and on the bad server.
Bad server:
[root#bad-server /]# DOMAIN=`vzctl exec 3333 'hostname -d'`
[root#bad-server /]# echo $DOMAIN
Executing command: hostname -d example.com
Good server:
[root#good-server ~]# DOMAIN=`vzctl exec 4444 'hostname -d'`
[root#good-server ~]# echo $DOMAIN
example.com
My knowledge is limited, but I have tried several things checking rsyslog and the grub.conf, but nothing seems out of the ordinary.
I have no clue why it's displaying the extra information.
Probably it's something simple / stupid, but I have been trying to solve this for hours now and I really have no clue...
So any help is really appreciated.
Added information:
Both servers use: kernel.printk = 7 4 1 7
(I don't know if that's useful)
Well (thanks to Aaron for pointing me in the right direction) I finally found the little culprit which was causing all the issues I experienced with this script (which worked for every other server, so no need to change that obviously).
The issues were caused by the VERBOSE leven set in vz.conf (located in /etc/vz/ directory). There is an option in there called "VERBOSE" and in my case it was set to 3.
According to OpenVZ's website it does the following:
Increments logging level up from the default. Can be used multiple times.
Default value is set to the value of VERBOSE parameter in the global
configuration file vz.conf(5), or to 0 if not set by VERBOSE parameter.
After I changed VERBOSE=3 to VERBOSE=0 my script worked fine once again (as it did for every other server). :-)
So a big shoutout to Aaron for pointing me in the right direction. The answer is easy when you know where to look!
Sorry to say, but I am kinda disappointed by ndim's reaction. This is the 2nd time he was very unhelpful and rude in his response after that. He clearly didn't read the issue I posted correctly. Oh well.
I would make sure to properly parse the output of the command. In this case, we are only interested in lines of the form
entered into CT 12345
One way of doing this would be to pipe everything through sed and having sed print only the number when the line looks as above (untested, and I always forget which braces/brackets/parens need a backslash in front of them):
whateverthecommand | sed -n 's/^entered into CT ([0-9]{1,})$/\1/p'

Entries in host file at particular line

I am looking for a script that can add host entries in a windows 7 host files at particular lines or just after the default text in host file.
I came across this thread in your forum here that can add/remove the lines perfectly but it adds the lines at the end of the file not at the top. can you please help with the same?
hey thanks for the link.
Since i have limited exposure to batch files, I wanted to check that method offline first
1) created a txt file dns.txt with some DNS entries
1.2.3.4 abcd.com
2.3.4.5 bcde.com
3.4.5.6 cdef.com
4.5.6.7 defg.com
2) created a batch file with one line code in it:
for /F %%i in (c:\temp\dns.txt) do echo %%i >> "D:\hosts"
3) but when i check the host file, it only has IP entries and not the hostnames against it.
==================
# localhost name resolution is handled within DNS itself.
# 127.0.0.1 localhost
# ::1 localhost
1.2.3.4
2.3.4.5
3.4.5.6
4.5.6.7
====================
what is wrong !!?

parse /etc/hosts for ip and hostname with puppet, and transpose into /etc/sysconfig/network-scripts/ifcfg-eth0

I am kicking machines with cobbler to install redhat 6 and also put in hostname.
I am managing my machines with /etc/hosts file right now (may switch to dns in the future) and would like to have puppet (or anything that will work) parse through /etc/hosts and find my ip address and hostname (based on the hostname supplied by cobbler at kickstart). The trick is that some machines have multiple IP's and hostnames in the /etc/hosts file, like such:
# Maintenance Network
192.168.80.192 testsrv01-maint
192.168.80.193 testsrv02-maint
192.168.80.194 testsrv03-maint
# Lights Out Network
192.168.120.192 testsrv01-ilo
192.168.120.193 testsrv02-ilo
192.168.120.194 testsrv03-ilo
# Primary Data Network
192.168.150.192 testsrv01-pri
192.168.150.193 testsrv02-pri
192.168.150.194 testsrv03-pri
# Secondary Data Network
192.168.200.192 testsrv01-sec
192.168.200.193 testsrv02-sec
192.168.200.194 testsrv03-sec
I need to capture each ip and hostname pair (in a line) and transpose into /etc/sysconfig/network-scripts/ifcfg-eth* (eth1, eth2, eth3, ...). Puppet will need to create as many ifcfg-eth* files as there are matches in /etc/hosts for the hostname.
I just need puppet to append the $IP and $hostname to the ifcfg-eth file, the rest of the content is common.
So how would I get 4x ifcfg-eth files for 'testsrv01', with puppet?
Puppet is a very ill fit for this task. This calls for a script, which Puppet does not support. Puppet allows you to declare a piece of machine state that can portably enforced on different platforms.
The task is simple enough for a Shell or Perl Script.
With Puppet, it would entail the following scripts
writing custom facts to retrieve each address / hostname pair
devising a defined type to render such a pair into an ethX file
You'd possibly even need to generate the respective manifest, so that the appropriate interface index is chosen for each address.
All things considered, you would not make use of Puppet's strengths and suffer some of its weaknesses.

Remove EC2's entry from resolv.conf

I have private DNS servers and I want to write them to resolv.conf with resolvconf on Debian on AWS/EC2.
There is a problem in the order of nameserver entries.
In my resolv.conf, EC2's default nameserver is always written at first line like so:
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 172.16.0.23
nameserver 10.0.1.185
nameserver 10.100.0.130
search ap-northeast-1.compute.internal
172.16.0.23 is EC2's default nameserver and others are mine.
How to remove EC2 entry? Or, how to move EC2 entry to third?
Here I have an interface file:
% ls -l /etc/resolvconf/run/interface/
-rw-r--r-- 1 root root 62 Jun 7 23:35 eth0
It seems that the file eth0 is automatically generated by dhcp so can't remove it permanently.
% cat /etc/resolvconf/run/interface/eth0
search ap-northeast-1.compute.internal
nameserver 172.16.0.23
My private DNS entry is here:
% cat /etc/resolvconf/resolv.conf.d/base
nameserver 10.0.1.185
nameserver 10.100.0.130
Please help.
I think I just solved a very similar problem. I was bothered by Amazon EC2's crappy internal DNS servers so I wanted to run a local caching dnsmasq daemon and use that in /etc/resolv.conf. At first I just did echo nameserver 127.0.0.1 > /etc/resolv.conf but then I realized that my change would eventually be overwritten by the DHCP client after a reboot or DHCP lease refresh.
What I've now done instead is to edit /etc/dhcp3/dhclient.conf and uncomment the line prepend domain-name-servers 127.0.0.1;. You should be able to use the prepend directive in a very similar way.
Update: These instructions are based on Ubuntu Linux but I imagine the general concept applies on other systems as well, even other DHCP clients must have similar configuration options.
I'm approaching this problem from the other direction (wanting the internal nameservers), much of what I've learned may be of interest.
There are several options to control name resolution in the VPC management console.
VPC -> DHCP option sets -> Create dhcp option set
You can specify your own name servers there.
http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_DHCP_Options.html
Be sure to attach this dhcp option set to your VPC to get it to take effect.
Alternatively (I found this out by mistake) local dns servers are not set if the following settings are disabled in VPC settings:
DnsHostnames
and
DnsSupport
http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-dns.html
Settings can also be overridden locally (which you'll notice if you move instances between vpcs). /etc/dhcp/dhclient.conf
The following line might be of interest:
prepend domain-name-servers
Changes, of course, take effect on dhclient start.
How do I assign a static DNS server to a private Amazon EC2 instance running Ubuntu, RHEL, or Amazon Linux?
Short Description
Default behavior for an EC2 instance associated with a virtual private cloud (VPC) is to request a DNS server address at startup using the Dynamic Host Configuration Protocol (DHCP). The VPC responds to DHCP requests with the address of an internal DNS server. The DNS server addresses returned in the DHCP response are written to the local /etc/resolv.conf file and are used for DNS name resolution requests. Any manual modifications to the resolv.conf file are overwritten when the instance is restarted.
Resolution
To configure an EC2 instance running Linux to use static DNS server entries, use a text editor such as vim to edit the file /etc/dhcp/dhclient.conf and add the following line to the end of the file:
supersede domain-name-servers xxx.xxx.xxx.xxx, xxx.xxx.xxx.xxx;
Ubuntu - dhclient.conf - DHCP client configuration file 
The supersede statement
supersede [ option declaration ] ;
If for some option the client should always use a locally-configured value or values
rather than whatever is supplied by the server, these values can be defined in the
supersede statement.
The prepend statement
prepend [ option declaration ] ;
If for some set of options the client should use a value you supply, and then use the
values supplied by the server, if any, these values can be defined in the prepend
statement. The prepend statement can only be used for options which allow more than one
value to be given. This restriction is not enforced - if you ignore it, the behaviour
will be unpredictable.
The append statement
append [ option declaration ] ;
If for some set of options the client should first use the values supplied by the server,
if any, and then use values you supply, these values can be defined in the append
statement. The append statement can only be used for options which allow more than one
value to be given. This restriction is not enforced - if you ignore it, the behaviour
will be unpredictable.
In here someone come with solution that basically replaces the file on boot using rc.local
https://forums.aws.amazon.com/thread.jspa?threadID=74497
Edit /etc/sysconfig/network-scripts/ifcfg-eth0 to say PEERDNS=no
Create a file called /etc/resolv.backup with what you want
Add the following 2 lines to /etc/rc.local:
rm -f /etc/resolv.conf cp /etc/resolv.backup /etc/resolv.conf
This is what we are doing for our servers in the environment.
interface "eth0"
{
prepend domain-name-servers 10.x.x.x;
supersede host-name "{Hostname}";
append domain-search "domain";
supersede domain-name "DOMAIN";
}
Hope this helps.
The following worked in a Debian stretch on AWS EC2.
Just create /etc/dhcp/dhclient-enter-hooks.d/nodnsupdate:
#!/bin/sh
make_resolv_conf(){
:
}
Then you can modify /etc/resolv.conf and it will persist your changes across restarts.
Setup in crontab as
#reboot cp -r /home/.../resolv.conf /etc/resolv.conf

cant get dnsmasq to push multiple search prefixes

I'm trying to get dnsmasq to push multiple search prefixes to windows machines. If I look in the MS dhcp server, it looks to be using dhcp option 135, but any attempt to configure that eg
dhcp-option=135,domain.local1,domain.local2
doesnt get pushed at all (I'm using tcpdump -i br0 -lenx -s 1500 port bootps or port bootpc | dhcpdump to view wat dnsmasq is sending)
I have minor success using dhcp option 15, but it only pushes a single name into the search prefix as displayed by ipconfig /all on windows
Any suggestions ?
Checking the ISC dhcp option list I found this:
119 Domain Search domain-search
One or more domain names, each enclosed in quotes and separated by commas
But note that dnsmasq actually provides you special option (although I'm not sure from which version it starts)
dhcp-option=option:domain-search,eng.apple.com,marketing.apple.com
Our client machines (Ubuntu 18 server using netplan/systemd-resolve) were not requesting DHCP option 119, but I could solve the problem by forcing the server (dnsmasq) to sentd that option in the reply anyway:
dhcp-option-force=option:domain-search,internal,maindomain.com
dhcp-option=option:domain-name,maindomain.com
Using the dhcp-option-force parameter makes sure that the list is sent to the clients regardless of what they ask for.

Resources