I'm learning about sed but it is very difficult to me understand it.
I have adsl with dynamic ip so and i want to put current ip on hosts file.
This following script just tells me the current wan ip address and no more:
IP=$(dig +short myip.opendns.com #resolver1.opendns.com)
echo $IP
The result:
192.42.7.73
So, i have a line on hosts file with the old ip address:
190.42.44.22 peep.strudel.com
and i want to update host file like this:
192.42.7.73 peep.strudel.com
How can i do it? I think i can use the hostname as pattern...
The reason of doing this is because my server is a client of my router, so it access the internet thru its gateway and not directly. And postfix always is logging me that "connect from unknown [x.x.x.x]" (where x.x.x.x is my wan ip!) and it can't resolve that ip. I think that maybe if i specify this relating with my fqdn host/domain, on hosts file it will works better.
Thanks
Sergio.
You can use a simple shell script:
#! /bin/bash
IP=$(dig +short myip.opendns.com #resolver1.opendns.com)
HOST="peep.strudel.com"
sed -i "/$HOST/ s/.*/$IP\t$HOST/g" /etc/hosts
Explanation:
sed -i "/$HOST/ s/.*/$IP\t$HOST/g" /etc/hosts means in the line which contains $HOST replace everything .* by $IP tab $HOST.
using sed
sed -r "s/^ *[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+( +peep.strudel.com)/$IP\1/"
.
[0-9]+\. find all lines that matches 1 or more digits with this pattern 4 consecutive times then pattern peep.strudel.com .The parenthesis around the pattern peep.strudel.com save it as \1 then replace the whole patten with your variable and your new ip.
another approach:instead of saving pattern to a variable named IP, you can execute your command line inside sed command line to get the new IP .
sed -r "s/^ *[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+( +peep.strudel.com)/$(dig +short myip.opendns.com #resolver1.opendns.com)\1/"
using gawk
gawk -v IP=$IP '/ *[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+( +peep.strudel.com).*/{print gensub(/ *[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+( +peep.strudel.com)/,IP"\\1","g")}'
You need to include the sed code inside double quotes so that the used variable got expanded.
sed "s/\b\([0-9]\{1,3\}\.\)\{1,3\}[0-9]\{1,3\}\b/$IP/g" file
Add -i parameter to save the changes made. In basic sed \(..\) called capturing group. \{min,max\} called range quantifier.
Example:
$ IP='192.42.7.73'
$ echo '190.42.44.22 peep.strudel.com' | sed "s/\b\([0-9]\{1,3\}\.\)\{1,3\}[0-9]\{1,3\}\b/$IP/g"
192.42.7.73 peep.strudel.com
Related
I am developing a deployment script "that I want to be able to run over again without double entries"
I am trying to add a sed command that will look for "address" field, if it doesn't exist, create it, if it does exist modify it to the correct IP Address.
This is what I have so far...
#!/bin/bash
ipaddress=192.168.1.1
sudo grep -q '^address' /etc/network/interfaces && sudo sed -i 's/^address.*/"address $ipaddress"/' /etc/network/interfaces || echo "address ${ipaddress}" >> /etc/network/interfaces
It will create the correct entry if no entry exists but I have all kinds of problems if the entry exists or is correct.
Any help would be greatly appreciated!
Final Answer based on response from "1stSi Dave" Below
The final working script that creates the entry if it doesn't exist or alters any existing address entry is:
sudo grep -q '^address' /etc/network/interfaces && sudo sed -i -e 's/^address.*/address '$ipaddress'/' /etc/network/interfaces || echo "address ${ipaddress}" >> /etc/network/interfaces
First, I think you need the '-e' command line option to your sed command:
sed -i -e 's/...'
Maybe that's a typo, because the rest of your command line indicates you want to append to the file, not edit in place. Next, single quotes are hiding the variable expansion you're trying to achieve in the sed command script. They are also preserving the double quotes in the output, which I don't think is what you want. Try this:
sed -e 's/^address.*/address '$ipaddress'/' /etc/network/interfaces
Third, you may want to include the possibility of white space preceding the "address" token. Finally, you probably do want to edit-in-place (with sed -i), because tacking on the edited line at the end of the file is probably not going to work.
I have a huge list of IP address in a file and I want to replace all the IP address to a specified string( Example : X.X.X.X).
#Example.txt
1,1.1.1.1
2,10.10.10.10
3,5.5.5.5
4,6.6.6.6
.........
I tried replacing using sed
$sed -e 's/[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}/x.x.x.x/g' example.txt
I couldn't achieve this. Can some one help me on how to replace the IP address with a specific string?
You were almost there! All that you have to do is escape the repetition braces:
sed -e 's/[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}/x.x.x.x/g' test.txt
Good day. I was reading another post regarding resolving hostnames to IPs and only using the first IP in the list.
I want to do the opposite and used the following script:
#!/bin/bash
IPLIST="/Users/mymac/Desktop/list2.txt"
for IP in 'cat $IPLIST'; do
domain=$(dig -x $IP +short | head -1)
echo -e "$domain" >> results.csv
done < domainlist.txt
I would like to give the script a list of 1000+ IP addresses collected from a firewall log, and resolve the list of destination IP's to domains. I only want one entry in the response file since I will be adding this to the CSV I exported from the firewall as another "column" in Excel. I could even use multiple responses as semi-colon separated on one line (or /,|,\,* etc). The list2.txt is a standard ascii file. I have tried EOF in Mac, Linux, Windows.
216.58.219.78
206.190.36.45
173.252.120.6
What I am getting now:
The domainlist.txt is getting an exact duplicate of list2.txt while the results has nothing. No error come up on the screen when I run the script either.
I am running Mac OS X with Macports.
Your script has a number of syntax and stylistic errors. The minimal fix is to change the quotes around the cat:
for IP in `cat $IPLIST`; do
Single quotes produce a literal string; backticks (or the much preferred syntax $(cat $IPLIST)) performs a command substitution, i.e. runs the command and inserts its output. But you should fix your quoting, and preferably read the file line by line instead. We can also get rid of the useless echo.
#!/bin/bash
IPLIST="/Users/mymac/Desktop/list2.txt"
while read IP; do
dig -x "$IP" +short | head -1
done < "$IPLIST" >results.csv
Seems that in your /etc/resolv.conf you configured a nameserver which does not support reverse lookups and that's why the responses are empty.
You can pass the DNS server which you want to use to the dig command. Lets say 8.8.8.8 (Google) for example:
dig #8.8.8.8 -x "$IP" +short | head -1
The commands returns the domain with a . appended. If you want to replace that you can additionally pipe to sed:
... | sed 's/.$//'
Newbie here,
I'm trying to write to an auto-generated /etc/network/interfaces file of a newely provisioned XEN Ubuntu (12.04/10.04/8.04) DomU server at boot time using (currently) sed.
The auto-generated file is formatted as below:
auto eth0
iface eth0 inet static
address 192.168.0.88
gateway 192.168.0.254
network 255.255.255.255
auto lo
iface lo inet loopback
Using sed, I'm trying to alter lines 1 & 2, add a third line, remove the gateway and last two lines, and append four extra lines at the very end.
I'm currently stuck on adding the third line, as the script adds this line everytime it's run:
#!/bin/bash
sed -i "1s/.*/auto lo eth0/" /tmp/interfaces
sed -i "2s/.*/iface lo inet loopback/" /tmp/interfaces
sed -i "2a\iface eth0 inet static" /tmp/interfaces
sed -i "s/auto lo//g" /tmp/interfaces
Is it possible to add the third line only if it doesn't exist using sed (or awk)?
Likewise, how can I delete the gateway and last two lines only if they don't exist?
I'm new to sed, so am wondering whether I should be looking at awk instead for achieving this?
You can do that with sed:
sed -i -e '4{/iface eth0 inet static/! i\
iface eth0 inet static
}'
You can group commands with braces. The commands in the braces will only execute on the third line. The i insert command will only execute on the third line and if the third line doesn't match the string between slashes (the ! after it tells it to execute when it doesn't match).
You can do the same to delete:
sed -i -e '3{/gateway/d}'
Here we delete the third line only if it contains the string gateway. You could probably be more generic and simply do:
sed -i -e '/gateway/d'
which will delete all lines that contain gateway, but maybe that's not what you want.
As for deleting the last lines, the easiest solution would be:
sed -i -e '${/auto lo/d;/iface lo inet loopback/d}'
sed -i -e '${/auto lo/d;/iface lo inet loopback/d}'
Where the d delete command is executed on the last line if it matches either auto lo or iface lo inet loopback. Executing it twice will delete the last two lines if they match the patterns.
If you want to add lines to the end of the file, you can do:
sed -i -e '$a\
newline1\
newline2\
newline3'
Or maybe only add them if the last line isn't a specific line:
sed -i -e '${/192\.168\.1\.1/!a\
newline1\
newline2\
newline3
}'
Hope this helps a little =)
This script do the job: https://github.com/JoeKuan/Network-Interfaces-Script
awk -f changeInterface.awk <interfaces file> <device=ethX>
<mode=(dhcp|static|manual)> [action=add|remove]
[address=<ip addr> netmask=<ip addr> <name=value> ...]
I've got a tcpdump command running from a bash script. looks something like this.
tcpdump -nttttAr /path/to/file -F /my/filter/file
The filter file has a combination of ip addresses and host names. i.e.
host 111.111.111.111 or host 112.112.112.112 and not (host abc.com or host def.com or host zyx.com).
And it works great - as long as the host names are all valid. My problem is sometimes these hostnames will not be valid and upon encountering one - tcpdump spits out
tcpdump: Unknown Host
I thought with the -n option it would skip dns lookup - but in anycase I need it to ignore the unknown host and continue along the filter file.
Any ideas?
Thank you in advance.
The -n option prevents conversion of IP addresses into names, but not the other way around. If you supply a hostname as an argument, it has to be looked up to get the IP address since packets only contain the numeric address and not the hostname. However, there ought to be a way to ignore invalid hostnames, but I can't find one. Perhaps you could pre-process your filter file using dig.
dig +short non-existent-domain.com # returns null
dig +short google.com # returns multiple IP addresses
This could probably be better, but it should show you hostnames in your filter file that aren't valid:
grep -Po '(?<=host )[^ )]*' filterfile | grep -v '[0-9]$' | xargs -I % sh -c 'echo -n "% "; echo $(dig +short %)' | grep -v ' [0-9]'
Any hostnames it prints didn't have IP addresses returned by dig.