How to change IP dynamically in hadoop conf.xml - shell

cat test.xml |sed -r 's/(\b[0-9]{1,3}\.){3}[0-9]{1,3}\b'/$ip_address/ >new.xml
Here, I am trying to replace pattern matching IP with new IP address. I am getting
unterminated s command. Please let me know what is going wrong

Related

SED command to add or update IP address from variable to /etc/network/interfaces

I am developing a deployment script "that I want to be able to run over again without double entries"
I am trying to add a sed command that will look for "address" field, if it doesn't exist, create it, if it does exist modify it to the correct IP Address.
This is what I have so far...
#!/bin/bash
ipaddress=192.168.1.1
sudo grep -q '^address' /etc/network/interfaces && sudo sed -i 's/^address.*/"address $ipaddress"/' /etc/network/interfaces || echo "address ${ipaddress}" >> /etc/network/interfaces
It will create the correct entry if no entry exists but I have all kinds of problems if the entry exists or is correct.
Any help would be greatly appreciated!
Final Answer based on response from "1stSi Dave" Below
The final working script that creates the entry if it doesn't exist or alters any existing address entry is:
sudo grep -q '^address' /etc/network/interfaces && sudo sed -i -e 's/^address.*/address '$ipaddress'/' /etc/network/interfaces || echo "address ${ipaddress}" >> /etc/network/interfaces
First, I think you need the '-e' command line option to your sed command:
sed -i -e 's/...'
Maybe that's a typo, because the rest of your command line indicates you want to append to the file, not edit in place. Next, single quotes are hiding the variable expansion you're trying to achieve in the sed command script. They are also preserving the double quotes in the output, which I don't think is what you want. Try this:
sed -e 's/^address.*/address '$ipaddress'/' /etc/network/interfaces
Third, you may want to include the possibility of white space preceding the "address" token. Finally, you probably do want to edit-in-place (with sed -i), because tacking on the edited line at the end of the file is probably not going to work.

Replace all IP addresses in a file to a specified string

I have a huge list of IP address in a file and I want to replace all the IP address to a specified string( Example : X.X.X.X).
#Example.txt
1,1.1.1.1
2,10.10.10.10
3,5.5.5.5
4,6.6.6.6
.........
I tried replacing using sed
$sed -e 's/[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}/x.x.x.x/g' example.txt
I couldn't achieve this. Can some one help me on how to replace the IP address with a specific string?
You were almost there! All that you have to do is escape the repetition braces:
sed -e 's/[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}/x.x.x.x/g' test.txt

bash script to perform dig -x

Good day. I was reading another post regarding resolving hostnames to IPs and only using the first IP in the list.
I want to do the opposite and used the following script:
#!/bin/bash
IPLIST="/Users/mymac/Desktop/list2.txt"
for IP in 'cat $IPLIST'; do
domain=$(dig -x $IP +short | head -1)
echo -e "$domain" >> results.csv
done < domainlist.txt
I would like to give the script a list of 1000+ IP addresses collected from a firewall log, and resolve the list of destination IP's to domains. I only want one entry in the response file since I will be adding this to the CSV I exported from the firewall as another "column" in Excel. I could even use multiple responses as semi-colon separated on one line (or /,|,\,* etc). The list2.txt is a standard ascii file. I have tried EOF in Mac, Linux, Windows.
216.58.219.78
206.190.36.45
173.252.120.6
What I am getting now:
The domainlist.txt is getting an exact duplicate of list2.txt while the results has nothing. No error come up on the screen when I run the script either.
I am running Mac OS X with Macports.
Your script has a number of syntax and stylistic errors. The minimal fix is to change the quotes around the cat:
for IP in `cat $IPLIST`; do
Single quotes produce a literal string; backticks (or the much preferred syntax $(cat $IPLIST)) performs a command substitution, i.e. runs the command and inserts its output. But you should fix your quoting, and preferably read the file line by line instead. We can also get rid of the useless echo.
#!/bin/bash
IPLIST="/Users/mymac/Desktop/list2.txt"
while read IP; do
dig -x "$IP" +short | head -1
done < "$IPLIST" >results.csv
Seems that in your /etc/resolv.conf you configured a nameserver which does not support reverse lookups and that's why the responses are empty.
You can pass the DNS server which you want to use to the dig command. Lets say 8.8.8.8 (Google) for example:
dig #8.8.8.8 -x "$IP" +short | head -1
The commands returns the domain with a . appended. If you want to replace that you can additionally pipe to sed:
... | sed 's/.$//'

sed: replace ip in hosts file, using hostname as pattern

I'm learning about sed but it is very difficult to me understand it.
I have adsl with dynamic ip so and i want to put current ip on hosts file.
This following script just tells me the current wan ip address and no more:
IP=$(dig +short myip.opendns.com #resolver1.opendns.com)
echo $IP
The result:
192.42.7.73
So, i have a line on hosts file with the old ip address:
190.42.44.22 peep.strudel.com
and i want to update host file like this:
192.42.7.73 peep.strudel.com
How can i do it? I think i can use the hostname as pattern...
The reason of doing this is because my server is a client of my router, so it access the internet thru its gateway and not directly. And postfix always is logging me that "connect from unknown [x.x.x.x]" (where x.x.x.x is my wan ip!) and it can't resolve that ip. I think that maybe if i specify this relating with my fqdn host/domain, on hosts file it will works better.
Thanks
Sergio.
You can use a simple shell script:
#! /bin/bash
IP=$(dig +short myip.opendns.com #resolver1.opendns.com)
HOST="peep.strudel.com"
sed -i "/$HOST/ s/.*/$IP\t$HOST/g" /etc/hosts
Explanation:
sed -i "/$HOST/ s/.*/$IP\t$HOST/g" /etc/hosts means in the line which contains $HOST replace everything .* by $IP tab $HOST.
using sed
sed -r "s/^ *[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+( +peep.strudel.com)/$IP\1/"
.
[0-9]+\. find all lines that matches 1 or more digits with this pattern 4 consecutive times then pattern peep.strudel.com .The parenthesis around the pattern peep.strudel.com save it as \1 then replace the whole patten with your variable and your new ip.
another approach:instead of saving pattern to a variable named IP, you can execute your command line inside sed command line to get the new IP .
sed -r "s/^ *[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+( +peep.strudel.com)/$(dig +short myip.opendns.com #resolver1.opendns.com)\1/"
using gawk
gawk -v IP=$IP '/ *[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+( +peep.strudel.com).*/{print gensub(/ *[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+( +peep.strudel.com)/,IP"\\1","g")}'
You need to include the sed code inside double quotes so that the used variable got expanded.
sed "s/\b\([0-9]\{1,3\}\.\)\{1,3\}[0-9]\{1,3\}\b/$IP/g" file
Add -i parameter to save the changes made. In basic sed \(..\) called capturing group. \{min,max\} called range quantifier.
Example:
$ IP='192.42.7.73'
$ echo '190.42.44.22 peep.strudel.com' | sed "s/\b\([0-9]\{1,3\}\.\)\{1,3\}[0-9]\{1,3\}\b/$IP/g"
192.42.7.73 peep.strudel.com

sed: modify a file using contents of another file

I've got a script which writes an ip address to a file ip.txt
I want to replace an ip address in an html file with the ip from ip.txt.
I've got a sed regex expression that matches an ip address, and I want to replace this matched text with the contents of ip.txt:
"s/\([0-9]\{1,3\}\.\)\{3\}[0-9]\{1,3\}//g"
How can I get sed to pull the contents of ip.txt and put it in the expression s/<search>/<*HERE*>/g?
Is there a better way to do it than this?
sed -e "s/\([0-9]\{1,3\}\.\)\{3\}[0-9]\{1,3\}/"`cat ip.txt`"/g"
Not much of an improvement, but you can replace
"`cat ip.txt`"
with
$(<ip_txt)
which will be replaced with the contents of the file and is slightly more efficient than using cat.
You can read the IP into a shell variable before running the sed command. Assuming that ip.txt is a single line containing only the IP address:
read -r ip < ip.txt
sed -e "s/\([0-9]\{1,3\}\.\)\{3\}[0-9]\{1,3\}/$ip/g" file.html > newfile.html
mv newfile.html file.html
POSIX sed (and therefore most of the available implementations of sed) supports the r file command to read a file when a line is matched. As long as you don't mind have the result containing newlines either side of where the IP address was, you could easily enough do it using r. The description says:
[1addr]r rfile
Copy the contents of rfile to standard output as described previously. If rfile does not exist or cannot be read, it shall be treated as if it were an empty file, causing no error condition.
This means that you don't get a chance to edit the contents of the file, whereas if it was read into the pattern space or hold space, you could could then modify the data.
This being the case, your command line substitution is about as good as you can do.

Resources