I have some Web URLs I am trying to do a nslookup against. All it does is check against the URL and print the ones not equals to some certain IP Address into a file. I am able to do it for one IP Address but i tried adding one more, and I am unable to get it to work.
SUB='.com'
for address in `cat linux.hosts`; do
if [[ "$address" == *"$SUB"* ]]; then
echo "Got [$address]"
nslookup $address \
| awk '!/155.55.66.55/ || !/155.55.66.54/' >> com.txt
fi
done
The linux.hosts file contains info like this
A B B { D E google.com }
A B B { D E twitter.com }
A B B { D E microsoft.com }
A B B { D E facebook.com }
I only want to get the string that has ".com" in it and do a nslookup that doesn't contain a certain IP Address.
The $nslookup address returns
Got [google.com]
Server: BBBB:BBBB:BBBB:BBBB::1
Address: BBBB:BBBB:BBBB:BBBB::1#53
Non-authoritative answer:
Name: google.com
Address: 155.55.66.55
Name: google.com
Address: 155.55.66.54
The | awk '!/155.55.66.55/ || ' >> com.txt works for some of the address that only contains 155.55.66.55 but if contains 155.55.66.54 it does not work, hence i am trying to add another check.
I only want to print the domains with address that doesn't contain
155.55.66.54 and 155.55.66.55.
The $nslookup address should only return
Got [twitter.com]
Server: BBBB:BBBB:BBBB:BBBB::1
Address: BBBB:BBBB:BBBB:BBBB::1#53
Non-authoritative answer:
Name: google.com
Address: 198.168.101.1
Name: google.com
Address: 198.168.101.2
Don't read lines with for.
Also, probably redirect only once, after the loop, to avoid having the shell opening the file and seeking to the end repeatedly inside the loop.
As a minor optimization, use grep to find the matching lines in the file before the main loop.
grep -F '.com' linux.hosts |
while read -r _ _ _ _ _ _ address _; do
nslookup "$address" |
awk '!/155.55.66.55/ || !/155.55.66.54/'
done >>com.txt # or maybe > com.txt if you want to overwrite
Related
Im writing a script that executes dig command on 2 domains, and after next cmd is host on output.
And always i will get for exmaple:
findUserServer=for r in $(dig +short $login.example.COM && dig +short $login.example.ORG); do host $r|awk '{print $NF}';done | awk -F "." '{print $1}';
1 output: >> asdf02 example
asdf02 - its a server name, its always same name starts "asdf".
Question: Have you any idea how to save to the variable only asdf02?
question+: asdf02 woudln't be always first, could be example asdf02
Should i do maybe a sed which looks on 4 first characters? If it's "asdf", then: [...]
Try not to pipe awk commands into each other and so:
for r in $(dig +short $login.example.COM && dig +short $login.example.ORG); do host $r;done | awk -F [.\ ] '/asdf02/ { print $10 }'
We use both a space and . as delimiters and then pattern match the output for the occurance of asdf02. If we find is, we print the address.
Run that through shellcheck.net ...
Try this.
findUserServer="$( for end in COM ORG; do
host $( dig +short $login.example.$end );
done | sed -n '/ asdf/{ s/^.* //; s/[.].*//; p; }' )"
This will run 2 digs and pipe the collective output through sed,
which will ignore lines that don't have asdf, and strip the matches clean for you.
Let me know if I missed details, because I don't have those exact values available.
I have an .yml file with three address entries and some other data, and an array containing three new addresses to replace these with
file.yml:
[...]
- address: s1.example.com
user: ubuntu
role: storage
- address: w1.example.com
user: ubuntu
role: worker
- address: w2.example.com
user: ubuntu
role: worker
[...]
array:
addr[0]: storage.domain.com
addr[1]: worker1.domain.com
addr[2]: worker2.domain.com
expected result:
[...]
- address: storage.domain.com
user: ubuntu
role: storage
- address: worker1.domain.com
user: ubuntu
role: worker
- address: worker2.domain.com
user: ubuntu
role: worker
[...]
I'm using sed, as I would like to write the new lines directly to the original file
I have tried a number of times, but the array incrementing always fails
Attempt 1
sed -i "s/- address: .*/- address: ${addr[$i]}/g" file.yml
This seems to exclusively write the first item in the array:
- address: storage.domain.com
[...]
- address: storage.domain.com
[...]
- address: storage.domain.com
[...]
Attempt 2
if (grep -e "- address:" file.yml); then
sed -i "s/- address: .*/- address: ${addr[$i]}/g"
((i++))
fi
This seems to grep all results at the same time, and forwards nothing to sed as I haven't figured that one out yet.
Output:
- address: s1.example.com
- address: w1.example.com
- address: w2.example.com
sed: no input files
Recently I started doing this with similar jobs:
Print all array members with some separator, ex. :. Put this as the first input to sed.
The first line (ie. the array members printed with some separator) are put into hold space
Then for each the - address: found:
I copy the hold space into pattern space, extract the first element of the array and append it with - address :. And print.
And remove the first element from hold space
The script below:
# replicate input
cat <<EOF >file.yml
[...]
- address: s1.example.com
user: ubuntu
role: storage
- address: w1.example.com
user: ubuntu
role: worker
- address: w2.example.com
user: ubuntu
role: worker
[...]
EOF
addr[0]=storage.domain.com
addr[1]=worker1.domain.com
addr[2]=worker2.domain.com
# the sed script
sed -nE '
1{
# hold the first line with array members separated with :
H
# don't print anything
d
}
# if the line is address
/- address: .*/{
g
s/\n//
# remove all except first array member from hold space
s/:.*//
# prepend it with _- address_
s/^/- address: /
# remove the first member from hold space
x
s/[^:]*://
x
}
# print the output
p
' <( IFS=':'; printf "%s\n" "${addr[*]}"; ) file.yml
and the same oneliner:
sed -n '1{;H;d};/- address: .*/{g;s/\n//;s/:.*//;s/^/- address: /;x;s/[^:]*://;x};p' <( IFS=':'; printf "%s\n" "${addr[*]}"; ) file.yml
will output:
[...]
- address: storage.domain.com
user: ubuntu
role: storage
- address: worker1.domain.com
user: ubuntu
role: worker
- address: worker2.domain.com
user: ubuntu
role: worker
[...]
#! /bin/bash
# initialise the array the way you want
addr[0]="storage.domain.com"
addr[1]="worker1.domain.com"
addr[2]="worker2.domain.com"
awk -F: -v addr="${addr[*]}" ' BEGIN{count=0 ; split(addr, addr_array, " ") }
{
if( $1 ~ /address/ ) {
for(i=1;i<=NF-1;++i){
printf "%s:", $i
}
printf "%s\n", addr_array[++count]
}
else
{
print
}
}' file.yml
If you want to overwrite the original file,
addr[0]="storage.domain.com"
addr[1]="worker1.domain.com"
addr[2]="worker2.domain.com"
gawk -i inplace -F: -v addr="${addr[*]}" ' BEGIN{count=0 ; split(addr, addr_array, " ") }
{
if( $1 ~ /address/ ) {
for(i=1;i<=NF-1;++i){
printf "%s:", $i
}
printf "%s\n", addr_array[++count]
}
else
{
print
}
}' file.yml
One possible solution is as follows:
addr=(storage.domain.com worker1.domain.com worker2.domain.com)
i=0
line_count=1
while IFS= read -r line; do
if [[ $line == *"- address:"* ]]; then
sed -i "${line_count}s/- address: .*/- address: ${addr[i]}/" file.yml
i=$((i+1))
fi
line_count=$((line_count+1))
done < file.yml
Above script iterates through the file, line by line and then replace the matching line with the content of the array.
This might work for you (GNU sed):
printf ": %s\n" "${addr[#]}" |
sed '/address:/R /dev/stdin' ymlFile |
sed '/address:/{N;s/:.*:/:/}'
Print the addr array out to stdout so that each address is on a separate line.
In the first invocation of sed, insert each address on a separate line following the regexp address:.
In the second invocation of sed remove the old addresses.
If I run below command on terminal it gives IP address value
ip route get 8.8.8.8 | grep -oP '(?<=src )(\d{1,3}.){4}'
But when I run same command with '2>&1' then it returns empty string:
output = ''
IO.popen("ip route get 8.8.8.8 | grep -oP '(?<=src )(\d{1,3}.){4}' 2>&1", 'r+') do |f|
output = f.read.strip
end
puts output.inspect
Please guide me to understand above scenario.
What modifications I need to add to get IP address.
Nothing to do with the redirection. In Ruby, backslashes in strings must be escaped. Just replace \ with \\:
output = ''
IO.popen("ip route get 8.8.8.8 | grep -oP '(?<=src )(\\d{1,3}.){4}' 2>&1", 'r+') do |f|
output = f.read.strip
end
puts output.inspect
I have an IP address set in a variable that I'd like to convert into a network address.
This only works for a single digit:
echo '192.168.1.2' | sed 's/.$/0/' => 192.168.1.0
echo '192.168.1.22' | sed 's/.$/0/' => 192.168.1.20
echo '192.168.1.223' | sed 's/.$/0/' => 192.168.1.220
I need a method to return the same network value if the last digit(s) change, i.e:
myip="192.168.1.2" => "192.168.1.0"
myip="192.168.1.22" => "192.168.1.0"
myip="192.168.1.223" => "192.168.1.0"
How can I replace any IP address with it's network address like above?
Pure bash solution without external commands:
echo "${myip%.*}.0"
for example:
$ echo "$myip"
192.168.1.22
$ echo "${myip%.*}.0"
192.168.1.0
Using sed
echo '192.168.1.2' | sed 's/\.[^.]*$/.0/'
sed 's/\.[^.]*$/.0/' <<< 192.168.1.22 # echo + pipe is not needed here
Logic: Replace everything from last . till end with .0
Using awk
awk -F. '{$NF=0}1' OFS=. <<< 192.168.1.22
awk '{$NF=0}1' FS=. OFS=. <<< 192.168.1.22
Logic: Split string with . and set last field to 0.
pure bash:
{ IFS=. read a b c _; echo $a.$b.$c.0; } <<< 192.168.1.22
( IFS=.; read -a ip; ip[3]=0; echo "${ip[*]}"; ) <<< 192.168.1.22
Logic: Read 4 parts of the IP address in 4 variables. Print first 3 and a 0.
Or by using a bash array, if you don't want to clutter environment with too many variables.
You can do this with awk using:
pax> awk -F. '{print $1"."$2"."$3".0"}' <<<12.34.56.78
12.34.56.0
With sed, it's possible to just replace all the digits at the end:
pax sed 's/[0-9]*$/0/' <<<12.34.56.78
12.34.56.0
However, all of those result in an extra process being started up, not something you need to worry about for a few IP addresses but it will make a difference if you're converting many of them.
To do it within bash only (not requiring another process), you can use:
pax> ip=12.34.56.78
pax> echo ${ip%.[0-9]*}.0
12.34.56.0
It is very simple to do with pure bash:
myip="192.168.1.2 "; echo "$myip ==> ${myip%.*}.0"
myip="192.168.1.22 "; echo "$myip ==> ${myip%.*}.0"
myip="192.168.1.223"; echo "$myip ==> ${myip%.*}.0"
Results in:
192.168.1.2 ==> 192.168.1.0
192.168.1.22 ==> 192.168.1.0
192.168.1.223 ==> 192.168.1.0
However, that is assuming the network has a CDIR of 24 (192.168.1.2/24).
If that is not what you will always use, this idea will break.
I'm trying to create a bash script which will be able to change the "allow from" ip address in the phpmyadmin command file (which im still not sure is possible to do) and restart apache
I'm currently trying to extract an ip address from a variable and after searching the web I still have no clue, here is what I have so far...
#bash shell script
#!/bin/bash
clear
echo "Get client IP address"
ip=$(last -i)
echo $ip
exit
echo "restart apache"
/etc/init.d/apache2 reload
I've tried adding the following line with no luck
ip=$(head -n 1 $ip)
If anyone can tell me how I can extract the first instance of an IP address from the variables $ip I would appreciate it very much.
ip=$(last -i | head -n 1 | awk '{print $3}')
Update:
ip=$(last -i | grep -Pom 1 '[0-9.]{7,15}')
You can use grep with read:
read ip < <(last -i | grep -o '[0-9]\+[.][0-9]\+[.][0-9]\+[.][0-9]\+')
read ip < <(last -i | grep -Eo '[0-9]+[.][0-9]+[.][0-9]+[.][0-9]+')
\b may also be helpful there. Just not sure about its compatibility.
And yet another:
ip=$(last -i | gawk 'BEGIN { RS = "[ \t\n]"; FS = "." } /^([0-9]+[.]){3}[0-9]+$/ && ! rshift(or(or($1, $2), or($3, $4)), 8) { print ; exit; }')
To get the first instance you can just do:
ip=$(last -i -1 | awk '{print $3}')
I'd just do
ip=$(last -i -1 | grep -Po '(\d+\.){3}\d+')
The above uses grep with Perl Compatible Regular Expressions which lets us use \d for digits. The regular expression looks for three repetitions of [0-9] followed by a dot (so, for example 123.45.123), then another stretch of digits. The -o flag causes grep to only print the matching line.
This approach has the advantage of working even when the number of fields per line changes (as is often the case, for example with system boot as the 2nd field). However, it needs GNU grep so if you need a more portable solution, use #konsolebox's answer instead.
Using bash only :
read -ra array < <(last -i)
ip="${array[2]}"
Or :
read -ra array < <(last -1 -i)
ip="${array[2]}"
Or if you're a nitpicker (and have a grep with -P), you can test the next:
while read -r testline
do
echo "input :=$testline="
read ip < <(grep -oP '\b(?:(?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2}))\b' <<< "$testline")
echo "result:=${ip:=NOTFOUND}="
echo
done <<EOF
some bla bla 127.0.0.1 some
10.10.10.10
bad one 300.200.300.400
some other bla 127.0.0.1 some another 10.1.1.0
10.10.10.10 10.1.1.0
bad one 300.200.300.400 and good 192.168.1.1
above is empty and no ip here too
EOF
It skips wrong ip adr, like 800.1.1.1 so, for the above test prints:
input :=some bla bla 127.0.0.1 some=
result:=127.0.0.1=
input :=10.10.10.10=
result:=10.10.10.10=
input :=bad one 300.200.300.400=
result:=NOTFOUND=
input :=some other bla 127.0.0.1 some another 10.1.1.0=
result:=127.0.0.1=
input :=10.10.10.10 10.1.1.0=
result:=10.10.10.10=
input :=bad one 300.200.300.400 and good 192.168.1.1=
result:=192.168.1.1=
input :==
result:=NOTFOUND=
input :=above is empty and no ip here too=
result:=NOTFOUND=
The \b is needed to skip matching an ip, like: 610.10.10.10, what is containing a valid ip (10.10.10.10).
The regex is taken from: https://metacpan.org/pod/Regexp::Common::net
Since I happen to have needed to do something in the same ballpark, here is a basic regular expression and an extended regular expression to loosly match an IP address (v4) making sure that there are 4 sequences of 1-3 numbers delimited by a 3 '.'.
# Basic Regular Expression to loosly match an IP address:
bre_match_ip="[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}"
# Extended Regular Expression to loosly match an IP address:
ere_match_ip="[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}"
Of course when matching IP (v4) addresses from a file (say HTML) it's quite easy to inadvertently match a version string or an url which contains versioning as part of its file path. The following is some Awk code I wrote a while ago for use in a Bash script to extract valid unique (no duplicates) IP addresses from a file. It avoids version numbers whether in the text or as part of an url and makes sure the IP numbers are in range.
I appreciate that this is overkill for the original poster and that it is not tailored for his needs but someone doing a search may come across this answer and find the fairly comprehensive nature of the code useful. The Awk code is thankfully well commented as it uses some slightly obscure aspects of Awk that the casual Awk user would probably not be familiar with.
awkExtractIPAddresses='
BEGIN {
# Regex to match an IP address like sequence (even if too long to be an IP).
# This is deliberately a loose match, the END section will check for IP
# address validity.
ipLikeSequence = "[0-9]+[.][0-9]+[.][0-9]+[.][0-9]+[0-9.]*";
# Regex to match a number sequence longer than 3 digits.
digitSequenceTooLongNotIP = "[0-9][0-9][0-9][0-9]+";
# Regex to match an IP address like sequence which is a version number.
# Equivalent to "(version|ver|v)[ .:]*" if "tolower($0)" was used.
versioningNotIP = "[Vv]([Ee][Rr]([Ss][Ii][Oo][Nn])?)?[ .:]*" ipLikeSequence;
# Regexes to match IP address like sequences next to forward slashes, to
# avoid version numbers in urls: e.g. http://web.com/libs/1.6.1.0/file.js
beginsWithFwdSlashNotIP = "[/]" ipLikeSequence;
endsWithFwdSlashNotIP = ipLikeSequence "[/]";
}
{
# Set line to the current line (more efficient than using $0 below).
line = $0;
# Replace sequences on line which will interfere with extracting genuine
# IPs. Use a replacement char and not the empty string to avoid accidentally
# creating a valid IP address from digits on either side of the removed
# sections. Use "/" as the replacement char for the 2 "FwdSlash" regexes so
# that multiple number dot slash sequences all get removed, as using "x"
# could result in inadvertently leaving such a sequence in place.
# e.g. "/lib1.6.1.0/1.2.3.4/5.6.7.8/file.js" leaves "/lib1.6.1.0xx/file.js"
gsub(digitSequenceTooLongNotIP, "x", line);
gsub(versioningNotIP, "x", line);
gsub(beginsWithFwdSlashNotIP, "/", line);
gsub(endsWithFwdSlashNotIP, "/", line);
# Loop through the current line matching IP address like sequences and
# storing them in the index of the array ipUniqueMatches. By using ipMatch
# as the array index duplicates are avoided and the values can be easily
# retrieved by the for loop in the END section. match() automatically sets
# the built in variables RSTART and RLENGTH.
while (match(line, ipLikeSequence))
{
ipMatch = substr(line, RSTART, RLENGTH);
ipUniqueMatches[ipMatch];
line = substr(line, RSTART + RLENGTH + 1);
}
}
END {
# Define some IP address related constants.
ipRangeMin = 0;
ipRangeMax = 255;
ipNumSegments = 4;
ipDelimiter = ".";
# Loop through the ipUniqueMatches array and print any valid IP addresses.
# The awk "for each" type of loop is different from the norm. It provides
# the indexes of the array and NOT the values of the array elements which
# is more usual in this type of loop.
for (ipMatch in ipUniqueMatches)
{
numSegments = split(ipMatch, ipSegments, ipDelimiter);
if (numSegments == ipNumSegments &&
ipSegments[1] >= ipRangeMin && ipSegments[1] <= ipRangeMax &&
ipSegments[2] >= ipRangeMin && ipSegments[2] <= ipRangeMax &&
ipSegments[3] >= ipRangeMin && ipSegments[3] <= ipRangeMax &&
ipSegments[4] >= ipRangeMin && ipSegments[4] <= ipRangeMax)
{
print ipMatch;
}
}
}'
# Extract valid IP addresses from $fileName, they will each be separated
# by a new line.
awkValidIpAddresses=$(awk "$awkExtractIPAddresses" < "$fileName")
I hope this is of interest.
You could use Awk.
ip=$(awk '{if(NR == 1) {print $3; exit;}}' < <(last -i))