I have the below script for whois lookup
for line in $(cat ips.txt)
do
echo $line
whois $line | grep OrgName | awk '{print $2,$NF}'
done
I am having the output
192.168.1.1
Internet Authority
How can I achieve the output in the below format ?
192.168.1.2 : Internet Authority
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04 LTS"
On the 'echo $line' line, the shell was asked to print the value of $line. The shell says ok - done.
Then the shell moves on to the next line, that basically says 'get string then pipe it to some string manipulation and print result'.
I believe 'print something on the screen' was asked from the shell twice, 1 by echo 2 by awk, which are from 2 separate lines , so the shell behaved as expected.
To prevent this you can contain the second line in $(), so that echo will print "$line + $(whatever comes out here)"
for line in $(cat ips.txt)
do
echo $line : $(whois $line | grep OrgName | awk '{print $2,$NF}')
done
Related
I have a log file with a lot of lines with the following format:
IP - - [Timestamp Zone] 'Command Weblink Format' - size
I want to write a script.sh that gives me the number of times each website has been clicked.
The command:
awk '{print $7}' server.log | sort -u
should give me a list which puts each unique weblink in a separate line. The command
grep 'Weblink1' server.log | wc -l
should give me the number of times the Weblink1 has been clicked. I want a command that converts each line created by the Awk command above to a variable and then create a loop that runs the grep command on the extracted weblink. I could use
while IFS='' read -r line || [[ -n "$line" ]]; do
echo "Text read from file: $line"
done
(source: Read a file line by line assigning the value to a variable) but I don't want to save the output of the Awk script in a .txt file.
My guess would be:
while IFS='' read -r line || [[ -n "$line" ]]; do
grep '$line' server.log | wc -l | ='$variabel' |
echo " $line was clicked $variable times "
done
But I'm not really familiar with connecting commands in a loop, as this is my first time. Would this loop work and how do I connect my loop and the Awk script?
Shell commands in a loop connect the same way they do without a loop, and you aren't very close. But yes, this can be done in a loop if you want the horribly inefficient way for some reason such as a learning experience:
awk '{print $7}' server.log |
sort -u |
while IFS= read -r line; do
n=$(grep -c "$line" server.log)
echo "$line" clicked $n times
done
# you only need the read || [ -n ] idiom if the input can end with an
# unterminated partial line (is illformed); awk print output can't.
# you don't really need the IFS= and -r because the data here is URLs
# which cannot contain whitespace and shouldn't contain backslash,
# but I left them in as good-habit-forming.
# in general variable expansions should be doublequoted
# to prevent wordsplitting and/or globbing, although in this case
# $line is a URL which cannot contain whitespace and practically
# cannot be a glob. $n is a number and definitely safe.
# grep -c does the count so you don't need wc -l
or more simply
awk '{print $7}' server.log |
sort -u |
while IFS= read -r line; do
echo "$line" clicked $(grep -c "$line" server.log) times
done
However if you just want the correct results, it is much more efficient and somewhat simpler to do it in one pass in awk:
awk '{n[$7]++}
END{for(i in n){
print i,"clicked",n[i],"times"}}' |
sort
# or GNU awk 4+ can do the sort itself, see the doc:
awk '{n[$7]++}
END{PROCINFO["sorted_in"]="#ind_str_asc";
for(i in n){
print i,"clicked",n[i],"times"}}'
The associative array n collects the values from the seventh field as keys, and on each line, the value for the extracted key is incremented. Thus, at the end, the keys in n are all the URLs in the file, and the value for each is the number of times it occurred.
I'm filtering the output of ifconfig on a macOS (High Sierra) to trigger an action using a simple if statement, though the action isn't triggered although the corresponding strings appear to match (when tested with echo).
When the script is run, if the network en1 is active, the expected action is to print OK.
I tried:
printf "%s" in the awk command
using a double (and simple) quote for "active"
holding active in a variable for the comparison
replacing en1_status=`...` by en1_status=$(...)
without success.
#!/bin/bash
en1_status=`ifconfig en1 | grep "status: " | awk '{print $2}'`
if [ "$en1_status" = active ]; then
echo OK
fi
And $en1_status seems to match active:
echo $en1_status
active
This generally means that you've got nonprintable characters -- like syntax highlighting, or DOS newlines -- in your variable.
One easy way this can happen is if you've got your grep instance set to always insert color codes, as with --color=always rather than the default (safer) --color=auto.
To track this down, compare the output of the following commands:
printf '%s' "$en1_status" | xxd
printf '%s' inactive | xxd
You can also moot grep's configuration by letting awk do the searching:
en1_status=$(ifconfig en1 | awk '/status: / { print $2 }')
Your solution works on MacOS, if you double quote the right hand side of the if comparison (I replaced inactive for my local test, because that interface is not active locally for me):
~/> cat test.sh
#!/bin/bash
en1_status=`ifconfig en1 | grep "status: " | awk '{print $2}'`
if [ "$en1_status" = "inactive" ]; then
echo OK
fi
And when I run it:
~/> ./test.sh
OK
I need to parse the line written bold below:
line="eth1 Link encap:Ethernet HWaddr 11:11:11:11:11:11"
This line may have more words unexpectedly such as
line="eth1 Link encap:Ethernet Extra HWaddr 11:11:11:11:11:11"
So, for parsing the MAC address correctly, I need to parse the line accordingly with a bash command.
echo $line | cut -d' ' -f5* works for the first line, while *echo $line | cut -d' ' -f6* works for the second. So, I need to parse only the last column of the line.
However, because of the device restriction, I can only use grep and cut command. Not sed, awk, rev,reverse, etc.
With grep:
echo $line | grep -o -E '[^ ]+$'
With cut, a solution can be made with an extra computation based on the word count, assuming the delimiter is a space:
nw=$(echo $line | wc -w)
echo $line | cut -d ' ' -f$nw-
If the MAC address is the last sequence of characters after a space, you can remove the longest match of "* " (asterisk and a space) pattern using pure Bash:
echo "${line##* }"
You can also extract the last 17 characters from the string:
echo "${line: -17}"
If you want a strict match at the end of the line (due to .*):
echo $(expr match "$line" '.*\(\([a-zA-Z0-9]\{2\}\:\)\{5\}[a-zA-Z0-9]\{2\}\)')
Using GNU grep:
grep -o -P '(?:[a-zA-Z0-9]{2}:){5}[a-zA-Z0-9]{2}' <<< "$line"
In the latter case, you may want to add the $ anchor for the end of the line. Of course, you don't have to use here string. You may want to use a pipe instead: echo "$line" | grep -o -P ....
I am working on a script that executes ssh to few systems (listed in lab.txt), run two commands, store the output of commands in two different variables and print them.
Here is the script used :
#!/bin/bash
while read host; do
ssh -n root#$host "$(STATUS=$(awk 'NR==1{print $1}' /etc/*release) \
OS=$(/opt/agent/bin/agent.sh status | awk 'NR==1{print $3 $4}'))"
echo $STATUS
echo $OS
done < lab.txt
The lab.txt file contains few Ips where I need to login, execute and print the command output.
~#] cat lab.txt
192.168.1.1
192.168.1.2
While executing the script, the ssh login prompt of 192.168.1.1 is shown and after entering the password, the output is shown blank. Same as for next ip 192.168.1.2
When I execute these command manually within 192.168.1.1, the following is returned.
~]# awk 'NR==1{print $1}' /etc/*release
CentOS
~]# /opt/agent/bin/agent.sh status | awk 'NR==1{print $3 $4}'
isrunning
What could be wrong with the script? Is there a better way of doing this?
As the comment says, you are setting the variables inside the bash session on the server side and trying to read them from the client side.
If you want to assign the variables in the client script you need to put the assignment in front of the ssh command, and separate the two assignments. Something like the following.
STATUS=`ssh -n root#$host 'awk \'NR==1{print $1}\' /etc/*release)`
OS=`ssh -n root#$host '/opt/agent/bin/agent.sh status | awk \'NR==1{print $3 $4}\''`
You need to do two ssh commands. It also simplifies things if you run awk on the client rather than the server, because quoting in the ssh command gets complicated.
while read host; do
STATUS=$(ssh -n root#$host 'cat /etc/*release' | awk 'NR==1{print $1}')
OS=$(ssh -n root#$host /opt/agent/bin/agent.sh status | awk 'NR==1{print $3 $4}')
echo $STATUS
echo $OS
done < lab.txt
with one ssh statement:
read STATUS OS < <(ssh -n root#$host "echo \
\$(awk 'NR==1{print \$1}' /etc/*release) \
\$(/opt/agent/bin/agent.sh status | awk 'NR==1{print \$3 \$4}')")
echo $STATUS
echo $OS
Explanation:
The <(command) syntax is called process substitution. You can use it anywhere where a file is expected.
Example:
sdiff <(echo -e "1\n2\n3") <(echo -e "1\n3")
The command sdiff expects two files as arguments. With the process substitution syntax you can use commands as arguments. ( e.g. fake files )
This is my code so far:
#! /bin/bash
Options=("1" "2" "3")
select opt in "${Options[#]}"
do
case "$REPLY" in
1)
who=$(whoami)
echo -e "you are: $who"
;;
2)
var=$(uptime | awk '{print $3}' | cut -d, -f 1)
echo $var
;;
3) break;;
*) echo "Invalid input";;
esac
done
It doesn't work. The first option works just fine. The second one however doesn't.
Further testing suggests bash doesn't like piping much, the output is simply "#", suggesting it's only the awk part of the command that actually gets executed. The command just works fine out in the shell (it reports the time it has been online only).
This line doesn't make any sense:
var=$(uptime | awk '{print 3}' | -d, -f 1)
The awk statement should be
awk '{print $3}'
and the cut statement should be
cut -d ',' -f 1
To be honest, you probably don't need the awk statement at all for what it seems you want to do. Just pipe uptime to the cut statement above.
var=$(uptime | cut -d ',' -f 1)
This happens when you use a text files from Windows/Dos and reuse it in unix.
The line terminators in each respective OS are different.
HTH
p.s. You'll notice a ^M at the end of a line when you open the text file in vi/vim.
Doesn't work says "cut: invalid byte or field list" – user2230627 Apr 1 at 3:34
: