awk printing random "e" that is not in string - bash

I am wanting to extract ip and port from a string.
Strings look like this.
destination x.x.x.x:yyyy
where x is ip and y is port
commandout=()
while IFS= read -r line # Read a line
do
commandout+=("$line") # Append line to the array
done < <(tmsh list ltm virtual $vip | grep destination)
for output in "$commandout";
do
if [[ $output == *"destination"* ]];then
#split off ip and port
ipport=$(echo $output | awk 'BEGIN{}{print $2}')
echo $ipport | awk 'BEGIN{FS=":"}{print $1}'
echo $ipport
fi
done
declare -p commandout
for some reason, awk is printing a random "e" after the ip address. But it only appears to do so after 2.
10.10.10.10
10.10.10.10:https
declare -a commandout='([0]=" destination 10.10.10.10:https")'
12.12.12.12e
12.12.12.12:https
declare -a commandout='([0]=" destination 12.12.12.12:https")'
UPDATE:
So I attempted another test. I found strange behavior and I am unsure how to fix it.
I declare the vipip before and after it is set.
declare -p vipip
vipip=$(tmsh list ltm virtual $vip | grep destination | awk 'BEGIN{}{print $2}' | awk 'BEGIN{FS=":"}{print $1}')
echo $vipip
declare -p vipip
echo "cyle loop"
results in the following. Note that the 12.12.12.12 doesn't have an "e" on the end of it
./findvips-final.scr: line 240: declare: vipip: not found
10.10.10.10
declare -- vipip="10.10.10.10"
cyle loop
declare -- vipip="10.10.10.10"
12.12.12.12
declare -- vipip="12.12.12.12"
cyle loop
If I comment out the declare statements, I get an "e"
#declare -p vipip
vipip=$(tmsh list ltm virtual $vip | grep destination | awk 'BEGIN{}{print $2}' | awk 'BEGIN{FS=":"}{print $1}')
echo $vipip
#declare -p vipip
echo "cyle loop"
results in
10.10.10.10
cyle loop
12.12.12.12e
cyle loop
I found the answer. I have a progress meter above this and I was getting the e off of complete.
echo -ne "$((100*$z/$count))% Complete\r"
I wrapped $vipip in qoutes on the echo and it is working like I thought. UGh wait a big waste of time.

You can straightaway set FS like below to extract Ip from your command, no need of loop, awk can search string also
your_command | awk -F'[ :]' '/destination/{gsub(/[^0-9.]/,"",$2); print $2}'
Explanation
-F'[ :]' - set field separator
'/destination/ - search for word destination in line/record/row
gsub(/[^0-9.]/,"",$2) - remove anything other than number and dot from second field ( so that random char like e, what you said above will be removed )
print $2 - print second field

Related

How can I read a stream of key=value pairs in bash if the values can be empty?

So I have several properties files that contains keys and values in the following format:
Key1=value1
Key2=Value2
etc..
So when I run this command to get the keys:
ssh 172.16.1.1 "grep -v '^#' /src/PTB.properties" | awk -F "=" '{print $1}'
I get the following output
userField
User
srcIP
srcPort
dstIP
dstPort
date
time
And when I run the same command but with {print $2} at the end to get the values:
ssh 172.16.1.1 "grep -v '^#' /src/PTB.properties" | awk -F "=" '{print $2}'
I get the following output:
User-Name
Full-Name
Source-IP-Address
Source-UDP-Port
Target-UDP-Port
Date
Time
So what I want to say is that some keys don't have a value.
That being said, I am running the following command to get those keys and values:
while IFS= read -r line; do
USER_PARAMETERS_KEYS+=( "$line" )
done < <( ssh $ip "grep -v '^#' ${PARAMETER_PATH[$i]}" | awk -F "=" '{print $1}' )
while IFS= read -r line; do
USER_PARAMETERS_VALUES+=( "$line" )
done < <( ssh $ip "grep -v '^#' ${PARAMETER_PATH[$i]}" | awk -F "=" '{print $2}' )
My problem is that USER_PARAMETERS_VALUES is not saving the null values. For example after running this command, there are only 7 items in USER_PARAMETERS_VALUES instead of 8 because the Key dstIP is empty.
Any idea how can I solve this issue? Thanks.
Wouldn't it make more sense to capture the whole line and parse it locally?
while IFS== read -r key value; do
keys+=("$key")
values+=("$value")
done < <(ssh 172.16.1.1 "grep -v '^#' /src/PTB.properties")
This also avoids the race condition you get where the remote file could change between the first and the second time you read it.
Having said that, a minimal change would be to have Awk print quotes around any empty value.

Bash script to read from a file and save the information in an array?

I want to read from a file that has host IPs written in it and save it in an array. So far I have tried this:
Host=`cat /home/hp3385/Desktop/config | egrep '^Host' | awk '{print $2}'`
But I don't think that it saves the information in an array. What is the type of the variable 'Host'? If it's not an array how can I convert it into one?
This is a sample data from the file /home/hp3385/Desktop/config:
############# Server1 #################
Host 8.8.8.8
Hostname google
############# Server2 ################
Host 8.8.4.4
Hostname google
The expected output is:
a=($'8.8.8.8' $'8.8.4.4')
You can try this
myarray=()
while read -r line; do
if echo "$line" | grep -q 'Host '; then
myarray+=($(echo "$line" | awk '/^Host/ {print $2}'))
fi
done < /home/hp3385/Desktop/config
Declaring an array:
ARRAY=(0 1 2 3 4 5)
So your array can be declared like this:
HOSTS=($(awk '/^Host/ {print $2}' YOUR_FILE))
If you want to know the amount of values in your array:
echo ${#HOSTS[*]}
To get an output of all values in your array (credit goes to triplee):
printf '%s\n' "${HOSTS[#]}"

Shell Script : Assign the outputs to different variables

In a shell script I need to assign the output of few values to different varialbes, need help please.
cat file1.txt
uid: user1
cn: User One
employeenumber: 1234567
absJobAction: HIRED
I need to assign the value of each attribute to different variables so that I can call them them in script. For example uid should be assigned to a new variable name current_uid and when $current_uid is called it should give user1 and so forth for all other attributes.
And if the output does not contain any of the attributes then that attribute value should be considered as "NULL". Example if the output does not have absJobAction then the value of $absJobAction should be "NULL"
This is what I did with my array
#!/bin/bash
IFS=$'\n'
array=($(cat /tmp/file1.txt | egrep -i '^uid:|^cn:|^employeenumber|^absJobAction'))
current_uid=`echo ${array[0]} | grep -w uid | awk -F ': ' '{print $2}'`
current_cn=`echo ${array[1]} | grep -w cn | awk -F ': ' '{print $2}'`
current_employeenumber=`echo ${array[2]} | grep -w employeenumber | awk -F ': ' '{print $2}'`
current_absJobAction=`echo ${array[3]} | grep -w absJobAction | awk -F ': ' '{print $2}'`
echo $current_uid
echo $current_cn
echo $current_employeenumber
echo $current_absJobAction
Output from sh /tmp/testscript.sh follows:
user1
User One
1234567
HIRED
#!/usr/bin/env bash
# assuming bash 4.0 or newer: create an associative array
declare -A vars=( )
while IFS= read -r line; do ## See http://mywiki.wooledge.org/BashFAQ/001
if [[ $line = *": "* ]]; then ## skip lines not containing ": "
key=${line%%": "*} ## strip everything after ": " for key
value=${line#*": "} ## strip everything before ": " for value
vars[$key]=$value
else
printf 'Skipping unrecognized line: <%s>\n' "$line" >&2
fi
done <file1.txt # or < <(ldapsearch ...)
# print all variables read, just to demonstrate
declare -p vars >&2
# extract and print a single variable by name
echo "Variable uid has value ${vars[uid]}"
Note that this must be run with bash yourscript, not sh yourscript.
By the way -- if you don't have bash 4.0, you might consider a different approach:
while IFS= read -r line; do
if [[ $line = *": "* ]]; then
key=${line%%": "*}
value=${line#*": "}
printf -v "ldap_$key" %s "$value"
fi
done <file1.txt # or < <(ldapsearch ...)
will create separate variables of the form "$ldap_cn" or "$ldap_uid", as opposed to putting everything in a single associative array.
Here's a simple example of what you are trying to do that should get you started. It assumes 1 set of data in the file. Although a tad brute-force, I believe its easy to understand.
Given a file called file.txt in the current directory with the following contents (absJobAction intentionally left out):
$ cat file1.txt
uid: user1
cn: User One
employeenumber: 1234567
$
This script gets each value into a local variable and prints it out:
# Use /bin/bash to run this script
#!/bin/bash
# Make SOURCEFILE a readonly variable. Make it uppercase to show its a constant. This is the file the LDAP values come from.
typeset -r SOURCEFILE=./file1.txt
# Each line sets a variable using awk.
# -F is the field delimiter. It's a colon and a space.
# Next is the value to look for. ^ matches the start of the line.
# When the above is found, return the second field ($2)
current_uid="$(awk -F': ' '/^uid/ {print $2}' ${SOURCEFILE})"
current_cn="$(awk -F': ' '/^cn/ {print $2}' ${SOURCEFILE})"
current_enbr="$(awk -F': ' '/^employeenumber/ {print $2}' ${SOURCEFILE})"
current_absja="$(awk -F': ' '/^absJobAction/ {print $2}' ${SOURCEFILE})"
# Print the contents of the variables. Note since absJobAction was not in the file,
# it's value is NULL.
echo "uid: ${current_uid}"
echo "cn: ${current_cn}"
echo "EmployeeNumber: ${current_enbr}"
echo "absJobAction: ${current_absja}"
~
When run:
$ ./test.sh
uid: user1
cn: User One
EmployeeNumber: 1234567
absJobAction:
$

Set variable from awk while parsing lines from a multiline file

I've got a txt file with several lines, each one describing a remote server, like this:
user#server:port:remote_working_path:whether_using_VPN
The : char separates the 4 fields.
I need to operate batch actions within each server, hence I need to parse each line and set appropriate variables. Right now, what I've coded is this:
while read server;
do
echo "$server" | awk -F ':' '{print $1}' &&
echo "$server" | awk -F ':' '{print $2}' &&
echo "$server" | awk -F ':' '{print $3}'
echo "$VPN"
declare $( echo "$server" | awk -F ':' '{print $VPN=$4}' )
echo 'VPN: '$VPN
done < $CUSTOMER_SERVERS_FILE
This script only prints the first 3 fields, and in my intentions should also set $VPN variable as the 4th field. However this seems way broken, and I'm being unable to fix it. How should I modify it so that $VPN = $4?
First, you don't need to use awk in this case. You could try to use something like :
while IFS=':' read -ra array; do
# "${array[0]}" => first field
# "${array[1]}" => second field
# ...
# "${array[#]}" => all fields
done < "$CUSTOMER_SERVERS_FILE"
Then if you want to set VPN variable with the 4th field, you could use :
while IFS=':' read -ra array; do
# ...
VPN="${array[3]}"
done < "$CUSTOMER_SERVERS_FILE"
Another solution :
while IFS=':' read -r address port path vpn trash; do
# The variables $adress $port $path and $vpn are assigned.
# $trash is set with other fields if there are more than 4 fields
done
Finally, when you want to assign the output of a command in a variable, you could do :
var="$(command)"
# or
var="`command`"

How to add multiple line of output one by one to a variable in Bash?

This might be a very basic question but I was not able to find solution. I have a script:
If I run w | awk '{print $1}' in command line in my server I get:
f931
smk591
sc271
bx972
gaw844
mbihk988
laid640
smk59
ycc951
Now I need to use this list in my bash script one by one and manipulate some operation on them. I need to check their group and print those are in specific group. The command to check their group is id username. How can I save them or iterate through them one by one in a loop.
what I have so far is
tmp=$(w | awk '{print $1})
But it only return first record! Appreciate any help.
Populate an array with the output of the command:
$ tmp=( $(printf "a\nb\nc\n") )
$ echo "${tmp[0]}"
a
$ echo "${tmp[1]}"
b
$ echo "${tmp[2]}"
c
Replace the printf with your command (i.e. tmp=( $(w | awk '{print $1}') )) and man bash for how to work with bash arrays.
For a lengthier, more robust and complete example:
$ cat ./tstarrays.sh
# saving multi-line awk output in a bash array, one element per line
# See http://www.thegeekstuff.com/2010/06/bash-array-tutorial/ for
# more operations you can perform on an array and its elements.
oSET="$-"; set -f # save original set flags and turn off globbing
oIFS="$IFS"; IFS=$'\n' # save original IFS and make IFS a newline
array=( $(
awk 'BEGIN{
print "the quick brown"
print " fox jumped\tover\tthe"
print "lazy dogs back "
}'
) )
IFS="$oIFS" # restore original IFS value
set +f -$oSET # restore original set flags
for (( i=0; i < ${#array[#]}; i++ ));
do
printf "array[%d] of length=%d: \"%s\"\n" "$i" "${#array[$i]}" "${array[$i]}"
done
printf -- "----------\n"
printf -- "array[#]=\n\"%s\"\n" "${array[#]}"
printf -- "----------\n"
printf -- "array[*]=\n\"%s\"\n" "${array[*]}"
.
$ ./tstarrays.sh
array[0] of length=22: "the quick brown"
array[1] of length=23: " fox jumped over the"
array[2] of length=21: "lazy dogs back "
----------
array[#]=
"the quick brown"
array[#]=
" fox jumped over the"
array[#]=
"lazy dogs back "
----------
array[*]=
"the quick brown fox jumped over the lazy dogs back "
A couple of non-obvious key points to make sure your array gets populated with exactly what your command outputs:
If your command output can contain globbing characters than you should disable globbing before the command (oSET="$-"; set -f) and re-enable it afterwards (set +f -$oSET).
If your command output can contain spaces then set IFS to a newline before the command (oIFS="$IFS"; IFS=$'\n') and set it back to it's old value after the command (IFS="$oIFS").
tmp=$(w | awk '{print $1}')
while read i
do
echo "$i"
done <<< "$tmp"
You can use a for loop, i.e.
for user in $(w | awk '{print $1}'); do echo $user; done
which in a script would look nicer as:
for user in $(w | awk '{print $1}')
do
echo $user
done
You can use the xargs command to do this:
w | awk '{print $1}' | xargs -I '{}' id '{}'
With the -I switch, xargs will take each line of its standard input separately, then construct and execute a command line by replacing the specified string '{}' in the command line template with the input line
I guess you should use who instead of w. Try this out,
who | awk '{print $1}' | xargs -n 1 id

Resources