Shell Script : Assign the outputs to different variables - shell

In a shell script I need to assign the output of few values to different varialbes, need help please.
cat file1.txt
uid: user1
cn: User One
employeenumber: 1234567
absJobAction: HIRED
I need to assign the value of each attribute to different variables so that I can call them them in script. For example uid should be assigned to a new variable name current_uid and when $current_uid is called it should give user1 and so forth for all other attributes.
And if the output does not contain any of the attributes then that attribute value should be considered as "NULL". Example if the output does not have absJobAction then the value of $absJobAction should be "NULL"
This is what I did with my array
#!/bin/bash
IFS=$'\n'
array=($(cat /tmp/file1.txt | egrep -i '^uid:|^cn:|^employeenumber|^absJobAction'))
current_uid=`echo ${array[0]} | grep -w uid | awk -F ': ' '{print $2}'`
current_cn=`echo ${array[1]} | grep -w cn | awk -F ': ' '{print $2}'`
current_employeenumber=`echo ${array[2]} | grep -w employeenumber | awk -F ': ' '{print $2}'`
current_absJobAction=`echo ${array[3]} | grep -w absJobAction | awk -F ': ' '{print $2}'`
echo $current_uid
echo $current_cn
echo $current_employeenumber
echo $current_absJobAction
Output from sh /tmp/testscript.sh follows:
user1
User One
1234567
HIRED

#!/usr/bin/env bash
# assuming bash 4.0 or newer: create an associative array
declare -A vars=( )
while IFS= read -r line; do ## See http://mywiki.wooledge.org/BashFAQ/001
if [[ $line = *": "* ]]; then ## skip lines not containing ": "
key=${line%%": "*} ## strip everything after ": " for key
value=${line#*": "} ## strip everything before ": " for value
vars[$key]=$value
else
printf 'Skipping unrecognized line: <%s>\n' "$line" >&2
fi
done <file1.txt # or < <(ldapsearch ...)
# print all variables read, just to demonstrate
declare -p vars >&2
# extract and print a single variable by name
echo "Variable uid has value ${vars[uid]}"
Note that this must be run with bash yourscript, not sh yourscript.
By the way -- if you don't have bash 4.0, you might consider a different approach:
while IFS= read -r line; do
if [[ $line = *": "* ]]; then
key=${line%%": "*}
value=${line#*": "}
printf -v "ldap_$key" %s "$value"
fi
done <file1.txt # or < <(ldapsearch ...)
will create separate variables of the form "$ldap_cn" or "$ldap_uid", as opposed to putting everything in a single associative array.

Here's a simple example of what you are trying to do that should get you started. It assumes 1 set of data in the file. Although a tad brute-force, I believe its easy to understand.
Given a file called file.txt in the current directory with the following contents (absJobAction intentionally left out):
$ cat file1.txt
uid: user1
cn: User One
employeenumber: 1234567
$
This script gets each value into a local variable and prints it out:
# Use /bin/bash to run this script
#!/bin/bash
# Make SOURCEFILE a readonly variable. Make it uppercase to show its a constant. This is the file the LDAP values come from.
typeset -r SOURCEFILE=./file1.txt
# Each line sets a variable using awk.
# -F is the field delimiter. It's a colon and a space.
# Next is the value to look for. ^ matches the start of the line.
# When the above is found, return the second field ($2)
current_uid="$(awk -F': ' '/^uid/ {print $2}' ${SOURCEFILE})"
current_cn="$(awk -F': ' '/^cn/ {print $2}' ${SOURCEFILE})"
current_enbr="$(awk -F': ' '/^employeenumber/ {print $2}' ${SOURCEFILE})"
current_absja="$(awk -F': ' '/^absJobAction/ {print $2}' ${SOURCEFILE})"
# Print the contents of the variables. Note since absJobAction was not in the file,
# it's value is NULL.
echo "uid: ${current_uid}"
echo "cn: ${current_cn}"
echo "EmployeeNumber: ${current_enbr}"
echo "absJobAction: ${current_absja}"
~
When run:
$ ./test.sh
uid: user1
cn: User One
EmployeeNumber: 1234567
absJobAction:
$

Related

Read multiple variables from file

I need to read a file that has lines like
user=username1
pass=password1
How can I read multiple lines like this into separate variables like username and password?
Would I use awk or grep? I have found ways to read lines into variables with grep but would I need to read the file for each individual item?
The end result is to use these variables to access a database via the command line. So I need to be able to read, store and use these values in other commands.
if the process which generates the file is safe and has shell syntax just source the file.
. ./file
Otherwise the file can be processes before to add quotes
perl -ne 'if (/^([A-Za-z_]\w*)=(.*)/) {$k=$1;$v=$2;$v=~s/\x27/\x27\\\x27\x27/g;print "$k=\x27$v\x27\n";}' <file >file2
. ./file2
If you want to use awk then
Input
$ cat file
user=username1
pass=password1
Reading
$ user=$(awk -F= '$1=="user"{print $2;exit}' file)
$ pass=$(awk -F= '$1=="pass"{print $2;exit}' file)
Output
$ echo $user
username1
$ echo $pass
password1
You could use a loop for your file perhaps, but this is probably the functionality you're looking for.
$ echo 'user=username1' | awk -F= '{print $2}'
username1
Using the -F flag sets the delimiter to = and we select the 2nd item from the row.
file.txt:
user=username1
pass=password1
user=username2
pass=password2
user=username3
pass=password3
Do to avoid browsing several times the file file.txt:
#!/usr/bin/env bash
func () {
echo "user:$1 pass:$2"
}
i=0
while IFS='' read -r line; do
if [ $i -eq 0 ]; then
i=1
user=$(echo ${line} | cut -f2 -d'=')
else
i=0
pass=$(echo ${line} | cut -f2 -d'=')
func "$user" "$pass"
fi
done < file.txt
Output:
user:username1 pass:password1
user:username2 pass:password2
user:username3 pass:password3

Unix file pattern issue: append changing value of variable pattern to copies of matching line

I have a file with contents:
abc|r=1,f=2,c=2
abc|r=1,f=2,c=2;r=3,f=4,c=8
I want a result like below:
abc|r=1,f=2,c=2|1
abc|r=1,f=2,c=2;r=3,f=4,c=8|1
abc|r=1,f=2,c=2;r=3,f=4,c=8|3
The third column value is r value. A new line would be inserted for each occurrence.
I have tried with:
for i in `cat $xxxx.txt`
do
#echo $i
live=$(echo $i | awk -F " " '{print $1}')
home=$(echo $i | awk -F " " '{print $2}')
echo $live
done
but is not working properly. I am a beginner to sed/awk and not sure how can I use them. Can someone please help on this?
awk to the rescue!
$ awk -F'[,;|]' '{c=0;
for(i=2;i<=NF;i++)
if(match($i,/^r=/)) a[c++]=substr($i,RSTART+2);
delim=substr($0,length($0))=="|"?"":"|";
for(i=0;i<c;i++) print $0 delim a[i]}' file
abc|r=1,f=2,c=2|1
abc|r=1,f=2,c=2;r=3,f=4,c=8|1
abc|r=1,f=2,c=2;r=3,f=4,c=8|3
Use an inner routine (made up of GNU grep, sed, and tr) to compile a second more elaborate sed command, the output of which needs further cleanup with more sed. Call the input file "foo".
sed -n $(grep -no 'r=[0-9]*' foo | \
sed 's/^[0-9]*/&s#.*#\&/;s/:r=/|/;s/.*/&#p;/' | \
tr -d '\n') foo | \
sed 's/|[0-9|]*|/|/'
Output:
abc|r=1,f=2,c=2|1
abc|r=1,f=2,c=2;r=3,f=4,c=8|1
abc|r=1,f=2,c=2;r=3,f=4,c=8|3
Looking at the inner sed code:
grep -no 'r=[0-9]*' foo | \
sed 's/^[0-9]*/&s#.*#\&/;s/:r=/|/;s/.*/&#p;/' | \
tr -d '\n'
It's purpose is to parse foo on-the-fly (when foo changes, so will the output), and in this instance come up with:
1s#.*#&|1#p;2s#.*#&|1#p;2s#.*#&|3#p;
Which is almost perfect, but it leaves in old data on the last line:
sed -n '1s#.*#&|1#p;2s#.*#&|1#p;2s#.*#&|3#p;' foo
abc|r=1,f=2,c=2|1
abc|r=1,f=2,c=2;r=3,f=4,c=8|1
abc|r=1,f=2,c=2;r=3,f=4,c=8|1|3
...which old data |1 is what the final sed 's/|[0-9|]*|/|/' removes.
Here is a pure bash solution. I wouldn't recommend actually using this, but it might help you understand better how to work with files in bash.
# Iterate over each line, splitting into three fields
# using | as the delimiter. (f3 is only there to make
# sure a trailing | is not included in the value of f2)
while IFS="|" read -r f1 f2 f3; do
# Create an array of variable groups from $f2, using ;
# as the delimiter
IFS=";" read -a groups <<< "$f2"
for group in "${groups[#]}"; do
# Get each variable from the group separately
# by splitting on ,
IFS=, read -a vars <<< "$group"
for var in "${vars[#]}"; do
# Split each assignment on =, create
# the variable for real, and quit once we
# have found r
IFS== read name value <<< "$var"
declare "$name=$value"
[[ $name == r ]] && break
done
# Output the desired line for the current value of r
printf '%s|%s|%s\n' "$f1" "$f2" "$r"
done
done < $xxxx.txt
Changes for ksh:
read -A instead of read -a.
typeset instead of declare.
If <<< is a problem, you can use a here document instead. For example:
IFS=";" read -A groups <<EOF
$f2
EOF

Set variable from awk while parsing lines from a multiline file

I've got a txt file with several lines, each one describing a remote server, like this:
user#server:port:remote_working_path:whether_using_VPN
The : char separates the 4 fields.
I need to operate batch actions within each server, hence I need to parse each line and set appropriate variables. Right now, what I've coded is this:
while read server;
do
echo "$server" | awk -F ':' '{print $1}' &&
echo "$server" | awk -F ':' '{print $2}' &&
echo "$server" | awk -F ':' '{print $3}'
echo "$VPN"
declare $( echo "$server" | awk -F ':' '{print $VPN=$4}' )
echo 'VPN: '$VPN
done < $CUSTOMER_SERVERS_FILE
This script only prints the first 3 fields, and in my intentions should also set $VPN variable as the 4th field. However this seems way broken, and I'm being unable to fix it. How should I modify it so that $VPN = $4?
First, you don't need to use awk in this case. You could try to use something like :
while IFS=':' read -ra array; do
# "${array[0]}" => first field
# "${array[1]}" => second field
# ...
# "${array[#]}" => all fields
done < "$CUSTOMER_SERVERS_FILE"
Then if you want to set VPN variable with the 4th field, you could use :
while IFS=':' read -ra array; do
# ...
VPN="${array[3]}"
done < "$CUSTOMER_SERVERS_FILE"
Another solution :
while IFS=':' read -r address port path vpn trash; do
# The variables $adress $port $path and $vpn are assigned.
# $trash is set with other fields if there are more than 4 fields
done
Finally, when you want to assign the output of a command in a variable, you could do :
var="$(command)"
# or
var="`command`"

How to add multiple line of output one by one to a variable in Bash?

This might be a very basic question but I was not able to find solution. I have a script:
If I run w | awk '{print $1}' in command line in my server I get:
f931
smk591
sc271
bx972
gaw844
mbihk988
laid640
smk59
ycc951
Now I need to use this list in my bash script one by one and manipulate some operation on them. I need to check their group and print those are in specific group. The command to check their group is id username. How can I save them or iterate through them one by one in a loop.
what I have so far is
tmp=$(w | awk '{print $1})
But it only return first record! Appreciate any help.
Populate an array with the output of the command:
$ tmp=( $(printf "a\nb\nc\n") )
$ echo "${tmp[0]}"
a
$ echo "${tmp[1]}"
b
$ echo "${tmp[2]}"
c
Replace the printf with your command (i.e. tmp=( $(w | awk '{print $1}') )) and man bash for how to work with bash arrays.
For a lengthier, more robust and complete example:
$ cat ./tstarrays.sh
# saving multi-line awk output in a bash array, one element per line
# See http://www.thegeekstuff.com/2010/06/bash-array-tutorial/ for
# more operations you can perform on an array and its elements.
oSET="$-"; set -f # save original set flags and turn off globbing
oIFS="$IFS"; IFS=$'\n' # save original IFS and make IFS a newline
array=( $(
awk 'BEGIN{
print "the quick brown"
print " fox jumped\tover\tthe"
print "lazy dogs back "
}'
) )
IFS="$oIFS" # restore original IFS value
set +f -$oSET # restore original set flags
for (( i=0; i < ${#array[#]}; i++ ));
do
printf "array[%d] of length=%d: \"%s\"\n" "$i" "${#array[$i]}" "${array[$i]}"
done
printf -- "----------\n"
printf -- "array[#]=\n\"%s\"\n" "${array[#]}"
printf -- "----------\n"
printf -- "array[*]=\n\"%s\"\n" "${array[*]}"
.
$ ./tstarrays.sh
array[0] of length=22: "the quick brown"
array[1] of length=23: " fox jumped over the"
array[2] of length=21: "lazy dogs back "
----------
array[#]=
"the quick brown"
array[#]=
" fox jumped over the"
array[#]=
"lazy dogs back "
----------
array[*]=
"the quick brown fox jumped over the lazy dogs back "
A couple of non-obvious key points to make sure your array gets populated with exactly what your command outputs:
If your command output can contain globbing characters than you should disable globbing before the command (oSET="$-"; set -f) and re-enable it afterwards (set +f -$oSET).
If your command output can contain spaces then set IFS to a newline before the command (oIFS="$IFS"; IFS=$'\n') and set it back to it's old value after the command (IFS="$oIFS").
tmp=$(w | awk '{print $1}')
while read i
do
echo "$i"
done <<< "$tmp"
You can use a for loop, i.e.
for user in $(w | awk '{print $1}'); do echo $user; done
which in a script would look nicer as:
for user in $(w | awk '{print $1}')
do
echo $user
done
You can use the xargs command to do this:
w | awk '{print $1}' | xargs -I '{}' id '{}'
With the -I switch, xargs will take each line of its standard input separately, then construct and execute a command line by replacing the specified string '{}' in the command line template with the input line
I guess you should use who instead of w. Try this out,
who | awk '{print $1}' | xargs -n 1 id

Bash script read specifc value from files of an entire folder

I have a problem creating a script that reads specific value from all the files of an entire folder
I have a number of email files in a directory and I need to extract from each file, 2 specific values.
After that I have to put them into a new file that looks like that:
--------------
To: value1
value2
--------------
This is what I want to do, but I don't know how to create the script:
# I am putting the name of the files into a temp file
`ls -l | awk '{print $9 }' >tmpfile`
# use for the name of a file
`date=`date +"%T"
# The first specific value from file (phone number)
var1=`cat tmpfile | grep "To: 0" | awk '{print $2 }' | cut -b -10 `
# The second specific value from file(subject)
var2=cat file | grep Subject | awk '{print $2$3$4$5$6$7$8$9$10 }'
# Put the first value in a new file on the first row
echo "To: 4"$var1"" > sms-$date
# Put the second value in the same file on the second row
echo ""$var2"" >>sms-$date
.......
and do the same for every file in the directory
I tried using while and for functions but I couldn't finalize the script
Thank You
I've made a few changes to your script, hopefully they will be useful to you:
#!/bin/bash
for file in *; do
var1=$(awk '/To: 0/ {print substr($2,0,10)}' "$file")
var2=$(awk '/Subject/ {for (i=2; i<=10; ++i) s=s$i; print s}' "$file")
outfile="sms-"$(date +"%T")
i=0
while [ -f "$outfile" ]; do outfile="sms-$date-"$((i++)); done
echo "To: 4$var1" > "$outfile"
echo "$var2" >> "$outfile"
done
The for loop just goes through every file in the folder that you run the script from.
I have added added an additional suffix $i to the end of the file name. If no file with the same date already exists, then the file will be created without the suffix. Otherwise the value of $i will keep increasing until there is no file with the same name.
I'm using $( ) rather than backticks, this is just a personal preference but it can be clearer in my opinion, especially when there are other quotes about.
There's not usually any need to pipe the output of grep to awk. You can do the search in awk using the / / syntax.
I have removed the cut -b -10 and replaced it with substr($2, 0, 10), which prints the first 10 characters from column 2.
It's not much shorter but I used a loop rather than the $2$3..., I think it looks a bit neater.
There's no need for all the extra " in the two output lines.
I sugest to try the following:
#!/bin/sh
RESULT_FILE=sms-`date +"%T"`
DIR=.
fgrep -l 'To: 0' "$DIR" | while read FILE; do
var1=`fgrep 'To: 0' "$FILE" | awk '{print $2 }' | cut -b -10`
var2=`fgrep 'Subject' "$FILE" | awk '{print $2$3$4$5$6$7$8$9$10 }'`
echo "To: 4$var1" >>"$RESULT_FIL"
echo "$var2" >>"$RESULT_FIL"
done

Resources