I have a txt file with blow format:
66.57.21 - john
88.43.23 - albert
10.10.11 - smith
I wanna to execute "connect.py 66.57.21 john" for each line and I wrote this bash script:
#!/bin/bash
while read LINE; do
awk -v number = "$LINE" '$1'
awk -v name = "$LINE" '$3'
connect.py $name $number
done < "$1"
but the bash script didn't work
What is the problem
#!/usr/bin/env bash
while read -r number _ name; do
connect.py "$name" "$number"
done < "$1"
If you are wanting to use awk, here is one way to do it:
awk -F" " '{system("connect.py " $3 " " $1)}' input.txt
The -F" " splits each line of input on spaces
$1 is the first word in the array (number in the original question)
$3 is he third word in the array (name in the original question)
wrapping "connect.py " $3 " " $1 in system() causes the shell to execute the command after the substitutions have been made
ie: connect.py john 66.57.21
Related
In a shell script I need to assign the output of few values to different varialbes, need help please.
cat file1.txt
uid: user1
cn: User One
employeenumber: 1234567
absJobAction: HIRED
I need to assign the value of each attribute to different variables so that I can call them them in script. For example uid should be assigned to a new variable name current_uid and when $current_uid is called it should give user1 and so forth for all other attributes.
And if the output does not contain any of the attributes then that attribute value should be considered as "NULL". Example if the output does not have absJobAction then the value of $absJobAction should be "NULL"
This is what I did with my array
#!/bin/bash
IFS=$'\n'
array=($(cat /tmp/file1.txt | egrep -i '^uid:|^cn:|^employeenumber|^absJobAction'))
current_uid=`echo ${array[0]} | grep -w uid | awk -F ': ' '{print $2}'`
current_cn=`echo ${array[1]} | grep -w cn | awk -F ': ' '{print $2}'`
current_employeenumber=`echo ${array[2]} | grep -w employeenumber | awk -F ': ' '{print $2}'`
current_absJobAction=`echo ${array[3]} | grep -w absJobAction | awk -F ': ' '{print $2}'`
echo $current_uid
echo $current_cn
echo $current_employeenumber
echo $current_absJobAction
Output from sh /tmp/testscript.sh follows:
user1
User One
1234567
HIRED
#!/usr/bin/env bash
# assuming bash 4.0 or newer: create an associative array
declare -A vars=( )
while IFS= read -r line; do ## See http://mywiki.wooledge.org/BashFAQ/001
if [[ $line = *": "* ]]; then ## skip lines not containing ": "
key=${line%%": "*} ## strip everything after ": " for key
value=${line#*": "} ## strip everything before ": " for value
vars[$key]=$value
else
printf 'Skipping unrecognized line: <%s>\n' "$line" >&2
fi
done <file1.txt # or < <(ldapsearch ...)
# print all variables read, just to demonstrate
declare -p vars >&2
# extract and print a single variable by name
echo "Variable uid has value ${vars[uid]}"
Note that this must be run with bash yourscript, not sh yourscript.
By the way -- if you don't have bash 4.0, you might consider a different approach:
while IFS= read -r line; do
if [[ $line = *": "* ]]; then
key=${line%%": "*}
value=${line#*": "}
printf -v "ldap_$key" %s "$value"
fi
done <file1.txt # or < <(ldapsearch ...)
will create separate variables of the form "$ldap_cn" or "$ldap_uid", as opposed to putting everything in a single associative array.
Here's a simple example of what you are trying to do that should get you started. It assumes 1 set of data in the file. Although a tad brute-force, I believe its easy to understand.
Given a file called file.txt in the current directory with the following contents (absJobAction intentionally left out):
$ cat file1.txt
uid: user1
cn: User One
employeenumber: 1234567
$
This script gets each value into a local variable and prints it out:
# Use /bin/bash to run this script
#!/bin/bash
# Make SOURCEFILE a readonly variable. Make it uppercase to show its a constant. This is the file the LDAP values come from.
typeset -r SOURCEFILE=./file1.txt
# Each line sets a variable using awk.
# -F is the field delimiter. It's a colon and a space.
# Next is the value to look for. ^ matches the start of the line.
# When the above is found, return the second field ($2)
current_uid="$(awk -F': ' '/^uid/ {print $2}' ${SOURCEFILE})"
current_cn="$(awk -F': ' '/^cn/ {print $2}' ${SOURCEFILE})"
current_enbr="$(awk -F': ' '/^employeenumber/ {print $2}' ${SOURCEFILE})"
current_absja="$(awk -F': ' '/^absJobAction/ {print $2}' ${SOURCEFILE})"
# Print the contents of the variables. Note since absJobAction was not in the file,
# it's value is NULL.
echo "uid: ${current_uid}"
echo "cn: ${current_cn}"
echo "EmployeeNumber: ${current_enbr}"
echo "absJobAction: ${current_absja}"
~
When run:
$ ./test.sh
uid: user1
cn: User One
EmployeeNumber: 1234567
absJobAction:
$
This might be a very basic question but I was not able to find solution. I have a script:
If I run w | awk '{print $1}' in command line in my server I get:
f931
smk591
sc271
bx972
gaw844
mbihk988
laid640
smk59
ycc951
Now I need to use this list in my bash script one by one and manipulate some operation on them. I need to check their group and print those are in specific group. The command to check their group is id username. How can I save them or iterate through them one by one in a loop.
what I have so far is
tmp=$(w | awk '{print $1})
But it only return first record! Appreciate any help.
Populate an array with the output of the command:
$ tmp=( $(printf "a\nb\nc\n") )
$ echo "${tmp[0]}"
a
$ echo "${tmp[1]}"
b
$ echo "${tmp[2]}"
c
Replace the printf with your command (i.e. tmp=( $(w | awk '{print $1}') )) and man bash for how to work with bash arrays.
For a lengthier, more robust and complete example:
$ cat ./tstarrays.sh
# saving multi-line awk output in a bash array, one element per line
# See http://www.thegeekstuff.com/2010/06/bash-array-tutorial/ for
# more operations you can perform on an array and its elements.
oSET="$-"; set -f # save original set flags and turn off globbing
oIFS="$IFS"; IFS=$'\n' # save original IFS and make IFS a newline
array=( $(
awk 'BEGIN{
print "the quick brown"
print " fox jumped\tover\tthe"
print "lazy dogs back "
}'
) )
IFS="$oIFS" # restore original IFS value
set +f -$oSET # restore original set flags
for (( i=0; i < ${#array[#]}; i++ ));
do
printf "array[%d] of length=%d: \"%s\"\n" "$i" "${#array[$i]}" "${array[$i]}"
done
printf -- "----------\n"
printf -- "array[#]=\n\"%s\"\n" "${array[#]}"
printf -- "----------\n"
printf -- "array[*]=\n\"%s\"\n" "${array[*]}"
.
$ ./tstarrays.sh
array[0] of length=22: "the quick brown"
array[1] of length=23: " fox jumped over the"
array[2] of length=21: "lazy dogs back "
----------
array[#]=
"the quick brown"
array[#]=
" fox jumped over the"
array[#]=
"lazy dogs back "
----------
array[*]=
"the quick brown fox jumped over the lazy dogs back "
A couple of non-obvious key points to make sure your array gets populated with exactly what your command outputs:
If your command output can contain globbing characters than you should disable globbing before the command (oSET="$-"; set -f) and re-enable it afterwards (set +f -$oSET).
If your command output can contain spaces then set IFS to a newline before the command (oIFS="$IFS"; IFS=$'\n') and set it back to it's old value after the command (IFS="$oIFS").
tmp=$(w | awk '{print $1}')
while read i
do
echo "$i"
done <<< "$tmp"
You can use a for loop, i.e.
for user in $(w | awk '{print $1}'); do echo $user; done
which in a script would look nicer as:
for user in $(w | awk '{print $1}')
do
echo $user
done
You can use the xargs command to do this:
w | awk '{print $1}' | xargs -I '{}' id '{}'
With the -I switch, xargs will take each line of its standard input separately, then construct and execute a command line by replacing the specified string '{}' in the command line template with the input line
I guess you should use who instead of w. Try this out,
who | awk '{print $1}' | xargs -n 1 id
In my Bash shell script, I would like to read a specific line from a file; that is delimited by : and assign each section to a variable for processing later.
For example I want to read the words found on line 2. The text file:
abc:01APR91:1:50
Jim:02DEC99:2:3
banana:today:three:0
Once I have "read" line 2, I should be able to echo the values as something like this:
echo "$name";
echo "$date";
echo "$number";
echo "$age";
The output would be:
Jim
02DEC99
2
3
For echoing a single line of a file, I quite like sed:
$ IFS=: read name date number age < <(sed -n 2p data)
$ echo $name
Jim
$ echo $date
02DEC99
$ echo $number
2
$ echo $age
3
$
This uses process substitution to get the output of sed to the read command. The sed command uses the -n option so it does not print each line (as it does by default); the 2p means 'when it is line 2, print the line'; data is simply the name of the file.
You can use this:
read name date number age <<< $(awk -F: 'NR==2{printf("%s %s %s %s\n", $1, $2, $3, $4)}' inFile)
echo "$name"
echo "$date"
echo "$number"
echo "$age"
I have a text file called db.txt.
Some sample lines from the file goes as such:
Harry Potter and the Sorcerer's Stone:J.K. Rowling:21.95:100:200
Harry Potter and the Chamber of Secrets:J.K. Rowling:21.95:150:300
Lord of the Rings, The Fellowship of the Ring:J.R.R. Tolkien:32.00:500:500
A Game of Thrones:George R.R. Martin:44.50:300:250
Then in my script, I have the following lines:
echo "Enter title:"
read TITLE
cut -d ":" -f 1 db.txt | grep -iw "$TITLE" | while read LINE
do
STRING="`echo $LINE | cut -d ":" -f 1`,"
STRING="$STRING `echo $LINE | cut -d ":" -f 2`, "
STRING=" \$$STRING`echo $LINE | cut -d ":" -f 3`,"
STRING=" $STRING`echo $LINE | cut -d ":" -f 4`,"
STRING=" $STRING`echo $LINE | cut -d ":" -f 5`"
done
Is there a way to grep a specific field from cut and then pass in the full line into the while loop?
For example, if I entered "Harry Potter",
it should display:
Harry Potter and the Sorcerer's Stone, J.K. Rowling, $21.95, 100, 200
Harry Potter and the Chamber of Secrets, J.K. Rowling, $21.95, 150, 300
You can do this without cut, and without grep if you're ok with bash's regular expression matching (or can use shell pattern matching instead).
The idea would be to read the file line by line, then split the line into an array.
Once you've got that, do the comparisons and output you want.
Here's a demo of the technique:
#! /bin/bash
echo "Title:"
read title
# shopt -s nocasematch # if you want case-insensitive matching
while read line ; do # this read takes data from input.txt, see
# end of loop
IFS=: read -a parts <<< "$line" # this splits the line on ":" into
# an array called parts
if [[ ${parts[0]} =~ $title ]] ; then # regex matching
printf "%s -- %s\n" "${parts[1]}" "${parts[2]}"
fi
done < input.txt
The next step up from grep and cut is awk. Unless you must do this using bash (is this homework?), then awk would make things considerably easier:
awk -F: '/harry potter/ { sub(/^/,"$",$(NF-2)); print }' IGNORECASE=1 OFS=", " db.txt
Test input:
Harry Potter and the Sorcerer's Stone:J.K. Rowling:21.95:100:200
Harry Potter and the Chamber of Secrets:J.K. Rowling:21.95:150:300
Lord of the Rings, The Fellowship of the Ring:J.R.R. Tolkien:32.00:500:500
A Game of Thrones:George R.R. Martin:44.50:300:250
Test output:
Harry Potter and the Sorcerer's Stone, J.K. Rowling, $21.95, 100, 200
Harry Potter and the Chamber of Secrets, J.K. Rowling, $21.95, 150, 300
read -p "Enter title: " TITLE
while IFS=: read title author price x y; do
if [[ ${title,,} == *${TITLE,,}* ]]; then
printf "%s, %s, $%s, %s, %s\n" "$title" "$author" "$price" "$x" "$y"
fi
done < db.txt
The test in the if command does a simple glob-match but case insensitively, so it will match if the user enters "potter".
Or, use sed to change the separators:
read -p "Enter title: " TITLE
sed '/'"$TITLE"'/I!d; s/:/, /g' db.txt
which means delete all lines that do not match the TITLE, then transform the separator.
The easiest method of doing this is to look over the grep results
#!/bin/bash
read -p "Enter title: " TITLE
FILENAME="db.txt"
IFS=$'\n'
for LINE in `grep -iw "Harry Potter" "$FILENAME"`; do
echo $LINE | awk 'BEGIN { FS = ":" } ; { print $1, $2, $3, $4, $5 }'
done
The IFS change changes the delimiter to a new line rather than a space and the FS in the awk command changes the delimiter to the : to allow access to the fields
I know you didn't specify it, but awk is probably the best tool to use for this task. It combines cut, sed, and grep into one convenient and easy to use tool. Well, convenient tool...
To understand awk, you have to understand a few things:
Awk is a programming language. It has built in logic and variables.
Awk assumes a read loop reading each and every line.
Awk programs must be surrounded by curly braces.
Not only curly braces, but Awk parsing variables start with dollar signs. Therefore, you need to put your Awk programs surrounded by single quotes to keep the shell out of it.
Awk automatically parses each line based upon the field separator. The default field separator is a while space, but you can change that via the -f parameter.
Each field gets a special variable. THe first field is $1, the next field is $2, etc. The entire line is $0.
Here's your Awk statement:
awk -F: '{
title = $1
author = $2
price = $3
pages_read_until_i_got_bored=$4
pages = $5
print "I read " pages_read_until_i_gob_bored "pages out of " $pages " pages of " $title " by " $author "."
}' $file
Of course, the whole thing could be a single line too:
awk -F: '{ print "I read " $4 " pages " out of " $5 " of " $1 " by " $2 "." }' $file
Just wanted to emphasize the programability of Awk and how it can be used to do this type of parsing.
If your question is how to enter this information and put it into environment variables, Glenn Jackman's answer is the best.
If you can use sed this would be a solution
read -p "Enter title: " TITLE
sed -n -e 's/^\([^:]\+:\)\{2\}/\0$/' -e 's/:/, /g' -e "/^$TITLE/Ip" db.txt
Short explanation what it does
-n tells sed not to print any lines
-e 's/^\([^:]\+:\)\{2\}/\0$/' matches for the 2nd : and adds a $ after it
-e 's/:/, /g' replaces all : with , and a following whitespace
-e "/^$TITLE/Ip" tells sed to print all lines which start with $TITLE (that's the p) and I tells sed to match case-insensitive
Here is a script which reads words from the file replaced.txt and displays the output each word in each line, But I want to display all the outputs in a single line.
#!/bin/sh
echo
echo "Enter the word to be translated"
read a
IFS=" " # Set the field separator
set $a # Breaks the string into $1, $2, ...
for a # a for loop by default loop through $1, $2, ...
do
{
b= grep "$a" replaced.txt | cut -f 2 -d" "
}
done
Content of "replaced.txt" file is given below:
hllo HELLO
m AM
rshbh RISHABH
jn JAIN
hw HOW
ws WAS
ur YOUR
dy DAY
This question can't be appropriate to what I asked, I just need the help to put output of the script in a single line.
Your entire script can be replaced by:
#!/bin/bash
echo
read -r -p "Enter the words to be translated: " a
echo $(printf "%s\n" $a | grep -Ff - replaced.txt | cut -f 2 -d ' ')
No need for a loop.
The echo with an unquoted argument removes embedded newlines and replaces each sequence of multiple spaces and/or tabs with one space.
One hackish-but-simple way to remove trailing newlines from the output of a command is to wrap it in printf %s "$(...) ". That is, you can change this:
b= grep "$a" replaced.txt | cut -f 2 -d" "
to this:
printf %s "$(grep "$a" replaced.txt | cut -f 2 -d" ") "
and add an echo command after the loop completes.
The $(...) notation sets up a "command substitution": the command grep "$a" replaced.txt | cut -f 2 -d" " is run in a subshell, and its output, minus any trailing newlines, is substituted into the argument-list. So, for example, if the command outputs DAY, then the above is equivalent to this:
printf %s "DAY "
(The printf %s ... notation is equivalent to echo -n ... — it outputs a string without adding a trailing newline — except that its behavior is more portably consistent, and it won't misbehave if the string you want to print happens to start with -n or -e or whatnot.)
You can also use
awk 'BEGIN { OFS=": "; ORS=" "; } NF >= 2 { print $2; }'
in a pipe after the cut.