I need some help writing a script that will compare the value(s) from running awk to user input.
i.e.:
cat employee.txt
100 Thomas Manager Sales $5,000
200 Jason Developer Technology $5,500
300 Sanjay Sysadmin Technology $7,000
400 Nisha Manager Marketing $9,500
500 Randy DBA Technology $6,000
EMPLOYERS=$(awk -F" " '{ print $2}' employee.txt)
echo "Enter the name of an employer: \c"
read employer
If/for/while employer is part of EMPLOYERS, echo a message.
Thanks in advance
or simply
...
awk -v emp=$employer '$2==emp{print "found"}' employee.txt
-v flag is for defining awk variables, here to pass shell value to script.
I would do it simpler:
echo "Enter the name of an employer: \c"
read employer
if [[ $(awk -F" " '{ print $2}' employee.txt | grep $employer) = $employer ]]
then
echo Found
fi
Related
this really has me stumped. Here is what I am trying to do:
I try to pipe an article from newsboat to a script. This script should then extract the Title and Url from the article.
Here is an example article:
Feed: NYT > Home Page
Title: Hit Pause on Brett Kavanaugh
Author: THE EDITORIAL BOARD
Link: https://www.nytimes.com/2018/09/26/opinion/kavanaugh-supreme-court-hearing-delay.html?partner=rss&emc=rss
Date: Thu, 27 Sep 2018 01:58:11 +0200
The integrity of the Supreme Court is at stake.
The article gets piped with a macro from newsboat:
macro R pipe-to "cat | ~/.scripts/newsboat_extract"
Here is the working script:
#!/bin/bash
cat > ~/newsboat #I do not really need this file, so if I can cut out saving to a file, I would prefer to
title="$(awk -F: '/^Title:/{for(i=2;i<=NF;++i)print $i}' ~/newsboat)"
url="$(awk -F: '/^Link:/{print $2 ":" $3}' ~/newsboat)"
printf '%s\n' "$title" "$url" >> newsboat_result
This delivers the expected output:
Hit Pause on Brett Kavanaugh
https://www.nytimes.com/2018/09/26/opinion/kavanaugh-supreme-court-hearing-delay.html?partner=rss&emc=rss
I would like to avoid saving to a file. However, saving to a variable does - for whatever reason - not work: And this is the script that is not working!
#!/bin/bash
article=$(cat)
title="$(awk -F: '/^Title:/{for(i=2;i<=NF;++i)print $i}' "$article")"
url="$(awk -F: '/^Link:/{print $2 ":" $3}' "$article")"
printf '%s\n' "$title" "$url" >> newsboat_result
the output turns to this:
#empty line
#empty line
I have completely no idea why the script would behave like this. It must have something to do how the variable is stored, right?
Any ideas? - I am pretty new at bash scripting and awk, so thankful also for any comments on how to solve this problem more efficiently.
""""""""""""
" SOLUTION "
""""""""""""
This did it, thank you!
#!/bin/bash
article=$(cat "${1:--}")
title="$(awk -F: '/^Title:/{for(i=2;i<=NF;++i)print $i}' <<< "$article")"
url="$(awk -F: '/^Link:/{print $2 ":" $3}' <<< "$article")"
printf '%s\n' "$title" "$url" >> newsboat_result
In your script, you are assuming that $ARTICLE is a plain file and you are making several operations on it. First you read it with cat and store the content in ~/newsboat, then you read it again with awk to extract the title, then you read it a third time to extract the URL.
This can't work with standard input; it can only be read once.
A quick fix is to work on the copy of it you made in the first operation:
#!/bin/bash
article=$1
feed_copy=~/newsboat
cat "${article:--}" > "$feed_copy" # Use stdin if parameter is not provided
title="$(awk -F: '/^Title:/ { for(i=2; i<=NF; ++i) print $i }' "$feed_copy")"
url="$(awk -F: '/^Link:/ { print $2 ":" $3 }' "$feed_copy")"
printf '%s\n' "$title" "$url" >> "$feed_copy"
Not tested, obviously, but that should work.
Notes:
reserve uppercase variable names for environment variables (this is a mere convention)
you should almost always quote your variables (cat "$article", not cat $article) unless you know what you are doing
avoid echo, use printf
There are other enhancements that could be made to this script but sorry, I lack the time.
[edit] Since you don't actually need the ~/newsboat file, here is a updated version that follows Charles Duffy's suggestion:
#!/bin/bash
feed_copy=$(cat "${1:--}")
title="$(awk -F: '/^Title:/ { for(i=2; i<=NF; ++i) print $i }' <<< "$feed_copy")"
url="$(awk -F: '/^Link:/ {print $2 ":" $3}' <<< "$feed_copy")"
printf '%s\n' "$title" "$url"
I have file a file named as file.yaml with the below content :
SystemType=Secondary-HA
Hostname America
I have a shell script filter.sh:
echo "Enter systemType:"
read systemType
SYSTYPE=`grep systemType /home/oracle/file.yaml | awk '{print $2}'`
if [ SYSTYPE=Secondary-HA ]
then
cat /home/oracle/file.yaml > /home/oracle/file2.txt
fi
hstname=`grep Hostname /home/oracle/file2.txt | awk '{print $2}'`
echo $hstname
Here I want to given the systemType only 'Secondary-HA' then only I should get the result as America. But as of now if I am giving the systemType 'Secondary', I am giving the same result as America. which I don't want.
Please let know. I am bit new in shell scripting.
You need to understand that the shell is white-space sensitive in certain places, for example when splitting arguments. Thus,
if [ x=y ]
must be written as
if [ x = y ]
In addition, I have replaced the anti-pattern
grep xyz file | awk '{print $2}'
with the much less expensive pipe-less
awk '/xyz/ {print $2}' file
Next, awk splits at white-space by default, not by =. You would need to say awk -F= to split at =. I have also uppercased systemType to SystemType, since that's what you tell us is in your yaml file. Mate, you need to be careful if you want to be in programming.
Here is the result:
echo "Enter systemType:"
read systemType
SYSTYPE=$(awk -F= '/SystemType/ {print $2}' /home/oracle/file.yaml)
if [ "$SYSTYPE" = "$systemType" ]; then
cp /home/oracle/file.yaml /home/oracle/file2.txt
fi
hstname=$(awk '/Hostname/ {print $2}' /home/oracle/file2.txt)
echo $hstname
I think I may have approached this problem the wrong way and I could really use a hand here.
I'm trying to print a report to the screen using awk. I want to list the users logged in and their full names next to each user. The only way I could figure it out is below, but it only shows my own info. Can I add it into a loop somehow to achieve this or did I go about it completely wrong?
This is what I have:
echo "User name" "|" "Full name"
echo "--------------------------"
echo -n "$USER " ; awk -v user="$USER" -F":" 'user==$1{print$5}' /etc/passwd
The $USER variable just contains your username.
You can pipe the who command to get the list of logged in users.
echo "User name" "|" "Full name"
echo "--------------------------"
who | while read username rest; do
echo -n "$username " ; awk -v user="$username" -F":" 'user==$1{print$5}' /etc/passwd
done
Whenever you find yourself writing a loop in shell to process text, you have the wrong solution.
who | awk '
BEGIN { print "User name|Full name\n--------------------------" }
NR==FNR { name[$1] = $5; next }
{ print $1, name[$1] }
' FS=":" /etc/passwd FS=" " -
Shell is just an environment from which to call tools and it has a language to sequence those calls. The shell tool to process text is awk.
At the moment here is what im doing
ret=$(ls -la | awk '{print $3 " " $9}')
usr=$(echo $ret | awk '{print $1}')
fil=$(echo $ret | awk '{print $2}')
The problem is that im not running an ls im running a command that takes time, so you can understand the logic.
Is there a way I can set the return value to set two external values, so something such as
ls -la | awk -r usr=x -r fil=y '{x=$3; y=$9}'
This way the command will be run once and i can minimize it to one line
It's not pretty, but if you really need to do this in one line you can use awk/bash's advanced meta-programming capabilities :)
eval $(ls -la | awk '{usr = $3 " " usr;fil = $9 " " fil} END{print "usr=\""usr"\";fil=\""fil"\""}')
To print:
echo -e $usr
echo -e $fil
Personally, I'd stick with what you have - it's much more readable and performance overhead is tiny compared to the above:
$time <three line approach>
real 0m0.017s
user 0m0.006s
sys 0m0.011s
$time <one line approach>
real 0m0.009s
user 0m0.004s
sys 0m0.007s
A workaround using read
usr=""
fil=""
while read u f; do usr="$usr\n$u"; fil="$fil\n$f"; done < <(ls -la | awk '{print $3 " " $9}')
For performance issue you could use <<<, but avoid it if the returned text is large:
while read u f; do usr="$usr\n$u"; fil="$fil\n$f"; done <<< $(ls -la | awk '{print $3 " " $9}')
A more portable way inspired from #WilliamPursell's answer:
$ usr=""
$ fil=""
$ while read u f; do usr="$usr\n$u"; fil="$fil\n$f"; done << EOF
> $(ls -la | awk '{print $3 " " $9}')
> EOF
What you want to do is capture the output of ls or any other command and then process it later.
ls=$(ls -l)
first=$(echo $ls | awk '{print $1}')
second=$(echo $ls | awk '{print $2}')
Using bash v4 associative array:
unset FILES
declare -A FILES
FILES=( ls -la | awk '{print $9 " " $3}' )
Print the list of owner & file:
for fil in ${!FILES[#]}
do
usr=${FILES["$fil"]}
echo -e "$usr" "\t" "$fil"
done
My apologies, I cannot test on my computer because my bash v3.2 does not support associative array :-(.
Please, report any issue...
The accepted answer uses process substitution, which is a bashism that only works on certain platforms. A more portable solution is to use a heredoc:
read u f << EOF
$( ls ... )
EOF
It is tempting to try:
ls ... | read u f
but the read then runs in a subshell. A common technique is:
ls ... | { read u f; # use $u and $f here; }
but to make the variables available in the remainder of the script, the interpolated heredoc is the most portable approach. Note that it requires the shell to read all of the output of the program into memory, so is not suitable if the output is expected to be large or if the process is long running.
You could use a bash array or the positional parameters as temporary holding place:
ret_ary=( $(command | awk '{print $3, $9}') )
usr=${ret_ary[0]}
fil=${ret_ary[1]}
set -- $(command | awk '{print $3, $9}')
usr=$1
fil=$2
I want to write a script to check for duplicates
For example: I have a text file with information in the format of /etc/passwd
alice:x:1008:555:William Williams:/home/bill:/bin/bash
bob:x:1018:588:Bobs Boos:/home/bob:/bin/bash
bob:x:1019:528:Robt Ross:/home/bob:/bin/bash
james:x:1012:518:Tilly James:/home/bob:/bin/bash
I want to simply check if there are duplicate users and if there are, output the line to standard error. So in the example above since bob appears twice my output would simply generate something like:
Error duplicate user
bob:x:1018:588:Bobs Boos:/home/bob:/bin/bash
bob:x:1019:528:Robt Ross:/home/bob:/bin/bash
Right now I have a while loop that reads each line and stores each piece of information in a variable using awk -F that is delimited with ":". After storing my username I am not too sure on the best approach to check to see if it already exists.
Some parts of my code:
while read line; do
echo $line
user=`echo $line | awk -F : '{print $1}'`
match=`grep $user $1`($1 is the txtfile)
if [ $? -ne 0 ]; then
echo "Unique user"
else
echo "Not unique user"
then somehow grep those lines and output it
fi
done
The matching does not produce the right results
Suggestions?
instead of re-inventing the wheel, use the following tools:
cut to extract first field
sort and uniq to keep duplicated lines only.
cut -d : -f 1 | sort | uniq -d | while read i ; do
echo "error: duplicate user $i"
done
Sounds like a job for awk to me:
% awk -F':' '
/:/ {
count[$1] += 1
}
END {
for (user in count) {
if (count[user] > 1) {
print user " appears in the file " count[user] " times."
}
}
}
' /etc/passwd
A perl-proposal:
perl -F: -lanE 'push #{$h{$F[0]}},$_; END{for $k (keys %h){if(#{$h{$k}}>1){say "Error";say for #{$h{$k}}}}}' file