Loops in Shell Scripting [closed] - bash

This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, visit the help center.
Closed 10 years ago.
I need help with this shell script.
Must use a loop of some sort.
Must use input data exactly as shown.
Output redirection should be accomplished within the script, not on the command line.
Here's the input files I have:
http://pastebin.com/m3f783597
Here's what the output needs to be:
http://pastebin.com/m2c53b25a
Here's my failed attempt:
http://pastebin.com/m2c60b41
And that failed attempt's output:
http://pastebin.com/m3460e78c

Here's the help. Try to follow these as much as possible before looking at my solution below. That will help you out more in the long run, and in the short runsince it's a certainty that your educator can see this as easily as you can.
If he finds you've plagiarized code, it will probably mean an instant fail.
Your "failed attempt" as you put it is here. It's actually not too bad for a first attempt.
echo -e "Name\t\t On-Call\t\t Phone"
for daycount in 2 1 4 5 7 6 3
do
for namecount in 3 2 6 1 7 4 5
do
day=`head -n $daycount p2input2|tail -n 1|cut -f 2 -d " "`
name=`head -n $namecount p2input1|tail -n 1|cut -f 1 -d " "`
phone=`head -n $namecount p2input1|tail -n 1|cut -f 2 -d " "`
echo -e "$name\c"
echo -e "\t\t$day\c"
echo -e "\t\t$phone"
continue
done
done
And here's the hints:
You have two loops, one inside the other, each occurring 7 times. That means 49 lines of output rather than 7. You want to process each day and look up up name and phone for that day (actually name for that day and phone for that name).
It's not really suitable hardcoding linenumbers (although I admit it is sneaky)- what if the order of data changes? Better to search on values.
Tabs make things messy, use spaces instead since then the output doesn't rely on terminal settings and you don't need to worry about misaligned tabs.
And, for completeness, here's the two input files and the expected output:
p2input1 p2input2
======== ========
Dave 734.838.9801 Bob Tuesday
Bob 313.123.4567 Carol Monday
Carol 248.344.5576 Ted Sunday
Mary 313.449.1390 Alice Wednesday
Ted 248.496.2204 Dave Thursday
Alice 616.556.4458 Mary Saturday
Frank 634.296.3357 Frank Friday
Expected output
===============
Name On-Call Phone
carol monday 248.344.5576
bob tuesday 313.123.4567
alice wednesday 616.556.4458
dave thursday 734.838.9801
frank friday 634.296.3357
mary saturday 313.449.1390
ted sunday 248.496.2204
Having said all that, and assuming you've gone away for at least two hours to try and get your version running, here's mine:
1 #!/bin/bash
2 spc20=" "
3 echo "Name On-Call Phone"
4 echo
5 for day in monday tuesday wednesday thursday friday saturday sunday
6 do
7 name=`grep -i " ${day}$" p2input2 | awk '{print $1}'`
8 name=`echo ${name} | tr '[A-Z]' '[a-z]'`
9 bigname=`echo "${name}${spc20}" | cut -c1-15`
10
11 bigday=`echo "${day}${spc20}" | cut -c1-15`
12
13 phone=`grep -i "^${name} " p2input1 | awk '{print $2}'`
14
15 echo "${bigname} ${bigday} ${phone}"
16 done
And the following description should help:
Line 1elects the right shell, not always necessary.
Line 2 gives us enough spaces to make formatting easier.
Lines 3-4 give us the title and blank line.
Lines 5-6 cycles through the days, one at a time.
Line 7 gives us a name for the day. 'grep -i " ${day}$"' searches for the given day (regardless of upper or lower case) at the end of a line in pinput2 while the awk statement gives you field 1 (the name).
Line 8 simply makes the name all lowercase.
Line 9 creates a string of the right size for output by adding 50 spaces then cutting off all at the end except for 15.
Line 11 does the same for the day.
Line 13 is very similar to line 7 except it searches pinput1, looks for the name at the start of the line and returns the phone number as the second field.
Line 15 just outputs the individual items.
Line 16 ends the loop.
So there you have it, enough hints to (hopefully) fix up your own code, and a sample as to how a professional would do it :-).
It would be wise to read up on the tools used, grep, tr, cut and awk.

This is homework, I assume?
Read up on the sort and paste commands: man sort, man paste

Pax has given a good answer, but this code invokes fewer processes (11 vs a minimum of 56 = 7 * 8). It uses an auxilliary data file to give the days of the week and their sequence number.
cat <<! >p2input3
1 Monday
2 Tuesday
3 Wednesday
4 Thursday
5 Friday
6 Saturday
7 Sunday
!
sort +1 p2input3 > p2.days
sort +1 p2input2 > p2.call
join -1 2 -2 2 p2.days p2.call | sort +2 > p2.duty
sort +0 p2input1 > p2.body
join -1 3 -2 1 p2.duty p2.body | sort +2n | tr '[A-Z]' '[a-z]' |
awk 'BEGIN { printf("%-14s %-14s %s\n", "Name", "On-Call", "Phone");
printf "\n"; }
{ printf("%-14s %-14s %s\n", $1, $2, $4);}'
rm -f p2input3 p2.days p2.call p2.duty p2.body
The join command is powerful, but requires the data in the two files in sorted order on the joining keys. The cat command gives a list of days and the day number. The first sort places that list in alphabetic order of day name. The second sort places the names of the people on duty in alphabetic order of day name too. The first join then combines those two files on day name, and then sorts based on user name, yielding the output:
Wednesday 3 Alice
Tuesday 2 Bob
Monday 1 Carol
Thursday 4 Dave
Friday 5 Frank
Saturday 6 Mary
Sunday 7 Ted
The last sort puts the names and phone numbers into alphabetic name order. The second join then combines the name + phone number list with the name + duty list, yielding a 4 column output. This is run through tr to make the data all lower case, and then formatted with awk, which demonstrates its power and simplicity nicely here (you could use Perl or Python instead, but frankly, that would be messier).
Perl has a motto: TMTOWTDI "There's more than one way to do it".
That often applies to shell scripting too.
I suppose my code does not use a loop...oh dear. Replace the initial cat command with:
for day in "1 Monday" "2 Tuesday" "3 Wednesday" "4 Thursday" \
"5 Friday" "6 Saturday" "7 Sunday"
do echo $day
done > p2input3
This now meets the letter of the rules.

Try this one:
sort file1.txt > file1sort.txt
sort file2.txt > file2sort.txt
join file2sort.txt file1sort.txt | column -t > result.txt
rm file1sort.txt file2sort.txt

Related

grep date ranges from feed

I saw it posted here that this would work but it is not working for me. I need something short and sweet like this command. using the python version of rsstail.
rsstail -dl -e 1 -U -a -u https://threatpost.com/feed/ -n 10 | grep -A 2 "2021/03/15 20[1-5]"
This should grab the last 5 hours but it doesn't.
Sample line from the feed follows
Updated: 2021/03/12 21:42:59 Title: Critical Security Hole Can Knock Smart Meters Offline Author: Tara Seals Link: https://threatpost.com/critical-security-smart-meter-offline/164753/ Description: Unpatched Schneider Electric PowerLogic ION/PM smart meters are open to dangerous attacks
You were heading on the right track with the grep - however there wasn't going to be a match on the date because 20[1-5] matches two digits and a digit in the range 1-5 in a row after the specific date - that would match
2021/03/15 2022 but not 2021/03/15 20:22.
Assuming you are in daylight hours you don't have to worry about spanning two days - imagine you ran at 2:00am you'd need yesterday 21:XX, 22:XX, 23:XX and today 00:XXam, 01:XXam.
So say you run at 11.am today - previous 5 hours 6/7/8/9/10 .. so you could do something like this.
grep -A 2 -E -e '2021/03/17 (06|07|08|09|10):'
OR even
grep -A 2 -E -e '2021/03/17 (0[6789]|10):'
You can auto-generate some of the query like this (again I've ignored cross-over of hour) NOTE: OSX Date & GNU date are different - this is OSX example -
FROMMIN=$( date -v -4M +'%M' )
TOMIN=$( date +'%M' )
## GIVES like this 2021/03/17 20:(44|45|46|47|48|)
MATCH=$( echo $( date +'%Y/%m/%d %H:(' )$( seq -s "|" $FROMMIN 1 $TOMIN )')' )
grep -A 2 -E -e "$MATCH"

Editing the output of uniq -c (Bash)

I have a list of words that I sorted using sort -f. Now, I want to use uniq -c in order to obtain a list without repeated words but with a counter on the left side. I also want the column with the numbers to be separated by a tab from the column with the words.
This is my list:
Monday day
Tuesday day
Easter holiday
Monday day
christmas holiday
Tuesday day
Friday day
Thursday day
thanksgiving holiday
And this is my desired output:
1 christmas holiday
1 Easter holiday
1 Friday day
2 Monday day
1 thanksgiving holiday
1 Thursday day
2 Tuesday day
I tried using the following command, though I get a tab before the numbers instead of between the numbers and the words.
sort -f | uniq -c | sed $'s/\t */\t/g'
What do I have to modify in order to get the output that I want?
You need to get the number in a capture group and copy it to the replacement so you can put the tab after it.
sort -f days.txt | uniq -c | sed $'s/^ *\([0-9]*\) */\\1\t/'
uniq -c doesn't put tab before the count, it just puts spaces.

bash sed/awk/something for printing lines from match till eof except first line

Well task should be easy i guess but i failed to find the answer.
I need to find some regex in file and print all lines from first match till the end of file but the first found one.
To find and print sed/awk do the trick:
awk '/regex/,0'
sed '/regex/,$p'
But i feel pretty silly that i can't exclude first matching line from output.
Any tips? Pretty sure that shouldn't be hard. I'm just lacking of knowledge.
PS I need exactly to:
find line matching regex
print all lines from this regex to the end of file
exclude first line
in one action.
Excluding line after only find-n-print action leads to more complicated solution
Example
$ cat file
11 1st Cookie
12 2nd Cookie
13 1st Dinner
14 1st Candy
15 2nd Candy
16 3rd Cookie
17 1st Cake
18 2nd Dinner
19 2nd Cake
Find and print:
$ awk '/Dinner/,0' file
13 1st Dinner
14 1st Candy
15 2nd Candy
16 3rd Cookie
17 1st Cake
18 2nd Dinner
19 2nd Cake
What i want to get:
14 1st Candy
15 2nd Candy
16 3rd Cookie
17 1st Cake
18 2nd Dinner
19 2nd Cake
another alternative
awk 'f; /Dinner/{f=1}' file
14 1st Candy
15 2nd Candy
16 3rd Cookie
17 1st Cake
In general, to include either start or end pattern, there are 2x2 combinations.
$ awk '/Start/{f=1} f; /End/{f=0}' file
Start
1
2
3
End
$ awk '/End/{f=0} f; /Start/{f=1}' file
1
2
3
$ awk '/Start/{f=1} /End/{f=0} f' file
# or
$ awk '/End/{f=0} /Start/{f=1} f' file
Start
1
2
3
$ awk 'f; /Start/{f=1} /End/{f=0}' file
# or
$ awk 'f; /End/{f=0} /Start/{f=1}' file
1
2
3
End
try:
awk '/Dinner/{a=1;next} a' Input_file
Looking for string Dinner here, if yes then make a variable named as value 1, putting next will skip all further statements and to avoid printing that specific line which has dinner string in it. Then mentioning a means checking if a is NOT NULL then not mentioned any action here so by default print of the current line will happen.
Here is a POSIX sed version:
$ sed -ne '/Dinner/{s///; :a' -e 'n;p;ba' -e '}' file
14 1st Candy
15 2nd Candy
16 3rd Cookie
17 1st Cake
18 2nd Dinner
19 2nd Cake
Explanation:
/pattern/ The desired pattern
s/// assuming you do not want the pattern included in the output - empty the pattern space.
:a; n; p; ba;
a loop that:
fetches a new line from input (n);
prints it (p);
branches back to label a :a; ...; ba;
With GNU sed:
$ sed -ne '/Dinner/{s///; :a;n;p;ba}' file

Checking for changes in two lists in Bash

I have these two files containing a list of items, where the quantity of each item is separated by a space. These lists are supposed to be already ordered and always having the same amount of items each, however I would prefer making a code that relies on the item name and not on the number of the line.
I need to have an output where only the changes are present, for example an echo for every item that has changed its associated value. I know I could use diff or meld for this, but I need a very specific output, because then I have to send a mail for every one of these changes, so I guess I should be using something like awk.
cat before.txt
Apples 3
Oranges 5
Bananas 7
Avocados 2
cat after.txt
Apples 3
Oranges 7
Bananas 7
Avocados 3
output wanted:
Oranges has changed form 5 to 7
Avocados has changed form 2 to 3
awk is your friend
awk 'NR==FNR{price[$1]=$2;next}
$1 in price{
if(price[$1]!=$2){
printf "%s has changed from %s to %s%s",$1,price[$1],$2,ORS
}
}' before.txt after.txt
Output
Oranges has changed from 5 to 7
Avocados has changed from 2 to 3
If you're new to awk consider buying [ Effective awk Programming ] by Arnold Robbins.
Not as great as other answer but simple to understand. This is not very economic way to do this task , however I have added it as it makes things simple . Of course if performance is not really a concern.
paste before.txt after.txt | awk '$2!=$4 {print $1 " are changes from " $2 " to " $4}'
Oranges are changes from 5 to 7
Avocados are changes from 2 to 3

Format words under each other

So, I started learning bash last week and I have to do a task where I should print the content of this file.txt:
1|george|01/02/2042
2|TPS Reports|03/01/2015
3|Go clubbing this weekend|
4|Metting with family|03/08/2015
5|Help Rose with dating boys|
6|Update hacking software for hacking StackExchange|09/30/2015
I have written this code:
while IFS='' read -r line || [[ -n $line ]]; do
IFS='|' read -ra ADDR <<< "$line"
echo -e "${ADDR[0]}: ${ADDR[1]} \t\t\t ${ADDR[2]}"
done < "$HOME_DIRECTORY_FILE"
So, basically this code, will go line by line, take each line, and split it into the array using the delimiter |, then print each part of the array on screen, output:
1: george 01/02/2042
2: TPS Reports 03/01/2015
3: Go clubbing this weekend
4: Metting with family 03/08/2015
5: Help Rose with dating boys
6: Update hacking software for hacking StackExchange 09/30/2015
You might think this is correct, but my instructor said the dates need to be under each other, like this:
1: george 01/02/2042
2: TPS Reports 03/01/2015
3: Go clubbing this weekend
4: Metting with family 03/08/2015
5: Help Rose with dating boys
6: Update hacking software for hacking StackExchange 09/30/2015
Is that achievable in bash? Or I should let go of this? My instructor gave the exact output example, spaced this way and said "the output should be neatly formatted (spacing)."
Use the column command. It does exactly what you're looking for.
For example:
$ cat input.txt
1|george|01/02/2042
2|TPS Reports|03/01/2015
3|Go clubbing this weekend|
4|Metting with family|03/08/2015
5|Help Rose with dating boys|
6|Update hacking software for hacking StackExchange|09/30/2015
$ column --separator \| --table input.txt
1 george 01/02/2042
2 TPS Reports 03/01/2015
3 Go clubbing this weekend
4 Metting with family 03/08/2015
5 Help Rose with dating boys
6 Update hacking software for hacking StackExchange 09/30/2015
You'll need to do a little pre-formatting to get your numbers to have :, but that should be the easy part (you can pipe the modified file into column).
You can also use printf, although it requires you to guess at the width of the middle column (which column computes for you).
while IFS='' read -r line || [[ -n $line ]]; do
IFS='|' read -ra ADDR <<< "$line"
printf "%d: %-30s %s\n" "${ADDR[0]}" "${ADDR[1]}" "${ADDR[2]}"
done < "$HOME_DIRECTORY_FILE"

Resources