Here is my code
#!bin/bash
IFS=$'\r\n'
GLOBIGNORE='*'
command eval
'array=($(<'$1'))'
sorted=($(sort <<<"${array[*]}"))
for ((i = -1; i <= ${array[-25]}; i--)); do
echo "${array[i]}" | awk -F "/| " '{print $2}'
done
I keep getting an error that says "line 5: array=($(<)): command not found"
This is my problem.
As a whole my code should read in a file as a command line argument, sort the elements, then print out column 2 of the last 25 lines. I haven't been able to test this far so if there's a problem there too any help would be appreciated.
This is some of what the file contains:
290729 123456
79076 12345
76789 123456789
59462 password
49952 iloveyou
33291 princess
21725 1234567
20901 rockyou
20553 12345678
16648 abc123
16227 nicole
15308 daniel
15163 babygirl
14726 monkey
14331 lovely
14103 jessica
13984 654321
13981 michael
13488 ashley
13456 qwerty
13272 111111
13134 iloveu
13028 000000
12714 michelle
11761 tigger
11489 sunshine
11289 chocolate
11112 password1
10836 soccer
10755 anthony
10731 friends
10560 butterfly
10547 purple
10508 angel
10167 jordan
9764 liverpool
9708 justin
9704 loveme
9610 fuckyou
9516 123123
9462 football
9310 secret
9153 andrea
9053 carlos
8976 jennifer
8960 joshua
8756 bubbles
8676 1234567890
8667 superman
8631 hannah
8537 amanda
8499 loveyou
8462 pretty
8404 basketball
8360 andrew
8310 angels
8285 tweety
8269 flower
8025 playboy
7901 hello
7866 elizabeth
7792 hottie
7766 tinkerbell
7735 charlie
7717 samantha
7654 barbie
7645 chelsea
7564 lovers
7536 teamo
7518 jasmine
7500 brandon
7419 666666
7333 shadow
7301 melissa
7241 eminem
7222 matthew
In Linux you can simply do a
sort -nbr file_to_sort | head -n 25 | awk '{print $2}'
read in a file as a command line argument, sort the elements, then
print out column 2 of the last 25 lines.
From that discription of the problem, I suggest:
#! /bin/sh
sort -bn $1 | tail -25 | awk '{print $2}'
As a rule, use the shell to operate on filenames, and never use the
shell to operate on data. Utilities like sort and awk are far
faster and more powerful than the shell when it comes to processing a
file.
Related
I have a data file like below:
Jones,Bill,235 S. Williams St.,Denver,CO,80221,(303) 244-7989
Smith,Tom,404 Polk Ave.,Los Angeles,CA,90003,(213) 879-5612
I want each line be separated by ',' and right justify columns, just like below
Jones Bill 235 S. Williams St. Denver CO 80221 (303) 244-7989
Smith Tom 404 Polk Ave. Los Angeles CA 90003 (213) 879-5612
This is my code and it doesn't work. Pls help, thx.
while read line
do
echo "$line" | awk -F, '{for(i=1;i<=NF;i++)printf "%15s" $i}'
echo
done < "datafile.txt"
You should not pipe each line to awk with a loop. Loops are slow and it decreases readability (have a look at Why is using a shell loop to process text considered bad practice?). Awk reads lines by default, just provide the filename as an argument:
awk -F, '{for(i=1;i<=NF;i++){printf "%22s",$i};print ""}' datafile.txt
Notice %15s would not be enough to yield a nice table, so I increased it to %22s. And you missed the comma , in printf statement, that is why it your attempt failed.
You could also use column -ts "," datafile.txt, but that would left justify the output.
Awk itself has for loop. Using a shell for loop can work too.
$ cat file
Jones,Bill,235 S. Williams St.,Denver,CO,80221,(303) 244-7989
Smith,Tom,404 Polk Ave.,Los Angeles,CA,90003,(213) 879-5612
$ awk -F, '{ for (i=1; i<=NF; i++) printf("%20s", $i); printf "\n"; }' file
Jones Bill 235 S. Williams St. Denver CO 80221 (303) 244-7989
Smith Tom 404 Polk Ave. Los Angeles CA 90003 (213) 879-5612
$
$ while read line; do echo "$line" | awk -F, '{ for (i=1; i<=NF; i++) printf("%20s", $i); printf "\n"; }'; done < file
Jones Bill 235 S. Williams St. Denver CO 80221 (303) 244-7989
Smith Tom 404 Polk Ave. Los Angeles CA 90003 (213) 879-5612
$
With column and rev:
$ rev file | column -ts, | rev
Jones Bill 235 S. Williams St. Denver CO 80221 (303) 244-7989
Smith Tom 404 Polk Ave. Los Angeles CA 90003 (213) 879-5612
or with GNU column (for -R) and seq:
$ column -ts, -R $(seq -s, 999) file
Jones Bill 235 S. Williams St. Denver CO 80221 (303) 244-7989
Smith Tom 404 Polk Ave. Los Angeles CA 90003 (213) 879-5612
I have a file that includes names, one on each line. I want to print the names alphabetically, but (and here is where it gets confusing at least for me) next to each name I must print the number of appearances of that name with exactly one space between the name and the number of appearances.
For example if the file includes these names:
Barry
Don
John
Sam
Harry
Don
Don
Sam
it must print
Barry 1
Don 3
Harry 1
John 1
Sam 2
Any ideas?
sort | uniq -c will get you very close, just with the columns reversed.
$ sort file | uniq -c
1 Barry
3 Don
1 Harry
1 John
2 Sam
If you really need them in the proscribed order you could swap them with awk.
$ sort test.txt | uniq -c | awk '{print $2, $1}'
Barry 1
Don 3
Harry 1
John 1
Sam 2
With awk :
% awk '{
a[$1]++
}
END{
for (i in a) {
print i, a[i]
}
}' file
Output:
Barry 1
Harry 1
Don 3
John 1
Sam 2
Given:
$ cat file
Barry
Don
John
Sam
Harry
Don
Don
Sam
You can do:
$ awk '{a[$1]++} END { for (e in a) print e, a[e] }' file | sort
Barry 1
Don 3
Harry 1
John 1
Sam 2
In bash, how can I read in a large .csv file and summarize the data? I need to get totals for each person.
example input:
joey 4
joey 3
joey 4
joey 6
paul 7
paul 3
paul 1
paul 4
trevor 5
trevor 6
henry 7
mark 8
mark 9
tom 0
It should end up like this in the end:
joey 17
paul 15
trevor 11
henry 7
mark 17
tom 2
list=`your example input | awk '{print $1}' | uniqe`
it gives You something like this:
joey
paul
trevor
henry
mark
tom
Now let's make a two for loops:
for i in $list
do
for j in `$list | grep $i | awk '{print $2}'`
do
counter=$counter+$j
done
echo "$i $j"
done
First loop is going by the names and second one is just counting results for each name. Guess it should work, and it's quite easy way.
I'm trying to write a shell script that prints the full names of users logged on to a machine. The finger command gives me a list of users, but there are many duplicates. How can I loop through and print out only the unique ones?
Edit:
This is the format of what finger gives me:
xxxx XX of group XXX pts/59 1:00 Feb 13 16:38
xxxx XX of group XXX pts/71 1:11 Feb 13 16:27
xxxx XX of group XXX pts/105 1d Feb 12 15:22
xxxx YY of group YYY pts/102 2:19 Feb 13 14:13
xxxx ZZ of group ZZZ pts/42 2d Feb 7 12:11
I'm trying to extract the full name (i.e. whatever comes before 'of group' in column 2), so I would be using awk together with finger.
What you want is actually fairly difficult in a shell script, here is, for example, my full output of finger(1):
Login Name TTY Idle Login Time Office Phone
martin Martin Tournoij *v0 1d Wed 14:11
martin Martin Tournoij pts/2 22 Wed 15:37
martin Martin Tournoij pts/5 41 Thu 23:16
martin Martin Tournoij pts/7 31 Thu 23:24
martin Martin Tournoij pts/8 Thu 23:29
You want the full name, but this may contain 1 space (as per my example), or it may just be 'Teller' (no space), or it may be 'Captain James T. Kirk' (3 spaces). So you can't just use the space as delimiter. You could use the character position of 'TTY' in the header as an indicator, but that's not very elegant IMHO (especially with shell scripting).
My solution is therefore slightly different, we get only the username from finger(1), then we get the full name from /etc/passwd
#!/bin/sh
prev=""
for u in $(finger | tail +2 | cut -w -f1 | sort); do
[ "$u" = "$prev" ] && continue
echo "$u $(grep "^$u" /etc/passwd | cut -d: -f5)"
prev="$u"
done
Which gives me both the username & login name:
martin Martin Tournoij
Obviously, you can also print just the real name (without the $u).
The sort and uniq BinUtils commands can be used to removed duplicates.
finger | sort -u
This will remove all duplicate lines, but you will still see similar lines due to how verbose the finger command is. If you just want a list of usernames, you can filter it out further to be very specific.
finger | cut -d ' ' -f1 | sort -u
Now, you can take this one step further, and remove the "header/label" line printed out by the finger command.
finger | cut -d ' ' -f1 | sort -u | grep -iv login
Hope this helps.
Other possible solution:
finger | tail -n +2 | awk '{ print $1 }' | sort | uniq
tail -n +2 to omit the first line.
awk '{ print $1 }' to extract the first column.
sort to prepare input for uniq.
uniq remove duplicates.
If you want to iterate use:
for user in $(finger | tail -n +2 | awk '{ print $1 }' | sort | uniq)
do
echo "$user"
done
Could this be simpler?
No spaces or any other special characters to worry about!
finger -l | awk '/^Login/'
Edit: To remove the content after of group
finger -l | awk '/^Login/' | sed 's/of group.*//g'
Output:
Login: xx Name: XX
Login: yy Name: YY
Login: zz Name: ZZ
I'm new to bash scripting. Getting really hooked on text mate also. Anyways I have a txt file that contains something like as follows.....
Smith:David:111 Oak St.:Colorado Springs:CO:80901
Smith:Tina:111 Oak St.:Colorado Springs:CO:80901
Martin:Steve:2233 Cascade St.:Colorado Springs:CO:80907
I'm trying to create a program that will read it in and display in the following format.
Jane Doe
5245 Anystreet St.
Any City, State 22222
John Doe
2245 Anystreet St.
Any City, State 22222
I can't find beginner friendly documentation on what commands to use and how to use them. A few sights give great in depth documentation but it's hard to follow. I would like to implement the sort command and sort it by zipcode but I can't get a good example to follow. Can someone help me figure this out or help me get started?
----------------------------------------Further Questions "Updated"------------------------------
Thanks for the advice and it has helped out greatly. I wrote the program a few different ways however I have a few last questions on one example another student did. Don't quite understand everything in their code. I've put my questions as comments.
The task is trivial if you can use awk:
awk -F: '{print $2,$1 ORS $3 ORS $4", "$5,$6 ORS}' file
Test:
$ cat file
Smith:David:111 Oak St.:Colorado Springs:CO:80901
Smith:Tina:111 Oak St.:Colorado Springs:CO:80901
Martin:Steve:2233 Cascade St.:Colorado Springs:CO:80907
$ awk -F: '{print $2,$1 ORS $3 ORS $4", "$5,$6 ORS}' file
David Smith
111 Oak St.
Colorado Springs, CO 80901
Tina Smith
111 Oak St.
Colorado Springs, CO 80901
Steve Martin
2233 Cascade St.
Colorado Springs, CO 80907
You can use arrays in bash and do:
while IFS=: read -ra line; do
printf "%s %s\n%s\n%s, %s %s %s\n\n" "${line[1]}" "${line[0]}" "${line[2]}" "${line[3]}" "${line[4]}" "
${line[5]}";
done < file
Output
David Smith
111 Oak St.
Colorado Springs, CO 80901
Tina Smith
111 Oak St.
Colorado Springs, CO 80901
Steve Martin
2233 Cascade St.
Colorado Springs, CO 80907
You can set IFS to colon and then exploit that, carefully:
$ cat > xyz
Smith:David:111 Oak St.:Colorado Springs:CO:80901
Smith:Tina:111 Oak St.:Colorado Springs:CO:80901
Martin:Steve:2233 Cascade St.:Colorado Springs:CO:80907
$ while read line
> do (IFS=:
> set -- $line
> echo $2 $1
> echo $3
> echo $4, $5 $6
> echo
> )
> done < xyz
David Smith
111 Oak St.
Colorado Springs, CO 80901
Tina Smith
111 Oak St.
Colorado Springs, CO 80901
Steve Martin
2233 Cascade St.
Colorado Springs, CO 80907
$
The use of a sub-shell is optional, but means that IFS in the main shell is not changed — usually a good idea.
Thy this simple one:
cat file | sed -r 's/(.*):(.*):(.*):(.*):(.*):(.*)/\1 \2\n\3\n\4,\5 \6\n/'