"bash shell" reading in sections from a file - bash

I'm new to bash scripting. Getting really hooked on text mate also. Anyways I have a txt file that contains something like as follows.....
Smith:David:111 Oak St.:Colorado Springs:CO:80901
Smith:Tina:111 Oak St.:Colorado Springs:CO:80901
Martin:Steve:2233 Cascade St.:Colorado Springs:CO:80907
I'm trying to create a program that will read it in and display in the following format.
Jane Doe
5245 Anystreet St.
Any City, State 22222
John Doe
2245 Anystreet St.
Any City, State 22222
I can't find beginner friendly documentation on what commands to use and how to use them. A few sights give great in depth documentation but it's hard to follow. I would like to implement the sort command and sort it by zipcode but I can't get a good example to follow. Can someone help me figure this out or help me get started?
----------------------------------------Further Questions "Updated"------------------------------
Thanks for the advice and it has helped out greatly. I wrote the program a few different ways however I have a few last questions on one example another student did. Don't quite understand everything in their code. I've put my questions as comments.

The task is trivial if you can use awk:
awk -F: '{print $2,$1 ORS $3 ORS $4", "$5,$6 ORS}' file
Test:
$ cat file
Smith:David:111 Oak St.:Colorado Springs:CO:80901
Smith:Tina:111 Oak St.:Colorado Springs:CO:80901
Martin:Steve:2233 Cascade St.:Colorado Springs:CO:80907
$ awk -F: '{print $2,$1 ORS $3 ORS $4", "$5,$6 ORS}' file
David Smith
111 Oak St.
Colorado Springs, CO 80901
Tina Smith
111 Oak St.
Colorado Springs, CO 80901
Steve Martin
2233 Cascade St.
Colorado Springs, CO 80907
You can use arrays in bash and do:
while IFS=: read -ra line; do
printf "%s %s\n%s\n%s, %s %s %s\n\n" "${line[1]}" "${line[0]}" "${line[2]}" "${line[3]}" "${line[4]}" "
${line[5]}";
done < file
Output
David Smith
111 Oak St.
Colorado Springs, CO 80901
Tina Smith
111 Oak St.
Colorado Springs, CO 80901
Steve Martin
2233 Cascade St.
Colorado Springs, CO 80907

You can set IFS to colon and then exploit that, carefully:
$ cat > xyz
Smith:David:111 Oak St.:Colorado Springs:CO:80901
Smith:Tina:111 Oak St.:Colorado Springs:CO:80901
Martin:Steve:2233 Cascade St.:Colorado Springs:CO:80907
$ while read line
> do (IFS=:
> set -- $line
> echo $2 $1
> echo $3
> echo $4, $5 $6
> echo
> )
> done < xyz
David Smith
111 Oak St.
Colorado Springs, CO 80901
Tina Smith
111 Oak St.
Colorado Springs, CO 80901
Steve Martin
2233 Cascade St.
Colorado Springs, CO 80907
$
The use of a sub-shell is optional, but means that IFS in the main shell is not changed — usually a good idea.

Thy this simple one:
cat file | sed -r 's/(.*):(.*):(.*):(.*):(.*):(.*)/\1 \2\n\3\n\4,\5 \6\n/'

Related

Adding data from an array to a new column in a file using bash [duplicate]

This question already has answers here:
Iterate over two arrays simultaneously in bash
(6 answers)
How to print two arrays side by side with bash script?
(2 answers)
Closed 2 years ago.
So I have Name age and city data:
name=(Alex Barbara Connor Daniel Matt Peter Stan)
age=(22 23 55 32 21 8 89)
city=(London Manchester Rome Alberta Naples Detroit Amsterdam)
and I want to set up the following as 3 column data with the headings Name Age and city, I can easily get the first column using
touch info.txt
echo "Name Age City" > info.txt
for n in ${name[#]}; do
echo $n >> info.txt
done
but I can't figure how to get the rest of the data, and I can't seem to find anywhere on how to add data that's different as a new column.
Any help would be greatly appreciated, thank you.
Try something like this:
name=(Alex Barbara Connor Daniel Matt Peter Stan)
age=(22 23 55 32 21 8 89)
city=(London Manchester Rome Alberta Naples Detroit Amsterdam)
touch info.txt
echo "Name Age City" > info.txt
for n in $(seq 0 6); do
echo ${name[$n]} ${age[$n]} ${city[$n]} >> info.txt
done
Output in info.txt:
Name Age City
Alex 22 London
Barbara 23 Manchester
Connor 55 Rome
Daniel 32 Alberta
Matt 21 Naples
Peter 8 Detroit
Stan 89 Amsterdam
JoseLinares solved your problem. For your information, here is a solution with the paste command whose purpose is exactly that: putting data from different sources in separate columns.
$ printf 'Name\tAge\tCity\n'
$ paste <(printf '%s\n' "${name[#]}") \
<(printf '%3d\n' "${age[#]}") \
<(printf '%s\n' "${city[#]}")
Name Age City
Alex 22 London
Barbara 23 Manchester
Connor 55 Rome
Daniel 32 Alberta
Matt 21 Naples
Peter 8 Detroit
Stan 89 Amsterdam
You can fix a specific width for each column (here 20 is used)
name=(Alex Barbara Connor Daniel Matt Peter Stan)
age=(22 23 55 32 21 8 89)
city=(London Manchester Rome Alberta Naples Detroit Amsterdam)
for i in "${!name[#]}"; do
printf "%-20s %-20s %-20s\n" "${name[i]}" "${age[i]}" "${city[i]}"
done
Output:
Alex 22 London
Barbara 23 Manchester
Connor 55 Rome
Daniel 32 Alberta
Matt 21 Naples
Peter 8 Detroit
Stan 89 Amsterdam

Shell: how to right justify columns for my file

I have a data file like below:
Jones,Bill,235 S. Williams St.,Denver,CO,80221,(303) 244-7989
Smith,Tom,404 Polk Ave.,Los Angeles,CA,90003,(213) 879-5612
I want each line be separated by ',' and right justify columns, just like below
Jones Bill 235 S. Williams St. Denver CO 80221 (303) 244-7989
Smith Tom 404 Polk Ave. Los Angeles CA 90003 (213) 879-5612
This is my code and it doesn't work. Pls help, thx.
while read line
do
echo "$line" | awk -F, '{for(i=1;i<=NF;i++)printf "%15s" $i}'
echo
done < "datafile.txt"
You should not pipe each line to awk with a loop. Loops are slow and it decreases readability (have a look at Why is using a shell loop to process text considered bad practice?). Awk reads lines by default, just provide the filename as an argument:
awk -F, '{for(i=1;i<=NF;i++){printf "%22s",$i};print ""}' datafile.txt
Notice %15s would not be enough to yield a nice table, so I increased it to %22s. And you missed the comma , in printf statement, that is why it your attempt failed.
You could also use column -ts "," datafile.txt, but that would left justify the output.
Awk itself has for loop. Using a shell for loop can work too.
$ cat file
Jones,Bill,235 S. Williams St.,Denver,CO,80221,(303) 244-7989
Smith,Tom,404 Polk Ave.,Los Angeles,CA,90003,(213) 879-5612
$ awk -F, '{ for (i=1; i<=NF; i++) printf("%20s", $i); printf "\n"; }' file
Jones Bill 235 S. Williams St. Denver CO 80221 (303) 244-7989
Smith Tom 404 Polk Ave. Los Angeles CA 90003 (213) 879-5612
$
$ while read line; do echo "$line" | awk -F, '{ for (i=1; i<=NF; i++) printf("%20s", $i); printf "\n"; }'; done < file
Jones Bill 235 S. Williams St. Denver CO 80221 (303) 244-7989
Smith Tom 404 Polk Ave. Los Angeles CA 90003 (213) 879-5612
$
With column and rev:
$ rev file | column -ts, | rev
Jones Bill 235 S. Williams St. Denver CO 80221 (303) 244-7989
Smith Tom 404 Polk Ave. Los Angeles CA 90003 (213) 879-5612
or with GNU column (for -R) and seq:
$ column -ts, -R $(seq -s, 999) file
Jones Bill 235 S. Williams St. Denver CO 80221 (303) 244-7989
Smith Tom 404 Polk Ave. Los Angeles CA 90003 (213) 879-5612

reading a file into an array in bash

Here is my code
#!bin/bash
IFS=$'\r\n'
GLOBIGNORE='*'
command eval
'array=($(<'$1'))'
sorted=($(sort <<<"${array[*]}"))
for ((i = -1; i <= ${array[-25]}; i--)); do
echo "${array[i]}" | awk -F "/| " '{print $2}'
done
I keep getting an error that says "line 5: array=($(<)): command not found"
This is my problem.
As a whole my code should read in a file as a command line argument, sort the elements, then print out column 2 of the last 25 lines. I haven't been able to test this far so if there's a problem there too any help would be appreciated.
This is some of what the file contains:
290729 123456
79076 12345
76789 123456789
59462 password
49952 iloveyou
33291 princess
21725 1234567
20901 rockyou
20553 12345678
16648 abc123
16227 nicole
15308 daniel
15163 babygirl
14726 monkey
14331 lovely
14103 jessica
13984 654321
13981 michael
13488 ashley
13456 qwerty
13272 111111
13134 iloveu
13028 000000
12714 michelle
11761 tigger
11489 sunshine
11289 chocolate
11112 password1
10836 soccer
10755 anthony
10731 friends
10560 butterfly
10547 purple
10508 angel
10167 jordan
9764 liverpool
9708 justin
9704 loveme
9610 fuckyou
9516 123123
9462 football
9310 secret
9153 andrea
9053 carlos
8976 jennifer
8960 joshua
8756 bubbles
8676 1234567890
8667 superman
8631 hannah
8537 amanda
8499 loveyou
8462 pretty
8404 basketball
8360 andrew
8310 angels
8285 tweety
8269 flower
8025 playboy
7901 hello
7866 elizabeth
7792 hottie
7766 tinkerbell
7735 charlie
7717 samantha
7654 barbie
7645 chelsea
7564 lovers
7536 teamo
7518 jasmine
7500 brandon
7419 666666
7333 shadow
7301 melissa
7241 eminem
7222 matthew
In Linux you can simply do a
sort -nbr file_to_sort | head -n 25 | awk '{print $2}'
read in a file as a command line argument, sort the elements, then
print out column 2 of the last 25 lines.
From that discription of the problem, I suggest:
#! /bin/sh
sort -bn $1 | tail -25 | awk '{print $2}'
As a rule, use the shell to operate on filenames, and never use the
shell to operate on data. Utilities like sort and awk are far
faster and more powerful than the shell when it comes to processing a
file.

Combine text from two files, output to another [duplicate]

This question already has answers here:
Inner join on two text files
(5 answers)
Closed 1 year ago.
i'm having a bit of a problem and i've been searching allll day. this is my first Unix class don't be to harsh.
so this may sound fairly simple, but i can't get it
I have two text files
file1
David 734.838.9801
Roberto‭ ‬313.123.4567
Sally‭ ‬248.344.5576
Mary‭ ‬313.449.1390
Ted‭ ‬248.496.2207
Alice‭ ‬616.556.4458
Frank‭ ‬634.296.1259
file2
Roberto Tuesday‭ ‬2
Sally Monday‭ ‬8
Ted Sunday‭ ‬16
Alice Wednesday‭ ‬23
David Thursday‭ ‬10
Mary Saturday‭ ‬14
Frank Friday‭ ‬15
I am trying to write a script using a looping structure that will combine both files and come out with the output below as a separate file
output:
Name On-Call Phone Start Time
Sally Monday 248.344.5576 8am
Roberto Tuesday 313.123.4567 2am
Alice‭ Wednesday‭ 616.556.4458‭ 11pm
David‭ Thursday‭ 734.838.9801‭ 10am
Frank‭ Friday‭ 634.296.1259‭ 3pm
Mary‭ Saturday‭ 313.449.1390‭ 2pm
Ted‭ ‬ Sunday‭ 248.496.2207‭ 4pm
This is what i tried( i know its horrible)
echo " Name On-Call Phone Start Time"
file="/home/xubuntu/date.txt"
file1="/home/xubuntu/name.txt"
while read name2 phone
do
while read name day time
do
echo "$name $day $phone $time"
done<"$file"
done<"$file1"
any help would be appreciated
First, sort the files using sort and then use this command:
paste file1 file2 | awk '{print $1,$4,$2,$5}'
This will bring you pretty close. After that you have to figure out how to format the time from the 24 hour format to the 12 hour format.
If you want to avoid using sort separately, you can bring in a little more complexity like this:
paste <(sort file1) <(sort file2) | awk '{print $1,$4,$2,$5}'
Finally, if you have not yet figured out how to print the time in 12 hour format, here is your full command:
paste <(sort file1) <(sort file2) | awk '{"date --date=\"" $5 ":00:00\" +%I%P" |& getline $5; print $1 " " $4 " " $2 " " $5 }'
You can use tabs (\t) in place of spaces as connectors to get a nicely formatted output.
In this case join command will also work,
join -1 1 -2 1 <(sort file1) <(sort file2)
Description
-1 -> file1
1 -> first field of file1 (common field)
-2 -> file2
1 -> first field of file2 (common field)
**cat file1**
David 734.838.9801
Roberto 313.123.4567
Sally 248.344.5576
Mary 313.449.1390
Ted 248.496.2207
Alice 616.556.4458
Frank 634.296.1259
**cat file2**
Roberto Tuesday 2
Sally Monday 8
Ted Sunday 16
Alice Wednesday 23
David Thursday 10
Mary Saturday 14
Frank Friday 15
output
Alice 616.556.4458 Wednesday 23
David 734.838.9801 Thursday 10
Frank 634.296.1259 Friday 15
Mary 313.449.1390 Saturday 14
Roberto 313.123.4567 Tuesday 2
Sally 248.344.5576 Monday 8
Ted 248.496.2207 Sunday 16

Comparision of old and files for new updated records and to insert new records in old files

I want to compare a file which comes with new records, to the same structured file which already have records and exists in the server.
Like old file is file1:
SKU PROD NAME CODE PRICE
A001 shirt jm s02 478
B002 jean jn j02 348
C003 mwear mw m02 567
New file which comes with new record is:
SKU PROD NAME CODE PRICE
A001 shirt jm s02 680
m01 mwear mw m02 567
c02 kurta kr k04 677
d12 dr d3 d03 400
Based on the new records, either old records can be updated or new records can be appended after the old records.
I need to write a unix shell sript for the above scenerio. Please help
Look for the different SKUs in the new file first. If a SKU is not found there, get it from the old file.
grep SKU file1.txt > out.txt
cat file1.txt file2.txt |\
grep -v SKU | cut -d" " -f1 | sort | uniq | while read sku
do
grep $sku file2.txt >> out.txt
if [[ $? -eq 1 ]] # If SKU was not in file2.txt
then
grep $sku file1.txt >> out.txt
fi
done
Result:
SKU PROD NAME CODE PRICE
A001 shirt jm s02 680
B002 jean jn j02 348
C003 mwear mw m02 567
c02 kurta kr k04 677
d12 dr d3 d03 400
m01 mwear mw m02 567
Look at the problem in reverse : your new file is basically your newest data. You could simply replace the old with the new, except that there may be some old data that is not in your new file.
So, a lazy way to do this might be to read your old data, and look for stuff that isn't in your new file, and add it if you need to.
Something like this (off the top of my head, couldn't test it)
while read old_data
do
# find the key
sku=$(echo $old_data | cut -d\ -f1)
# Look for key in new data, paste the line if it's not there.
grep sku new_file > /dev/null || echo old_data >> new_file
done < old_file

Resources