issue with if statement in bash - bash

I have issue with an if statement. In WEDI_RC is saved log file in the following format:
name_of_file date number_of_starts
I want to compare first argument $1 with first column and if it is true than increment number of starts. When I start my script it works but just with one file, eg:
file1.c 11:23:07 1
file1.c 11:23:14 2
file1.c 11:23:17 3
file1.c 11:23:22 4
file2.c 11:23:28 1
file2.c 11:23:35 2
file2.c 11:24:10 3
file2.c 11:24:40 4
file2.c 11:24:53 5
file1.c 11:25:13 1
file1.c 11:25:49 2
file2.c 11:26:01 1
file2.c 11:28:12 2
Every time when I change file it begin counts from 1. I need to continue with counting when it ends.
Hope you understand me.
while read -r line
do
echo "line:"
echo $line
if [ "$1"="$($line | grep ^$1)" ]; then
number=$(echo $line | grep $1 | awk -F'[ ]' '{print $3}')
else
echo "error"
fi
done < $WEDI_RC
echo "file"
((number++))
echo $1 `date +"%T"` $number >> $WEDI_RC

There are at least two ways to resolve the problem. The most succinct is probably:
echo "$1 $(date +"%T") $(($(grep -c "^$1 " "$WEDI_RC") + 1))" >> "$WEDI_RC"
However, if you want to have counts for each file separately, you can do that using an associative array, assuming you have Bash version 4.x (not 3.x as is provided on Mac OS X, for example). This code assumes the file is correctly formatted (so that the counts do not reset to 1 each time the file name changes).
declare -A files # Associative array
while read -r file time count # Split line into three variables
do
echo "line: $file $time $count" # One echo - not two
files[$file]="$count" # Record the current maximum for file
done < "$WEDI_RC"
echo "$1 $(date +"%T") $(( ${files[$1]} + 1 ))" >> "$WEDI_RC"
The code uses read to split the line into three separate variables. It echoes what it read and records the current count. When the loop's done, it echoes the data to append to the file. If the file is new (not mentioned in the file yet), then you will get a 1 added.
If you need to deal with the broken file as input, then you can amend the code to count the number of entries for a file, instead of trusting the count value. The bare-array reference notation used in the (( … )) operation is necessary when incrementing the variable; you can't use ${array[sub]}++ with the increment (or decrement) operator because that evaluates to the value of the array element, not its name!
declare -A files # Associative array
while read -r file time count # Split line into three variables
do
echo "line: $file $time $count" # One echo - not two
((files[$file]++)) # Count the occurrences of file
done < "$WEDI_RC"
echo "$1 $(date +"%T") $(( ${files[$1]} + 1 ))" >> "$WEDI_RC"
You can even detect whether the format is in the broken or fixed style:
declare -A files # Associative array
while read -r file time count # Split line into three variables
do
echo "line: $file $time $count" # One echo - not two
if [ $((files[$file]++)) != "$count" ]
then echo "$0: warning - count out of sync: ${files[$file]} vs $count" >&2
fi
done < "$WEDI_RC"
echo "$1 $(date +"%T") $(( ${files[$1]} + 1 ))" >> "$WEDI_RC"

I don't get exactly what you want to achieve with your test [ "$1"="$($line | grep ^$1)" ] but it seems you are checking that the line start with the first argument.
If it is so, I think you can either:
provide the -o option to grep so that it print just the matched output (so $1)
use [[ "$line" =~ ^"$1" ]] as test.

Related

Creating files in succession

How would one go about creating a script for creating 25 empty files in succession? (I.e 1-25, 26-51, 52-77)
I can create files 1-25 but I’m having trouble figuring out how to create a script that continues that process from where it left off, every time I run the script.
#!/bin/bash
higher=$( find files -type f -exec basename {} \; | sort -n | tail -1 )
if [[ "$higher" == "" ]]
then
start=1
end=25
else
(( start = higher + 1 ))
(( end = start + 25 ))
fi
echo "$start --> $end"
for i in $(seq $start 1 $end)
do
touch files/"$i"
done
I put my files in a directory called "files".
hence the find on directory "files".
for each file found, I run a basename on it. That will return only integer values, since the files all have a number filename.
sort -n puts them in order.
tail -1 extracts the highest number.
if there are no files, higher will be empty, so the indexes will be 1 and 25.
otherwise, they will be higher + 1, and higher + 26.
I used seq for the for loop to avoid problems with variables inside a range definition (you did {1..25})
#! /usr/bin/env bash
declare -r base="${1:-base-%d.txt}"
declare -r lot="${2:-25}"
declare -i idx=1
declare -i n=0
printf -v filename "${base}" ${idx}
while [[ -e "${filename}" ]]; do
idx+=1
printf -v filename "${base}" "${idx}"
done
while [[ $n -lt $lot ]]; do
printf -v filename "${base}" ${idx}
if [[ ! -e "${filename}" ]]; then
> "$filename"
n+=1
fi
idx+=1
done
This script accepts two optional parameters.
The first is the basename of your future files with a %d token automatically replaced by the file number. Default value is base-%d.txt;
The number of file to create. Default value is 25.
How script works:
Variable declarations
base: file basename (constant)
lot: number of file to create (constant)
idx: search index
n: counter for new files
Search files already created from 1
The loop stop at first hole in the numbering
Loop to create empty files
The condition in the loop allows to fill in the numbering holes
> filename create an empty file

Simple bash program which compares values

I have a file which contains varoius data (date,time,speed, distance from the front, distance from the back), the file looks like this, just with more rows:
2003.09.23.,05:05:21:64,134,177,101
2009.03.10.,17:46:17:81,57,102,57
2018.01.05.,00:30:37:04,354,145,156
2011.07.11.,23:21:53:43,310,125,47
2011.06.26.,07:42:10:30,383,180,171
I'm trying to write a simple Bash program, which tells the dates and times when the 'distance from the front' is less than the provided parameter ($1)
So far I wrote:
#!/bin/bash
if [ $# -eq 0 -o $# -gt 1 ]
then
echo "wrong number of parameters"
fi
i=0
fdistance=()
input='auto.txt'
while IFS= read -r line
do
year=${line::4}
month=${line:5:2}
day=${line:8:2}
hour=${line:12:2}
min=${line:15:2}
sec=${line:18:2}
hthsec=${line:21:2}
fdistance=$(cut -d, -f 4)
if [ "$fdistance[$i]" -lt "$1" ]
then
echo "$year[$i]:$month[$i]:$day[$i],$hour[$i]:$min[$i]:$sec[$i]:$hthsec[$i]"
fi
i=`expr $i + 1`
done < "$input"
but this gives the error "whole expression required" and doesn't work at all.
If you have the option of using awk, the entire process can be reduced to:
awk -F, -v dist=150 '$4<dist {split($1,d,"."); print d[1]":"d[2]":"d[3]","$2}' file
Where in the example above, any record with distance (field 4, $4) less than the dist variable value takes the date field (field 1, $1) and splits() the field into the array d on "." where the first 3 elements will be year, mo, day and then simply prints the output of those three elements separated by ":" (which eliminates the stray "." at the end of the field). The time (field 2, $2) is output unchanged.
Example Use/Output
With your sample data in file, you can do:
$ awk -F, -v dist=150 '$4<dist {split($1,d,"."); print d[1]":"d[2]":"d[3]","$2}' file
2009:03:10,17:46:17:81
2018:01:05,00:30:37:04
2011:07:11,23:21:53:43
Which provides the records in the requested format where the distance is less than 150. If you call awk from within your script you can pass the 150 in from the 1st argument to your script.
You can also accomplish this task by substituting a ':' for each '.' in the first field with gsub() and outputting a substring of the first field with substr() that drops the last character, e.g.
awk -F, -v dist=150 '$4<dist {gsub(/[.]/,":",$1); print substr($1,0,length($1)-1),$2}' file
(same output)
While parsing the data is a great exercise for leaning string handling in shell or bash, in practice awk will be Orders of Magnitude faster than a shell script. Processing a million line file -- the difference in runtime can be seconds with awk compared to minutes (or hours) with a shell script.
If this is an exercise to learn string handling in your shell, just put this in your hip pocket for later understanding that awk is the real Swiss Army-Knife for text processing. (well worth the effort to learn)
Would you try the following:
#/bin/bash
if (( $# != 1 )); then
echo "usage: $0 max_distance_from_the_front" >& 2 # output error message to the stderr
exit 1
fi
input="auto.txt"
while IFS=, read -r mydate mytime speed fdist bdist; do # split csv and assign variables
mydate=${mydate%.}; mydate=${mydate//./:} # reformat the date string
if (( fdist < $1 )); then # if the front disatce is less than $1
echo "$mydate,$mytime" # then print the date and time
fi
done < "$input"
Sample output with the same parameter as Keldorn:
$ ./test.sh 130
2009:03:10,17:46:17:81
2011:07:11,23:21:53:43
There are a few odd things in your script:
Why is fdistance an array. It is not necessary (and here done wrong) since the file is read line by line.
What is the cut of the line fdistance=$(cut -d, -f 4) supposed to cut, what's the input?
(Note: When invalid parameters, better end the script right away. Added in the example below.)
Here is a working version (apart from the parsing of the date, but that is not what your question was about so I skipped it):
#!/usr/bin/env bash
if [ $# -eq 0 -o $# -gt 1 ]
then
echo "wrong number of parameters"
exit 1
fi
input='auto.txt'
while IFS= read -r line
do
fdistance=$(echo "$line" | awk '{split($0,a,","); print a[4]}')
if [ "$fdistance" -lt "$1" ]
then
echo $line
fi
done < "$input"
Sample output:
$ ./test.sh 130
2009.03.10.,17:46:17:81,57,102,57
2011.07.11.,23:21:53:43,310,125,47
$

shell - iterating a loop for a (validated) supplied number or range of numbers

I need to accept input from user (i.e. 'read').
This input can be either a single positive number or a range of numbers (in the form X-Y ).
I then need to validate this input and perform an iterative loop through the range of numbers (or just once in the case of only X).
examples:
1) User supplies: "8" or "8-"
Loop runs only a single time supplying the number 8
2) User supplies: "13-22"
Loop runs 11 times (13 through 22) referencing the number 13.
3) User supplies: "22-13"
Probably should behave like #2 above...but I'm open to other clean ways to validate the input.
I have worked on the following so far, which isn't very clean, complete, or even 100% correct for what I was asking - but it shows the effort and idea I'm going for:
echo "line?"; read -r deleteline
case "$deleteline" in
''|*[!0-9\-]*) echo "not a number";;
[1-9]|[1-9][0-9]*);;
esac
deleteline_lb=$(echo $deleteline|awk -F "-" '{print $1}')
deleteline_ub=$(echo $deleteline|awk -F "-" '{print $2}')
if [ ! $deleteline_lb = "" ] && [ ! "$deleteline_ub" = "" ]; then
delete_line_count=1
delete_line_count=$(expr $deleteline_ub - $deleteline_lb)
if [ $delete_line_count -le 0 ]; then
delete_line_count=1
fi
fi
i=1; while [ $i -le $delete_line_count ]; do
echo $deleteline_lb $i
i=$(($i + 1))
done
This needs to run in sh, things like seq are not supported - so stick with posix compliant methods...
To clarify I am looking to do the following (pseudo-code):
1) accept input from user
2) validate if input is in the form "#" or "#-#" (range).
3) Execute chosen (arbitrary) code path based on proper/improper input.
4) If single # is given then store that to variable to perform future operations against.
5) If range is given, store both numbers in variable to be able to perform the operation against the lower # up to the higher number. More specifically it would be "(higher #) - (lower #) + 1". So if range were 12-17 then we need to perform operation against 12, 6x. (17 - 12 + 1). IOW, 12-17 inclusive.
6) A way to easily denote if data set is range vs single number is also desired so that code path to each can be easily branched.
thanks for helping!
UPDATE:
Using my basic code I reworked it (with a bit of input from a friend), and basically came up with this:
while true;do
printf "\\delete lines? [e=Exit] ";read -r deleteline
case "$deleteline" in
[Ee]) break;;
*)
echo "$deleteline" | egrep -q '^[[:digit:]-]*$'
if [ $? -ne 0 ]; then
printf "\\n input is not a number.\\n"
else
delete_range_start=`echo $deleteline|awk -F "-" '{print $1}'`
delete_range_end=`echo $deleteline|awk -F "-" '{print $2}'`
if [ $delete_range_end -lt $delete_range_start ]; then
printf "\\n upper range must be higher than lower range.\\n"
else
if [ "$delete_range_end" = "" ]; then
delete_range_end=$delete_range_start
elif [ $delete_range_end -gt $lineNumbers ]; then
printf "\\Setting range to last entry\\n"
fi
break
fi
fi
;;
esac
done
deleteline=$delete_range_start
deleteloop=`expr $delete_range_end - $delete_range_start + 1`
i=1
while [ $i -le $deleteloop ]; do
# Insert all processing code in here
i=`expr $i + 1`
done
If you have a posix compliant awk, try this:
echo "$userInput" | awk -F- '
($1+0==$1&&$2+0==$2){
for(i=$1;($1<$2?i<=$2:i>=$2);)
print ($1<$2?i++:i--);
next
}
$1+0==$1{
print $1;
next
}
$2+0==$2{
print $2;
next
}
($1+0!=$1&&$2+0!=$2){
exit 1
}'
The script check if the 2 fields (separated with -) are numbers. If so, it prints these numbers in an ascending or descending way depending if the first number is greater or lower than the second one.
If only one input, the script just prints it.
If none of the field are number, it exits with a non zero value.
This script could be the validation step of a shell script like this:
$ cat test.sh
#!/bin/sh
echo -n "range: "
read -r range
validated_input=$(echo "$range" | awk -F- '($1+0==$1&&$2+0==$2){for(i=$1;($1<$2?i<=$2:i>=$2);)print ($1<$2?i++:i--);next}$1+0==$1{print $1;next}$2+0==$2{print $2;next}($1+0!=$1&&$2+0!=$2){exit 1}')
if [ $? -ne 0 ]; then
echo "Incorrect range" >&2
fi
for i in $validated_input; do
echo "$i"
done
Examples:
$ ./test.sh
range: 10-6
10
9
8
7
6
$ ./test.sh
range: 8-
8
$ ./test.sh
range: hello
Incorrect range

Process files in pairs

I have a list of files:
file_name_FOO31101.txt
file_name_FOO31102.txt
file_name_FOO31103.txt
file_name_FOO31104.txt
And I want to use pairs of files for input into a downstream program such as:
program_call file_name_01.txt file_name_02.txt
program_call file_name_03.txt file_name_04.txt
...
I do not want:
program_call file_name_02.txt file_name_03.txt
I need to do this in a loop as follows:
#!/bin/bash
FILES=path/to/files
for file in $FILES/*.txt;
do
stem=$( basename "${file}" ) # stem : file_name_FOO31104_info.txt
output_base=$( echo $stem | cut -d'_' -f 1,2,3 ) # output_base : FOO31104_info.txt
id=$( echo $stem | cut -d'_' -f 3 ) # get the first field : FOO31104
number=$( echo -n $id | tail -c 2 ) # get the last two digits : 04
echo $id $((id+1))
done
But this does not produce what I want.
In each loop I want to call a program once, with two files as input (last 2 digits of first file always odd 01, last 2 digits of second file always even 02)
I actually wouldn't use a for loop at all. A while loop that shifts files off is a perfectly reasonable way to do this.
# here, we're overriding the argument list with the list of files
# ...you can do this in a function if you want to keep the global argument list intact
set -- "$FILES"/*.txt ## without these quotes paths with spaces break
# handle the case where no files were found matching our glob
[[ -e $1 || -L $1 ]] || { echo "No .txt found in $FILES" >&2; exit 1; }
# here, we're doing our own loop over those arguments
while (( "$#" > 1 )); do ## continue in the loop only w/ 2-or-more remaining
echo "Processing files $1 and $2" ## ...substitute your own logic here...
shift 2 || break ## break even if test doesn't handle this case
done
# ...and add your own handling for the case where there's an odd number of files.
(( "$#" )) && echo "Left over file $1 still exists"
Note that the $#s are quoted inside (( )) here for StackOverflow's syntax highlighting, not because they otherwise need to be. :)
By the way -- consider using bash's native string manipulation.
stem=${file##*/}
IFS=_ read -r p1 p2 id p_rest <<<"$stem"
number=${id:$(( ${#id} - 2 ))}
output_base="${p1}${p2}${id}"
echo "$id $((10#number + 1))" # 10# ensures interpretation as decimal, not octal

bash script using multiple while loops and read line

I am trying to write a bash script to create some playlists of music. The part that has me stuck is the while loop for read line. I figure I am over thinking this so I turned to stackoverflow for assistance.
# The first while loop is how many playlists I want to create
i=1
while [ $i -le $plist ]
do
echo -e "iteration $i"
i=$[$i + 1]
z=0
# This while loop is for the length of time I want the playlist to be
while [ $z -le $TOTAL ]
do
echo -e "Count $z"
z=$[$z + xxx]
# This while loop is for reading the track list previously generated.
# It would read the line, calculate the track length,
# add to $z, cp the track to a folder
while read line
do
secs=$(metaflac --show-total-samples --show-sample-rate "$line" | tr '\n' ' '
| awk '{print $1/$2}' -)
z=$[$z + $secs]
cp $line to destination folder
done
done
done

Resources