I have a file which contains varoius data (date,time,speed, distance from the front, distance from the back), the file looks like this, just with more rows:
2003.09.23.,05:05:21:64,134,177,101
2009.03.10.,17:46:17:81,57,102,57
2018.01.05.,00:30:37:04,354,145,156
2011.07.11.,23:21:53:43,310,125,47
2011.06.26.,07:42:10:30,383,180,171
I'm trying to write a simple Bash program, which tells the dates and times when the 'distance from the front' is less than the provided parameter ($1)
So far I wrote:
#!/bin/bash
if [ $# -eq 0 -o $# -gt 1 ]
then
echo "wrong number of parameters"
fi
i=0
fdistance=()
input='auto.txt'
while IFS= read -r line
do
year=${line::4}
month=${line:5:2}
day=${line:8:2}
hour=${line:12:2}
min=${line:15:2}
sec=${line:18:2}
hthsec=${line:21:2}
fdistance=$(cut -d, -f 4)
if [ "$fdistance[$i]" -lt "$1" ]
then
echo "$year[$i]:$month[$i]:$day[$i],$hour[$i]:$min[$i]:$sec[$i]:$hthsec[$i]"
fi
i=`expr $i + 1`
done < "$input"
but this gives the error "whole expression required" and doesn't work at all.
If you have the option of using awk, the entire process can be reduced to:
awk -F, -v dist=150 '$4<dist {split($1,d,"."); print d[1]":"d[2]":"d[3]","$2}' file
Where in the example above, any record with distance (field 4, $4) less than the dist variable value takes the date field (field 1, $1) and splits() the field into the array d on "." where the first 3 elements will be year, mo, day and then simply prints the output of those three elements separated by ":" (which eliminates the stray "." at the end of the field). The time (field 2, $2) is output unchanged.
Example Use/Output
With your sample data in file, you can do:
$ awk -F, -v dist=150 '$4<dist {split($1,d,"."); print d[1]":"d[2]":"d[3]","$2}' file
2009:03:10,17:46:17:81
2018:01:05,00:30:37:04
2011:07:11,23:21:53:43
Which provides the records in the requested format where the distance is less than 150. If you call awk from within your script you can pass the 150 in from the 1st argument to your script.
You can also accomplish this task by substituting a ':' for each '.' in the first field with gsub() and outputting a substring of the first field with substr() that drops the last character, e.g.
awk -F, -v dist=150 '$4<dist {gsub(/[.]/,":",$1); print substr($1,0,length($1)-1),$2}' file
(same output)
While parsing the data is a great exercise for leaning string handling in shell or bash, in practice awk will be Orders of Magnitude faster than a shell script. Processing a million line file -- the difference in runtime can be seconds with awk compared to minutes (or hours) with a shell script.
If this is an exercise to learn string handling in your shell, just put this in your hip pocket for later understanding that awk is the real Swiss Army-Knife for text processing. (well worth the effort to learn)
Would you try the following:
#/bin/bash
if (( $# != 1 )); then
echo "usage: $0 max_distance_from_the_front" >& 2 # output error message to the stderr
exit 1
fi
input="auto.txt"
while IFS=, read -r mydate mytime speed fdist bdist; do # split csv and assign variables
mydate=${mydate%.}; mydate=${mydate//./:} # reformat the date string
if (( fdist < $1 )); then # if the front disatce is less than $1
echo "$mydate,$mytime" # then print the date and time
fi
done < "$input"
Sample output with the same parameter as Keldorn:
$ ./test.sh 130
2009:03:10,17:46:17:81
2011:07:11,23:21:53:43
There are a few odd things in your script:
Why is fdistance an array. It is not necessary (and here done wrong) since the file is read line by line.
What is the cut of the line fdistance=$(cut -d, -f 4) supposed to cut, what's the input?
(Note: When invalid parameters, better end the script right away. Added in the example below.)
Here is a working version (apart from the parsing of the date, but that is not what your question was about so I skipped it):
#!/usr/bin/env bash
if [ $# -eq 0 -o $# -gt 1 ]
then
echo "wrong number of parameters"
exit 1
fi
input='auto.txt'
while IFS= read -r line
do
fdistance=$(echo "$line" | awk '{split($0,a,","); print a[4]}')
if [ "$fdistance" -lt "$1" ]
then
echo $line
fi
done < "$input"
Sample output:
$ ./test.sh 130
2009.03.10.,17:46:17:81,57,102,57
2011.07.11.,23:21:53:43,310,125,47
$
I have bash script which checks presence of certain files and that the content has a valid format. It uses variable prefixes so i can easily add/remove new files w/o the need of further adjustments.
Problem is that i need to run this on AIX servers where bash is not present. I've adjusted the script except the part with variable prefixes. After some attempts i am lost and have no idea how to properly migrate the following piece of code so it runs under sh ( $(echo ${!ifile_#}) ). Alternatively i have ksh or csh if plain sh is not an option.
Thank you in advance for any help/hints
#!/bin/sh
# Source files
ifile_one="/path/to/file/one.csv"
ifile_two="/path/to/file/two.csv"
ifile_three="/path/to/file/three.csv"
ifile_five="/path/to/file/four.csv"
min_columns='10'
existing_files=""
nonexisting_files=""
valid_files=""
invalid_files=""
# Check that defined input-files exists and can be read.
for input_file in $(echo ${!ifile_#})
do
if [ -r ${!input_file} ]; then
existing_files+="${!input_file} "
else
nonexisting_files+="${!input_file} "
fi
done
echo "$existing_files"
echo "$nonexisting_files"
# Check that defined input files have proper number of columns.
for input_file_a in $(echo "$existing_files")
do
check=$(grep -v "^$" $input_file_a | sed 's/[^;]//g' | awk -v min_columns="$min_columns" '{ if (length == min_columns) {print "OK"} else {print "KO"} }' | grep -i KO)
if [ ! -z "$check" ]; then
invalid_files+="${input_file_a} "
else
valid_files+="${input_file_a} "
fi
done
echo "$invalid_files"
echo "$valid_files"
Bash returns expected output (of the four ECHOes):
/path/to/file/one.csv /path/to/file/two.csv /path/to/file/three.csv
/path/to/file/four.csv
/path/to/file/three.csv
/path/to/file/one.csv /path/to/file/two.csv
ksh/sh throws:
./report.sh[14]: "${!ifile_#}": 0403-011 The specified substitution is not valid for this command.
Thanks #Benjamin W. and #user1934428 , ksh93 arrays are the answer.
So bellow code works for me as desired.
#!/bin/ksh93
typeset -A ifile
ifile[one]="/path/to/file/one.csv"
ifile[two]="/path/to/file/two.csv"
ifile[three]="/path/to/file/three.csv"
ifile[whatever]="/path/to/file/something.csv"
existing_files=""
nonexisting_files=""
for input_file in "${!ifile[#]}"
do
if [ -r ${ifile[$input_file]} ]; then
existing_files+="${ifile[$input_file]} "
else
nonexisting_files+="${ifile[$input_file]} "
fi
done
I'm working on a script that compares two very similar decimal numbers and I want the script to print the portion that the two numbers share. For example, suppose I have the numbers 42.86579 and 42.84578. Since both numbers have the 42.8 portion in common, I would like the script to output 42.8. How should I go about implementing this?
You could search for the longest common prefix in two strings with sed:
$ x=42.86579
$ y=42.84578
$ sed "s/\(.*\).* \1.*/\1/" <<< "$x $y"
42.8
Or slightly more concisely using GNU grep:
$ grep -Po '(.*).* \K\1' <<< "$x $y"
42.8
a=42.86579
b=42.84578
[[ ${a%.*} != ${b%.*} ]] && exit
for ((i=0;i<${#a};i++)); do
if [[ ${a:$i:1} == ${b:$i:1} ]]; then
echo -n ${a:$i:1}
else
break
fi
done
Output:
42.8
See: Bash's parameter expansion
The output comes from a command I run from our netscaler. It outputs the following ... One thing to note is that the middle two numbers change but the even/odd criteria is always on the last digit. We never have more than 2 digits, so we'll never hit 10.
WC-01-WEB1
WC-01-WEB4
WC-01-WEB3
WC-01-WEB5
WC-01-WEB8
I need to populate a file called "even" and "odds." If we're dealing with numbers I can figure it out, but having the number within a string is throwing me off.
Example code but I'm missing the part where I need to match the string.
if [ $even_servers -eq 0 ]
then
echo $line >> evenfile
else
echo $line >> oddfile
fi
This is a simple awk command:
awk '/[02468]$/{print > "evenfile"}; /[13579]$/{print > "oddfile"}' input.txt
There must be better way.
How about this version:
for v in `cat <my_file>`; do export type=`echo $v | awk -F 'WEB' '{print $2%2}'`; if [ $type -eq 0 ]; then echo $v >> evenfile ; else echo $v >> oddfile; fi; done
I assume your list of servers is stored in the filename <my_file>. The basic idea is to tokenize on WEB using awk and process the chars after WEB to determine even-ness. Once this is known, we export the value to a variable type and use this to selectively dump to the appropriate file.
For the case when the name is the output of another command:
export var=`<another command>`; export type=`echo $var | awk -F 'WEB' '{print $2%2}'`; if [ $type -eq 0 ]; then echo $var >> evenfile ; else echo $var >> oddfile; fi;
Replace <another command> with your perl script.
As always grep is your friend:
grep "[2468]$" input_file > evenfile
grep "[^2468]$" input_file > oddfile
I hope this helps.
I'm gonna tear my hair out: I have this script:
#!/bin/bash
if [[ $# -eq 2 ]]
then
total=0
IFS=' '
while read one two; do
total=$((total+two))
done < $2
echo "Total: $total"
fi
Its supposed to add up my gas receipts I have saved in a file in this format:
3/9/13 21.76
output:
./getgas: line 9: 21.76: syntax error: invalid arithmetic operator (error token is ".76")
I read online that its possible to do float math in bash, and I found an an example script that works and it has:
function float_eval()
{
local stat=0
local result=0.0
if [[ $# -gt 0 ]]; then
result=$(echo "scale=$float_scale; $*" | bc -q 2>/dev/null)
stat=$?
if [[ $stat -eq 0 && -z "$result" ]]; then stat=1; fi
fi
echo $result
return $stat
}
which looks awesome, and runs no problem
WTF is going on here. I can easily do this is C but this crap is making me mad
EDIT: I don't anything about awk. It looks promising but I don't even know how to run those one-liners you guys posted
awk '{ sum += $2 } END { printf("Total: %.2f\n", sum); }' $2
Add up column 2 (that's the $2 in the awk script) of the file named by shell script argument $2 (rife with opportunities for confusion) and print the result at the end.
I don't [know] anything about awk. It looks promising but I don't even know how to run those one-liners you guys posted.
In the context of your script:
#!/bin/bash
if [[ $# -eq 2 ]]
then
awk '{ sum += $2 } END { printf("Total: %.2f\n", sum); }' $2
else
echo "Usage: $0 arg1 receipts-file" >&2; exit 1
fi
Or just write it on the command line, substituting the receipts file name for the $2 after the awk command. Or leave that blank and redirect from the file. Or type the dates and values in. Or, …
Your script demands two arguments, but doesn't use the first one, which is a bit puzzling.
As noted in the comments, you could simplify that to:
#!/bin/bash
exec awk '{ sum += $2 } END { printf("Total: %.2f\n", sum) }' "$#"
Or even use the shebang to full power:
#!/usr/bin/awk -f
{ sum += $2 }
END { printf("Total: %.2f\n", sum) }
The kernel will execute awk for you, and that's the awk script written out as a two line program. Of course, if awk is in /bin/awk, then you have to fix the shebang line; the shell looks in many places for awk and will probably find it. So there are advantages to sticking with a shell script. Both these revisions simply sum what's on standard input if there are no files specified, or what is in all the files specified if there is one or more files specified on the command line.
In bash you can only operate on integers. The example script you posted uses bc which is an arbitrary-precision calculation, included with most UNIX-like OS-es. So the script prepares an expression and pipes it to bc (the initial scale=... expression configures the number of significant digits bc should display.
A simplified example would be:
echo -e 'scale=2\n1.234+5.67\nquit' | bc
You could also use awk:
awk 'BEGIN{print 1.234+5.67}'