I'm a bit confused by the done < $1 notation.
I'm trying to write a program "sumnums" that reads in a file called "nums" that has a couple rows of numbers. Then it should print out the rows of the numbers followed by a sum of all the numbers.
Currently I have:
#!/bin/bash
sum=0;
while read myline
do
echo "Before for; Current line: \"$myline\""
done
for i in $myline; do
sum=$(expr $sum + $i)
done < $1
echo "Total sum is: $sum"
and it outputs the list of the numbers from nums correctly then says
./sumnums: line 10: $1: ambiguous redirect, then outputs Total sum is: 0.
So somehow it isn't adding. How do I rearrange these lines to fix the program and get rid of the "ambiguous redirect"?
Assuming your filename is in $1 (that is, that your script was called with ./yourscript nums):
#!/bin/bash
[[ $1 ]] || set -- nums ## use $1 if already set; otherwise, override with "nums"
sum=0
while read -r i; do ## read from stdin (which is redirected by < for this loop)
sum=$(( sum + i )) ## ...treat what we read as a number, and add it to our sum
done <"$1" ## with stdin reading from $1 for this loop
echo "Total sum is: $sum"
If $1 doesn't contain your filename, then use something that does contain your filename in its place, or just hardcode the actual filename itself.
Notes:
<"$1" is applied to a while read loop. This is essential, because read is (in this context) the command that actually consumes content from the file. It can make sense to redirect stdin to a for loop, but only if something inside that loop is reading from stdin.
$(( )) is modern POSIX sh arithmetic syntax. expr is legacy syntax; don't use it.
awk to the rescue!
awk '{for(i=1;i<=NF;i++) sum+=$i} END{print "Total sum is: " sum}' file
bash is not the right tool for this task.
Related
I have a file which contains varoius data (date,time,speed, distance from the front, distance from the back), the file looks like this, just with more rows:
2003.09.23.,05:05:21:64,134,177,101
2009.03.10.,17:46:17:81,57,102,57
2018.01.05.,00:30:37:04,354,145,156
2011.07.11.,23:21:53:43,310,125,47
2011.06.26.,07:42:10:30,383,180,171
I'm trying to write a simple Bash program, which tells the dates and times when the 'distance from the front' is less than the provided parameter ($1)
So far I wrote:
#!/bin/bash
if [ $# -eq 0 -o $# -gt 1 ]
then
echo "wrong number of parameters"
fi
i=0
fdistance=()
input='auto.txt'
while IFS= read -r line
do
year=${line::4}
month=${line:5:2}
day=${line:8:2}
hour=${line:12:2}
min=${line:15:2}
sec=${line:18:2}
hthsec=${line:21:2}
fdistance=$(cut -d, -f 4)
if [ "$fdistance[$i]" -lt "$1" ]
then
echo "$year[$i]:$month[$i]:$day[$i],$hour[$i]:$min[$i]:$sec[$i]:$hthsec[$i]"
fi
i=`expr $i + 1`
done < "$input"
but this gives the error "whole expression required" and doesn't work at all.
If you have the option of using awk, the entire process can be reduced to:
awk -F, -v dist=150 '$4<dist {split($1,d,"."); print d[1]":"d[2]":"d[3]","$2}' file
Where in the example above, any record with distance (field 4, $4) less than the dist variable value takes the date field (field 1, $1) and splits() the field into the array d on "." where the first 3 elements will be year, mo, day and then simply prints the output of those three elements separated by ":" (which eliminates the stray "." at the end of the field). The time (field 2, $2) is output unchanged.
Example Use/Output
With your sample data in file, you can do:
$ awk -F, -v dist=150 '$4<dist {split($1,d,"."); print d[1]":"d[2]":"d[3]","$2}' file
2009:03:10,17:46:17:81
2018:01:05,00:30:37:04
2011:07:11,23:21:53:43
Which provides the records in the requested format where the distance is less than 150. If you call awk from within your script you can pass the 150 in from the 1st argument to your script.
You can also accomplish this task by substituting a ':' for each '.' in the first field with gsub() and outputting a substring of the first field with substr() that drops the last character, e.g.
awk -F, -v dist=150 '$4<dist {gsub(/[.]/,":",$1); print substr($1,0,length($1)-1),$2}' file
(same output)
While parsing the data is a great exercise for leaning string handling in shell or bash, in practice awk will be Orders of Magnitude faster than a shell script. Processing a million line file -- the difference in runtime can be seconds with awk compared to minutes (or hours) with a shell script.
If this is an exercise to learn string handling in your shell, just put this in your hip pocket for later understanding that awk is the real Swiss Army-Knife for text processing. (well worth the effort to learn)
Would you try the following:
#/bin/bash
if (( $# != 1 )); then
echo "usage: $0 max_distance_from_the_front" >& 2 # output error message to the stderr
exit 1
fi
input="auto.txt"
while IFS=, read -r mydate mytime speed fdist bdist; do # split csv and assign variables
mydate=${mydate%.}; mydate=${mydate//./:} # reformat the date string
if (( fdist < $1 )); then # if the front disatce is less than $1
echo "$mydate,$mytime" # then print the date and time
fi
done < "$input"
Sample output with the same parameter as Keldorn:
$ ./test.sh 130
2009:03:10,17:46:17:81
2011:07:11,23:21:53:43
There are a few odd things in your script:
Why is fdistance an array. It is not necessary (and here done wrong) since the file is read line by line.
What is the cut of the line fdistance=$(cut -d, -f 4) supposed to cut, what's the input?
(Note: When invalid parameters, better end the script right away. Added in the example below.)
Here is a working version (apart from the parsing of the date, but that is not what your question was about so I skipped it):
#!/usr/bin/env bash
if [ $# -eq 0 -o $# -gt 1 ]
then
echo "wrong number of parameters"
exit 1
fi
input='auto.txt'
while IFS= read -r line
do
fdistance=$(echo "$line" | awk '{split($0,a,","); print a[4]}')
if [ "$fdistance" -lt "$1" ]
then
echo $line
fi
done < "$input"
Sample output:
$ ./test.sh 130
2009.03.10.,17:46:17:81,57,102,57
2011.07.11.,23:21:53:43,310,125,47
$
I have created a hex to ASCII converter for strings in bash. The application I'm on changes characters (anything but [0-9],[A-Z],[a-z]) , in a string to its corresponding %hexadecimal. Eg: / changes to %2F in a string
I want to retain the ASCII characters as it is. Below is my code:
NAME=%2fhome%40%21%23
C_NAME=""
for (( i=0; i<${#NAME}; i++ )); do
CHK=$(echo "{NAME:$i:1}" | grep -v "\%" &> /dev/null;echo $?)
if [[ ${CHK} -eq 0 ]]; then
C_NAME=`echo "$C_NAME${NAME:$i:1}"`
else
HEX=`echo "${NAME:$i:3}" | sed "s/%//"`
C_NAME=`echo -n "$C_NAME";printf "\x$HEX"`
continue 2
fi
done
echo "$C_NAME"
OUTPUT:
/2fhome#40!21#23
EXPECTED:
/home#!#
So basically the conversion is happening, but not in place. Its retaining the hex values as well, which tells me the continue 2 statement is probably not working as I expect in my code. Any workarounds please.
You only have one loop so I assume you expected that continue 2 skips the current and next iteration of the current loop, however, the documentation help continue clearly states
continue [n]
[...]
If N is specified, resumes the Nth enclosing loop.
There is no built-in to skip the current and also the next iteration of the current loop, but in your case you can use (( i += 2 )) instead of continue 2.
Using the structure of your script with some simplifications and corrections:
#!/bin/bash
name=%2fhome%40%21%23
c_name=""
for (( i=0; i<${#name}; i++ )); do
c=${name:i:1}
if [[ $c != % ]]; then
c_name=$c_name$c
else
hex=${name:i+1:2}
printf -v c_name "%s\x$hex" "$c_name"
(( i += 2 )) # stolen from Dudi Boy's answer
fi
done
echo "$c_name"
Always use lower case or mixed case variables to avoid the chance of name collisions with shell or environment variables
Always use $() instead of backticks
Most of the echo commands you use aren't necessary
You can avoid using sed and grep
Variables should never be included in the format string of printf but it can't be avoided easily here (you could use echo -e "\x$hex" instead though)
You can do math inside parameter expansions
% doesn't need to be escaped in your grep command
You could eliminate the $hex variable if you used its value directly:
printf -v c_name "%s\x${name:i+1:2}" "$c_name"
I really enjoyed your exercise and decided to solve it with awk (my current study).
Hope you like it as well.
cat script.awk
BEGIN {RS = "%[[:xdigit:]]+"} { # redefine record separtor to RegEx (gawk specific)
decNum = strtonum("0x"substr(RT, 2)); # remove prefix # from record separator, convert hex num to dec
outputStr = outputStr""$0""sprintf("%c", decNum); # reconstruct output string
}
END {print outputStr}
The output
echo %2fhome%40%21%23 |awk -f script.awk
/home#!#
OK so Ive been at this for a couple days,im new to this whole bash UNIX system thing i just got into it but I am trying to write a script where the user inputs an integer and the script will take that integer and print out a triangle using the integer that was inputted as a base and decreasing until it reaches zero. An example would be:
reverse_triangle.bash 4
****
***
**
*
so this is what I have so far but when I run it nothing happens I have no idea what is wrong
#!/bin/bash
input=$1
count=1
for (( i=$input; i>=$count;i-- ))
do
for (( j=1; j>=i; j++ ))
do
echo -n "*"
done
echo
done
exit 0
when I try to run it nothing happens it just goes to the next line. help would be greatly appreciated :)
As I said in a comment, your test is wrong: you need
for (( j=1; j<=i; j++ ))
instead of
for (( j=1; j>=i; j++ ))
Otherwise, this loop is only executed when i=1, and it becomes an infinite loop.
Now if you want another way to solve that, in a much better way:
#!/bin/bash
[[ $1 = +([[:digit:]]) ]] || { printf >&2 'Argument must be a number\n'; exit 1; }
number=$((10#$1))
for ((;number>=1;--number)); do
printf -v spn '%*s' "$number"
printf '%s\n' "${spn// /*}"
done
Why is it better? first off, we check that the argument is really a number. Without this, your code is subject to arbitrary code injection. Also, we make sure that the number is understood in radix 10 with 10#$1. Otherwise, an argument like 09 would raise an error.
We don't really need an extra variable for the loop, the provided argument is good enough. Now the trick: to print n times a pattern, a cool method is to store n spaces in a variable with printf: %*s will expand to n spaces, where n is the corresponding argument found by printf.
For example:
printf '%s%*s%s\n' hello 42 world
would print:
hello world
(with 42 spaces).
Editor's note: %*s will NOT generally expand to n spaces, as evidenced by above output, which contains 37 spaces.
Instead, the argument that * is mapped to,42, is the field width for the sfield, which maps to the following argument,world, causing string world to be left-space-padded to a length of 42; since world has a character count of 5, 37 spaces are used for padding.
To make the example work as intended, use printf '%s%*s%s\n' hello 42 '' world - note the empty string argument following 42, which ensures that the entire field is made up of padding, i.e., spaces (you'd get the same effect if no arguments followed 42).
With printf's -v option, we can store any string formatted by printf into a variable; here we're storing $number spaces in spn. Finally, we replace all spaces by the character *, using the expansion ${spn// /*}.
Yet another possibility:
#!/bin/bash
[[ $1 = +([[:digit:]]) ]] || { printf >&2 'Argument must be a number\n'; exit 1; }
printf -v s '%*s' $((10#1))
s=${s// /*}
while [[ $s ]]; do
printf '%s\n' "$s"
s=${s%?}
done
This time we construct the variable s that contains a bunch of * (number given by user), using the previous technique. Then we have a while loop that loops while s is non empty. At each iteration we print the content of s and we remove a character with the expansion ${s%?} that removes the last character of s.
Building on gniourf_gniourf's helpful answer:
The following is simpler and performs significantly better:
#!/bin/bash
count=$1 # (... number-validation code omitted for brevity)
# Create the 1st line, composed of $count '*' chars, and store in var. $line.
printf -v line '%.s*' $(seq $count)
# Count from $count down to 1.
while (( count-- )); do
# Print a *substring* of the 1st line based on the current value of $count.
printf "%.${count}s\n" "$line"
done
printf -v line '*%.s' $(seq $count) is a trick that prints * $count times, thanks to %.s* resulting in * for each argument supplied, irrespective of the arguments' values (thanks to %.s, which effectively ignores its argument). $(seq $count) expands to $count arguments, resulting in a string composed of $count * chars. overall, which - thanks to -v line, is stored in variable $line.
printf "%.${count}s\n" "$line" prints a substring from the beginning of $line that is $count chars. long.
I'm gonna tear my hair out: I have this script:
#!/bin/bash
if [[ $# -eq 2 ]]
then
total=0
IFS=' '
while read one two; do
total=$((total+two))
done < $2
echo "Total: $total"
fi
Its supposed to add up my gas receipts I have saved in a file in this format:
3/9/13 21.76
output:
./getgas: line 9: 21.76: syntax error: invalid arithmetic operator (error token is ".76")
I read online that its possible to do float math in bash, and I found an an example script that works and it has:
function float_eval()
{
local stat=0
local result=0.0
if [[ $# -gt 0 ]]; then
result=$(echo "scale=$float_scale; $*" | bc -q 2>/dev/null)
stat=$?
if [[ $stat -eq 0 && -z "$result" ]]; then stat=1; fi
fi
echo $result
return $stat
}
which looks awesome, and runs no problem
WTF is going on here. I can easily do this is C but this crap is making me mad
EDIT: I don't anything about awk. It looks promising but I don't even know how to run those one-liners you guys posted
awk '{ sum += $2 } END { printf("Total: %.2f\n", sum); }' $2
Add up column 2 (that's the $2 in the awk script) of the file named by shell script argument $2 (rife with opportunities for confusion) and print the result at the end.
I don't [know] anything about awk. It looks promising but I don't even know how to run those one-liners you guys posted.
In the context of your script:
#!/bin/bash
if [[ $# -eq 2 ]]
then
awk '{ sum += $2 } END { printf("Total: %.2f\n", sum); }' $2
else
echo "Usage: $0 arg1 receipts-file" >&2; exit 1
fi
Or just write it on the command line, substituting the receipts file name for the $2 after the awk command. Or leave that blank and redirect from the file. Or type the dates and values in. Or, …
Your script demands two arguments, but doesn't use the first one, which is a bit puzzling.
As noted in the comments, you could simplify that to:
#!/bin/bash
exec awk '{ sum += $2 } END { printf("Total: %.2f\n", sum) }' "$#"
Or even use the shebang to full power:
#!/usr/bin/awk -f
{ sum += $2 }
END { printf("Total: %.2f\n", sum) }
The kernel will execute awk for you, and that's the awk script written out as a two line program. Of course, if awk is in /bin/awk, then you have to fix the shebang line; the shell looks in many places for awk and will probably find it. So there are advantages to sticking with a shell script. Both these revisions simply sum what's on standard input if there are no files specified, or what is in all the files specified if there is one or more files specified on the command line.
In bash you can only operate on integers. The example script you posted uses bc which is an arbitrary-precision calculation, included with most UNIX-like OS-es. So the script prepares an expression and pipes it to bc (the initial scale=... expression configures the number of significant digits bc should display.
A simplified example would be:
echo -e 'scale=2\n1.234+5.67\nquit' | bc
You could also use awk:
awk 'BEGIN{print 1.234+5.67}'
Say you have a simple loop
while read line
do
printf "${line#*//}\n"
done < text.txt
Is there an elegant way of printing the current iteration with the output? Something like
0 The
1 quick
2 brown
3 fox
I am hoping to avoid setting a variable and incrementing it on each loop.
To do this, you would need to increment a counter on each iteration (like you are trying to avoid).
count=0
while read -r line; do
printf '%d %s\n' "$count" "${line*//}"
(( count++ ))
done < test.txt
EDIT: After some more thought, you can do it without a counter if you have bash version 4 or higher:
mapfile -t arr < test.txt
for i in "${!arr[#]}"; do
printf '%d %s' "$i" "${arr[i]}"
done
The mapfile builtin reads the entire contents of the file into the array. You can then iterate over the indices of the array, which will be the line numbers and access that element.
You don't often see it, but you can have multiple commands in the condition clause of a while loop. The following still requires an explicit counter variable, but the arrangement may be more suitable or appealing for some uses.
while ((i++)); read -r line
do
echo "$i $line"
done < inputfile
The while condition is satisfied by whatever the last command returns (read in this case).
Some people prefer to include the do on the same line. This is what that would look like:
while ((i++)); read -r line; do
echo "$i $line"
done < inputfile
You can use a range to go through, it can be an array, a string, a input line or a list.
In this example, i use a list of numbers [0..10] is used with an increment of 2, as well.
#!/bin/bash
for i in {0..10..2}; do
echo " $i times"
done
The output is:
0 times
2 times
4 times
6 times
8 times
10 times
To print the index regardless of the loop range, you have to use a variable "COUNTER=0" and increase it in each iteration "COUNTER+1".
my solution prints each iteration, the FOR traverses an inputline and increments by one each iteration, also shows each of words in the inputline:
#!/bin/bash
COUNTER=0
line="this is a sample input line"
for word in $line; do
echo "This i a word number $COUNTER: $word"
COUNTER=$((COUNTER+1))
done
The output is:
This i a word number 0: this
This i a word number 1: is
This i a word number 2: a
This i a word number 3: sample
This i a word number 4: input
This i a word number 5: line
to see more about loops: enter link description here
to test your scripts: enter link description here
n=0
cat test.txt | while read line; do
printf "%7s %s\n" "$n" "${line#*//}"
n=$((n+1))
done
This will work in Bourne shell as well, of course.
If you really want to avoid incrementing a variable, you can pipe the output through grep or awk:
cat test.txt | while read line; do
printf " %s\n" "${line#*//}"
done | grep -n .
or
awk '{sub(/.*\/\//, ""); print NR,$0}' test.txt
Update: Other answers posted here are better, especially those of #Graham and #DennisWilliamson.
Something very like this should suit:
tr -s ' ' '\n' <test.txt | nl -ba
You can add a -v0 flag to the nl command if you want indexing from 0.