For a framework I'm writing I would like to measure how much a piece of (bash) code takes to execute and then print the elapsed time in seconds.
For measuring, I do the following:
start=$(date +%s%N)
# do something here
elapsed_time=$(($(date +%s%N) - start))
This gives me the elapsed seconds concatenated with the elapsed nanos. If I now divide this by 1000000, I'll get the time in ms
elapsed_time_in_ms=$(($elapsed time / 1000000))
This is all nice and seems to work but the real problem is that I want to print it in this format:
12.789s
where before the . are the seconds and after the dot are last 3 digits of the ms value.
How would I achieve something like that?
EDIT
I am aware that the time would probably not of much use, still I would like to implement this (even if only for cosmetic reasons :-)).
EDIT 2
For anyone facing the same problem:
In the end I've chosen to use time as it doesn't require a fork and seems to be the most portable solution.
Have a look at the it function and the global total_elapsed_time variable here to see how I implemented this.
You can use bc command:
elapsed_time_in_ms=$(echo "scale=3;$elapsed_time/1000000" | bc)
The scale basically sets the number of digits you want after the .
This can also be done using shell parameter expansion or via printf. For example, running the following two sequences one after the other printed out 1349883230.715 and 1349883230.721 in one run, and 1349884003.025 and 1349884003.032 in another. (The %N date format fills with leading zeroes.)
s=$(date +%s.%N); s=${s%??????}; echo $s
t=$(printf "%20.3f" $(date +%s.%N)); echo $t
As mentioned in man bash under Pattern Matching, special pattern character ? matches any single character. As mentioned under Parameter Expansion, the form
${parameter%word}
removes a matching suffix pattern: “If the pattern matches a trailing portion ... the expansion is the expanded value of parameter with the shortest matching pattern (the "%" case) or the longest matching pattern (the "%%" case) deleted.”
Just cut the digits after the first three off from the nanoseconds.
printf "%d.%.3ss\n" date +%S date +%N # It might not be terribly unwise to run date only once, btw.
-- print a digit followed by a dot, then treat the next argument as a string and print only the first three characters.
$ i=0; while [ $i -lt 9 ];do i=$((i+1)); sleep 0.1; \
printf "%d.%.3ss\n" `date +%S` `date +%N` ;done
33.917s
34.025s
34.133s
34.240s
34.348s
34.457s
34.566s
34.674s
34.784s
As using bc implie forks which take time, I prefer this pure bash solution:
start=$(date +%s%N);sleep 1.1;elapsed_time=$(($(date +%s%N) - start))
_Z9=000000000
[ ${#elapsed_time} -lt 9 ] && \
elapsed_time=${_Z9:0:9-${#elapsed_time}}$elapsed_time;
printf "%.3fs\n" ${elapsed_time:0:${#elapsed_time}-9
}.${elapsed_time:${#elapsed_time}-9}
print:
1.107s
For that kind of things, I've wrote a little bash source file that will do
the job quickly.
It use on Linux /proc/timer_list, or on Linux /proc/uptime or date +%s%n not well tested on other systems (feed-back would be welcome ;).
It use two counters (one for each invocation and the other as an overall counter) and accept some arguments (read comments):
. elap.bash
elap -R ; for i in {1..10};do sleep .1; elap Counter $i;done;elap -t total
0.110488478 Counter 1
0.111014783 Counter 2
0.117158015 Counter 3
0.112897232 Counter 4
0.111928207 Counter 5
0.108822248 Counter 6
0.113464053 Counter 7
0.117487421 Counter 8
0.115716626 Counter 9
0.110493433 Counter 10
0.008513430 1.137983926 total
And, as bc could be useful:
time echo 'scale=1000;pi=4*a(1);0'|bc -l
0
real 0m1.590s
user 0m0.768s
sys 0m0.008s
or
elap -R;elap BC answer is: $(echo 'scale=1000;pi=4*a(1);0'|bc -l )
1.539957483 BC answer is: 0
Using this script with /proc/timer_list could do:
elap -R ; for i in {1..10};do elap Counter $i;done;elap -t total
0.001299574 Counter 1
0.001574097 Counter 2
0.005771637 Counter 3
0.001428803 Counter 4
0.010423721 Counter 5
0.004037965 Counter 6
0.001392464 Counter 7
0.008092812 Counter 8
0.001634280 Counter 9
0.001365652 Counter 10
0.008201473 0.045222478 total
While same script whithout access to /proc, using fork to date +%s%N give:
elap -R ; for i in {1..10};do elap Counter $i;done;elap -t total
0.012148259 Counter 1
0.013415551 Counter 2
0.008279329 Counter 3
0.013700332 Counter 4
0.012837796 Counter 5
0.015562929 Counter 6
0.008062369 Counter 7
0.016810494 Counter 8
0.011537439 Counter 9
0.009731194 Counter 10
0.012959840 0.135045532 total
Where we could see that fork have a cost (near 1/100th sec in this case).
Well, finally, for matching exact format for SO question this little script could be patched, for sample in this way:
eval "$(sed -e < elap.bash '
/6d/{ s/6d.%/10.3f/g;p;N;
:a;
s/^.*\n//g;N;s/" \\\n[ \t]*"/./; p;
s/^.*//g; N;/elaP_elap2/ba; }')"
elap -R ; for i in {1..10};do elap Counter $i;done;elap -t total
0.001s Counter 1
0.006s Counter 2
0.007s Counter 3
0.004s Counter 4
0.003s Counter 5
0.002s Counter 6
0.001s Counter 7
0.001s Counter 8
0.006s Counter 9
0.002s Counter 10
0.004s 0.038s total
38s total
Related
I need to create a bash script that creates 100 files with random numbers in them.
I tried to do it using:
for i in {1..100}; do $RANDOM > RANDOM.txt
I don't know if that's the correct way to do it.
And then I need to give the files reading writing and execution permissions based on the number inside the file. No idea how to do that.
I tried using:
if [ $i%2==0 ]
then
echo chmod +rw $RANDOM.txt
but that doesn't seem to work
Just got some feedback, it turns out I was doing everything wrong.
I had to create 100 files 1.txt to 100.txt I used touch {1..100}.txt and then paste 1 random number in each of those files. Should I use echo or shuf to do this?
I think it'd be simplest to use chmod with octal permissions, like 0777 for rwxrwxrwx etc.
Example:
#!/bin/bash
for((i=0; i<100; ++i)) {
rnd=$RANDOM # pick a random number
(( perm = rnd % 512 )) # make it in the range [0, 512) (0000-0777 oct)
printf -v octperm "%04o" $perm # convert to octal
file=$rnd.txt # not sure if you meant to name it after the number
echo $rnd > $file
chmod $octperm $file # chmod with octal number
}
Excerpt of files:
-r-xrw--wx 1 ted users 5 15 dec 17.53 6515.txt
---x-wxrwx 1 ted users 5 15 dec 17.53 6751.txt
-rwx-w--w- 1 ted users 5 15 dec 17.53 8146.txt
-rw-r----- 1 ted users 5 15 dec 17.53 8608.txt
--w--w---x 1 ted users 5 15 dec 17.53 8849.txt
--wx----wx 1 ted users 5 15 dec 17.53 8899.txt
--wxrwx-wx 1 ted users 5 15 dec 17.53 8955.txt
-rw-r-xrw- 1 ted users 5 15 dec 17.53 9134.txt
...
If you want to take your current umask into consideration, you could do that too, by masking away the bits in the permission indicated by the umask.
#!/bin/bash
(( um = ~$(umask) )) # bitwise negated umask
for((i=0; i<100; ++i)) {
rnd=$RANDOM
(( perm = (rnd % 01000) & um )) # [0000,0777] bitwise AND umask
printf -v octperm "%04o" $perm
file=$i.$rnd.txt # using $i. to make name unique
echo $rnd > $file
chmod $octperm $file
}
If your umask is currently 0022 the above example would not create any files writeable for group and/or others while the other (normal) permissions would be random.
First, you need to echo the random number, not use it as a command.
Second, if you want to use the same random number as the filename and content, you need to save it to a variable. Otherwise you'll get a different number each time you write $RANDOM.
Third, that's not how you do arithmetic and conditions inside [], any shell scripting tutorial should show the correct way. You can also use a bash arithmetic expression with (( expression )).
#!/bin/bash
for i in {1..100}
do
r=$RANDOM
echo "$r" > "$r.txt"
if (( i % 2 == 0 ))
then
chmod +rw "$r.txt"
fi
done
From Ted Lyngmo's answer
With some bashisms, like using integer variables properties and avoiding forks...
declare -i um=" ~$(umask) " i rnd perm
for((i=100;i--;)){
rnd=RANDOM
perm=' ( rnd % 01000 ) & um '
printf -v file file-%03d-%04X.txt $i $rnd
printf -v octperm "%04o" "$perm"
echo "$rnd" > "$file"
chmod "$octperm" "$file"
}
(Filename is built with file number as decimal AND random number in hexadecimal)
About performances
Maybe a little quicker, because of avoiding forks and using integers.
( The for((;;)){ ;} syntax used here is not quicker, just different (shorter)...
In fact, for ((i=100;i--;)) ;do ...;done is (insensibly) slower than for i in {1..100};do ...;done! I just wanted to use this unusual syntax for extreme bashism... ;)
Some comparison:
export TIMEFORMAT=$'(%U + %S) / \e[1m%R\e[0m : %P'
About forks, trying 1'000 variable assignment for formatting, using printf:
time for i in {1..1000};do var=$(printf "foo");done
(0.773 + 0.347) / 1.058 : 105.93
time for i in {1..1000};do printf -v var "foo";done
(0.006 + 0.000) / 0.006 : 99.80
From 1.5 seconds to 6 milliseconds on my host!!! There are no discussion: forks (syntax $(printf...)) is to be avoided!!
About integer properties (using 100'000 binary operations):
declare -i intvar
time for i in {1..100000};do var=$(( ( 431214 % 01000 ) & -19 ));done
(0.272 + 0.005) / 0.278 : 99.46
time for i in {1..100000};do intvar=' ( 431214 % 01000 ) & -19 ';done
(0.209 + 0.000) / 0.209 : 99.87
From 0,28 seconds to 0.21 seconds, this is less significant, but.
About for i in { vs for ((i= (now using 1'000'000 loops):
time for i in {1..1000000};do :;done
(1.600 + 0.000) / 1.602 : 99.86
time for ((i=1000000;i--;));do :;done
(1.880 + 0.001) / 1.882 : 99.95
But this is clearly less significant (care about memory consumtion, using braces).
With awk, you could try following once. This program also take care of closing the open files in backend(in case we get error "too many files opened" once). Written and tested in GNU awk.
awk -v numberOfFiles="100" '
BEGIN{
srand()
for(i=1;i<=numberOfFiles;i++){
out=(int(1 + rand() * 10000))
print out > (out".txt")
system("chmod +rw " out".txt")
close(out)
}
}'
I have created an awk variable named numberOfFiles where I have given 100(as per need to create 100 files), you can keep it as per your need too, in future if you need to change it and we need not to change anything in rest of program here.
I would like to know how can I parse an foreach specifying hours, minutes, seconds and milliseconds. Something like this:
DURATION=00:14:17.82 // (00 hours, 14 minutes, 17 seconds and 82ms)
for ((i=1;i<=DURATION;i++)); do
// output the hour, minute, second and ms here
done
I want to iterate every milisecond, second, minute, hour! Like this:
00:14:17.82
00:14:17.83
00:14:17.84
00:14:17.85
...
00:14:18.01
00:14:18.02
...
01:00:10.15
01:00:10.16
01:00:10.17
...
I just know creating this for seconds and just seconds! But how can I make for hours, minutes, seconds and ms? Thank you so much!
NOTE: The following ideas assume a max DURATION of 23:59:59:99 or 24:00:00.00; upper ranges can certainly be increased but keep in mind the extra overhead (cpu and/or memory) for brace expansions, especially if generating huge ranges that dwarf DURATION (ie, generating large volumes of timestamps that you'll never process).
If you're going to manually enter the value for DURATION then you could manually enter the following brace expansion series:
for i in {00..00}:{00..14}:{00..17}.{00..82}
do
echo $i
done
NOTE: This relatively small set of of brace expansions was almost instantaneous and took up ~6 MB (RAM) on my system.
This generates:
00:00:00.00
00:00:00.01
00:00:00.02
00:00:00.03
00:00:00.04
... snip ...
00:14:17.78
00:14:17.79
00:14:17.80
00:14:17.81
00:14:17.82
Alternatively:
DURATION=00:14:17.82
for i in {00..23}:{00..59}:{00..59}.{00..99}
do
[[ "${i}" > "${DURATION}" ]] && break
# do other stuff
done
NOTE:
on my system this took about about 10 seconds and 1.6 GB (RAM) to expand the series of brace expansions before the loop was processed
this particular set of brace expansions creates 8.64 million timestamps but ...
we break out of the loop after processing ~ 86K timestamps (or ~1% of the timestamps generated by the brace expansions) so ...
we wasted a lot of cpu cycles and RAM generating timestamps that we never got around to using
Another idea is to write a 4-deep set of for loops and break out once we hit 00:14:17.82, eg:
DURATION='00:14:17.82'
for a in {00..23}; do
for b in {00..59}; do
for c in {00..59}; do
for d in {00..99}; do
[[ "${a}:${b}:${c}.${d}" > "${DURATION}" ]] && break 4
# do other stuff here
done
done
done
done
NOTE: While each brace expansion is fast (and uses a miniscule amount of RAM), the nested/repeated expansions is going to eat up a lot of time the longer it takes to get to ${DURATION}; however, unlike the previous ideas this one will at least let you start processing your timestamp values immediately, with intermittent (brief) delays as a new brace expansion is performed.
Of course we could just skip the brace expansion overhead and use 'normal' for loops, eg:
DURATION='00:14:17.82'
for ((a=0; a<=23; a++)); do
for ((b=0; b<=59; b++)); do
for ((c=0; c<=59; c++)); do
for ((d=0; d<=99; d++)); do
printf -v ts "%02d:%02d:%02d.%02d" ${a} ${b} ${c} ${d}
[[ "${ts}" > "${DURATION}" ]] && break 4
# do other stuff here
done
done
done
done
NOTES:
this eliminates the cpu/time/RAM overhead of the brace expansions
assumes you have a newer version of bash that supports printf -v
And for the masochists:
DURATION='00:14:17.82'
for ((i=0; i<8640000; i++)); do
hr=$(( ( i / 100 / 60 / 60 ) % 24 ))
min=$(( ( i / 100 / 60 ) % 60 ))
sec=$(( ( i / 100 ) % 60 ))
msec=$(( i % 100 ))
printf -v ts "%02d:%02d:%02d.%02d" ${hr} ${min} ${sec} ${msec}
[[ "${ts}" > "${DURATION}" ]] && break
# do other stuff
done
Just iterate over milliseconds and convert to the representation you want upon use. First, we write conversion functions:
millis_to_human() {
local h m s ms
# calculate all those hours, minutes, seconds...
((
ms = $1,
s = ms / 100,
ms %= 100,
m = s / 60,
s %= 60,
h = m / 60,
h %= 60,
1
))
printf "%02d:%02d:%02d.%02d\n" "$h" "$m" "$s" "$ms"
}
human_to_millis() {
# Match with regex
[[ $1 =~ ([0-9]*):([0-9]*):([0-9]*).([0-9]*) ]]
# Using 10# so that the number is always in base 10, even with leading 0.
echo $(( ( (
10#${BASH_REMATCH[1]} * 60 +
10#${BASH_REMATCH[2]} ) * 60 +
10#${BASH_REMATCH[3]} ) * 100 +
10#${BASH_REMATCH[4]} ))
}
Then just:
duration='00:14:17.82'
duration_ms=$(human_to_millis "$duration")
for ((i = 1; i <= duration_ms; ++i)); do
millis_to_human "$i"
done
I'm trying to write a bash script that calculates the average of numbers by rows and columns. An example of a text file that I'm reading in is:
1 2 3 4 5
4 6 7 8 0
There is an unknown number of rows and unknown number of columns. Currently, I'm just trying to sum each row with a while loop. The desired output is:
1 2 3 4 5 Sum = 15
4 6 7 8 0 Sum = 25
And so on and so forth with each row. Currently this is the code I have:
while read i
do
echo "num: $i"
(( sum=$sum+$i ))
echo "sum: $sum"
done < $2
To call the program it's stats -r test_file. "-r" indicates rows--I haven't started columns quite yet. My current code actually just takes the first number of each column and adds them together and then the rest of the numbers error out as a syntax error. It says the error comes from like 16, which is the (( sum=$sum+$i )) line but I honestly can't figure out what the problem is. I should tell you I'm extremely new to bash scripting and I have googled and searched high and low for the answer for this and can't find it. Any help is greatly appreciated.
You are reading the file line by line, and summing line is not an arithmetic operation. Try this:
while read i
do
sum=0
for num in $i
do
sum=$(($sum + $num))
done
echo "$i Sum: $sum"
done < $2
just split each number from every line using for loop. I hope this helps.
Another non bash way (con: OP asked for bash, pro: does not depend on bashisms, works with floats).
awk '{c=0;for(i=1;i<=NF;++i){c+=$i};print $0, "Sum:", c}'
Another way (not a pure bash):
while read line
do
sum=$(sed 's/[ ]\+/+/g' <<< "$line" | bc -q)
echo "$line Sum = $sum"
done < filename
Using the numsum -r util covers the row addition, but the output format needs a little glue, by inefficiently paste-ing a few utils:
paste "$2" \
<(yes "Sum =" | head -$(wc -l < "$2") ) \
<(numsum -r "$2")
Output:
1 2 3 4 5 Sum = 15
4 6 7 8 0 Sum = 25
Note -- to run the above line on a given file foo, first initialize $2 like so:
set -- "" foo
paste "$2" <(yes "Sum =" | head -$(wc -l < "$2") ) <(numsum -r "$2")
I want to run a command (such as ls -lrt) 49 times and every time 20 milliseconds after the previous run. What I have written in my bash file is:
for i in `seq 1 49`;
do
v=6.$((i*20)
sleep $v && ls -lrt
done
But it apparently does not differentiate cases like where i equals to 4 with the one that i equals to 40 as both result in v=6.8. What I need is to wait 6.080 for i=4 and 6.800 for i=40.
You can use printf to format the number:
printf -v v '6.%03d' $((i*20))
-v v specifies that the variable $v should hold the result.
how about v=$(echo "scale=2;6+$i*0.02"|bc)
this will keep increasing if the result was greater than 7, although it won't happen till 49. But personally I think it is better than string concatenation.
This question already has answers here:
Random number from a range in a Bash Script
(19 answers)
Closed 7 years ago.
How would I generate an inclusive random number between 1 to 10 in Bash Shell Script?
Would it be $(RANDOM 1+10)?
$(( ( RANDOM % 10 ) + 1 ))
EDIT. Changed brackets into parenthesis according to the comment.
http://web.archive.org/web/20150206070451/http://islandlinux.org/howto/generate-random-numbers-bash-scripting
Simplest solution would be to use tool which allows you to directly specify ranges, like gnu shuf
shuf -i1-10 -n1
If you want to use $RANDOM, it would be more precise to throw out the last 8 numbers in 0...32767, and just treat it as 0...32759, since taking 0...32767 mod 10 you get the following distribution
0-8 each: 3277
8-9 each: 3276
So, slightly slower but more precise would be
while :; do ran=$RANDOM; ((ran < 32760)) && echo $(((ran%10)+1)) && break; done
To generate random numbers with bash use the $RANDOM internal Bash function. Note that $RANDOM should not be used to generate an encryption key. $RANDOM is generated by using your current process ID (PID) and the current time/date as defined by the number of seconds elapsed since 1970.
echo $RANDOM % 10 + 1 | bc
You can also use /dev/urandom:
grep -m1 -ao '[0-9]' /dev/urandom | sed s/0/10/ | head -n1
To generate in the range: {0,..,9}
r=$(( $RANDOM % 10 )); echo $r
To generate in the range: {40,..,49}
r=$(( $RANDOM % 10 + 40 )); echo $r
Here is example of pseudo-random generator when neither $RANDOM nor /dev/urandom is available
echo $(date +%S) | grep -o .$ | sed s/0/10/