get runtime of a program from shell - shell

I have the following script named timed which I got from here.
#!/bin/sh
START=$(date +%s)
STARTnano=$(date +%N)
$# # Command-line arguments, i.e. the program you wish to time
END=$(date +%s)
ENDnano=$(date +%N)
DIFF=$(( (10#$END - 10#$START) * 1000000000 ))
DIFFnano=$(( (10#$ENDnano - 10#$STARTnano) ))
DIFF=$(( 10#$DIFF + 10#$DIFFnano ))
DIFFu=$(( 10#$DIFF / 1000000000 )) # Get correct units
DIFFd=$(( 10#$DIFF - 10#$DIFFu )) # Get figures after decimal place
echo "'$#' took ${DIFFu}.${DIFFd} seconds to finish."
I'm trying to use it to measure the runtime of a program.
It doesn't work. I keep getting an error that it lacks a ')'. Can someone point out what's going wrong?

To get the execution time of a script it will be precise if you use time command.
for example
$time ls
**output of ls**
real 0m0.008s
user 0m0.000s
sys 0m0.008s
There is meaning for what is meant by real, user and sys. Use 'man time' for more information.
you can substitute ls with any command(or script)

Related

Bash: Calculate the time differences in hours from input like "YYYYMMDDHH"

I have two dates in forms like: YYYYMMDDHH and want to calculate the differences (in hours) between these two dates. For example
start_date=1996010100
end_date=1996010122
which stands for two dates: 1996-01-01 00:00:00 and 1996-01-01 22:00:00. I want to use date to calculate the difference in hours, the result shall be 22 hours. I tried with
START=$(date -d "$start_date" +"%s")
END=$(date -d "$end_date" +"%s")
HOURS=$(bc -l <<< "($END - $START) / 3600")
but it failed...
So how can I do this? Thanks!
For performance reasons we want to limit the number of sub-process calls we need to invoke:
use bash substring functionality to convert inputs into usable date/time strings
use bash math to replace bc call
bash substring functionality to break the inputs into a usable date/time format, eg:
# convert to usable date/time format:
$ start_date=1996010100
$ echo "${start_date:0:4}-${start_date:4:2}-${start_date:6:2} ${start_date:8:2}:00:00"
1996-01-01 00:00:00
# convert to epoch/seconds:
$ start=$(date -d "${start_date:0:4}-${start_date:4:2}-${start_date:6:2} ${start_date:8:2}:00:00" +"%s")
$ echo $start
820476000
Applying to ${end_date} and using bash math:
$ end_date=1996010122
$ end=$(date -d "${end_date:0:4}-${end_date:4:2}-${end_date:6:2} ${end_date:8:2}:00:00" +"%s")
$ echo $end
820555200
$ hours=$(( (end - start) / 3600))
$ echo $hours
22
This leaves us with 2 sub-process calls ($(date ...)). While other languages/tools (awk, perl, etc) can likely speed this up a bit, if you need to store the result in a bash variable then you're looking at needing at least 1 sub-process call (ie, hours=$(awk/perl/??? ...)).
If performance is really important (eg, needing to perform this 1000's of times) take a look at this SO answer that uses a fifo, background date process and io redirection ... yeah, a bit more coding and a bit more convoluted but also a bit faster for large volumes of operations.
busybox date can do the trick
start_date=1996010100
end_date=1996010122
START=$(busybox date -D "%Y%m%d%H" -d "$start_date" +"%s")
END=$(busybox date -D "%Y%m%d%H" -d "$end_date" +"%s")
HOURS=$(bc -l <<< "scale=0;($END - $START) / 3600")
echo $HOURS
If it's possible for you to use a more fully-featured scripting language like Python, it'll provide a much more pleasant and understandable date parsing experience, and is probably installed by default (datetime is also a standard Python library)
Structured with shell vars
start_date=1996010100
end_date=1996010122
python -c "import datetime ; td = datetime.datetime.strptime('${end_date}', '%Y%m%d%H') - datetime.datetime.strptime('${start_date}', '%Y%m%d%H') ; print(int(td.total_seconds() / 3600))"
Structured to read dates and format code from stdin
echo '%Y%m%d%H' 1996010100 1996010122 | python -c "import datetime,sys ; fmt, date_start, date_end = sys.stdin.read().strip().split() ; td = datetime.datetime.strptime(date_end, fmt) - datetime.datetime.strptime(date_start, fmt) ; print(int(td.total_seconds() / 3600))"
Should work with both Python 3 and Python 2.7
format codes available here (1989 C standard)
https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes
which stands for two dates: 1996-01-01 00:00:00
So convert it to that form if it stands for it.
start_date=1996010100
start_date=$(sed -E 's/(....)(..)(..)(..)/\1-\2-\3 \4:00:00/' <<<"$start_date")
start=$(date -d "$start_date" +"%s")
and the same with end.
the most simple way is to install "dateutils" using this command
sudo apt-get install dateutils
Run these commands to get the difference in seconds:
dateutils.ddiff -i '%Y%m%d%H%M%S' 20200817040001 20210817040101
output:
31536060s
next step: Simply divide by 86400 to get the number of days or similarly for hours and minutes :)

Bash date subtraction +%H:%M:%S

I looked all over online and can't seem to find anything that addresses what I am trying to do. I am using Bash on an Unbuntu VM.
I created the following script
start_code=$(date +%H:%M:%S)
end_code=$(date +%H:%M:%S)
echo $start_code
for i in {1..1000};
do echo $RANDOM >> filename.txt;
done
echo $end_code
The code works fine, but is there any way that I can subtract the variables start_code from end_code??
I have tried this many different ways one being total_code=$(($start_code - $end_code))
but I get errors or nothing returned with everything that I have tried. As I'm brand new to Bash and I'm not even sure if I can do this. Any help would be greatly appreciated.
There are better ways to compute time lapsed. You can do the subtraction with something like:
$ cat a.sh
#!/bin/sh
foo() {
# This function does stuff
sleep ${1-2}
}
start=$(date +%s)
foo 3
end=$(date +%s)
echo "time lapsed: $((end - start))"
time foo 4
$ ./a.sh
time lapsed: 3
real 0m4.006s
user 0m0.001s
sys 0m0.001s
In the above, we first store a time stamp before calling the function foo that just sleeps for 3 seconds. Then we compute a new time stamp and subtract. Then we do the same using the time builtin and let the function sleep for 4 seconds.

Time condition loop in shell

I have just started learning shell script recently, so I don't know much about it.
I am trying to find example of time based while loop but not having any luck.
I want to run a loop for specific amount of time, let's say 1 hour. So loop runs for an hour and then ends automatically.
Edit: This loop will run continiously without any sleep, so the loop condition should be based on loop's start time and current time, not on sleep.
The best way to do this is using the $SECONDS variable, which has a count of the time that the script (or shell) has been running for. The below sample shows how to run a while loop for 3 seconds.
#! /bin/bash
end=$((SECONDS+3))
while [ $SECONDS -lt $end ]; do
# Do what you want.
:
done
Caveat: All solutions in this answer - except the ksh one - can return up to (but not including) 1 second early, since they're based on an integral-seconds counter that advances based on the real-time (system) clock rather than based on when code execution started.
bash, ksh, zsh solution, using special shell variable $SECONDS:
Slightly simplified version of #bsravanin's answer.
Loosely speaking, $SECONDS contains the number of seconds elapsed so far in a script.
In bash and zsh you get integral seconds advancing by the pulse of the system (real-time) clock - i.e., counting behind the scenes does not truly start at 0(!), but at whatever fraction since the last full time-of-day second the script happened to be started at or the SECONDS variable was reset.
By contrast, ksh operates as one would expect: counting truly starts at 0 when you reset $SECONDS; furthermore, $SECONDS reports fractional seconds in ksh.
Therefore, the only shell in which this solution works reasonably predictably and precisely is ksh. That said, for rough measurements and timeouts it may still be usable in bash and zsh.
Note: The following uses a bash shebang line; simply substituting ksh or zsh for bash will make the script run with these shells, too.
#!/usr/bin/env bash
secs=3600 # Set interval (duration) in seconds.
SECONDS=0 # Reset $SECONDS; counting of seconds will (re)start from 0(-ish).
while (( SECONDS < secs )); do # Loop until interval has elapsed.
# ...
done
Solution for POSIX-features-only shells, such as sh (dash) on Ubuntu ($SECONDS is not POSIX-compliant)
Cleaned-up version of #dcpomero's answer.
Uses epoch time returned by date +%s (seconds elapsed since 1 January 1970) and POSIX syntax for the conditional.
Caveat: date +%s itself (specifically, the %s format) is not POSIX-compliant, but it'll work on (at least) Linux, FreeBSD, and OSX.
#!/bin/sh
secs=3600 # Set interval (duration) in seconds.
endTime=$(( $(date +%s) + secs )) # Calculate end time.
while [ $(date +%s) -lt $endTime ]; do # Loop until interval has elapsed.
# ...
done
You can try this
starttime = `date +%s`
while [ $(( $(date +%s) - 3600 )) -lt $starttime ]; do
done
where 'date +%s' gives the current time in seconds.
You can use the loop command, available here, like so:
$ loop './do_thing.sh' --for-duration 1h --every 5s
Which will do the your thing every five seconds for one hour.
date +%s will give you the seconds since the epoch, so something like
startTime = `date +%s`
timeSpan = #some number of seconds
endTime = timeSpan + startTime
while (( `date +%s` < endTime )) ; do
#code
done
You might need some edits, since my bash is rusty
You can explore the -d option of date.
Below is a shell script snippet to exemplify. It is similar to other answers, but may be more useful in different scenarios.
# set -e to exit if the time provided by argument 1 is not valid for date.
# The variable stop_date will store the seconds since 1970-01-01 00:00:00
# UTC, according to the date specified by -d "$1".
set -e
stop_date=$(date -d "$1" "+%s")
set +e
echo -e "Starting at $(date)"
echo -e "Finishing at $(date -d "$1")"
# Repeat the loop while the current date is less than stop_date
while [ $(date "+%s") -lt ${stop_date} ]; do
# your commands that will run until stop_date
done
You can then call the script in the many different ways date understands:
$ ./the_script.sh "1 hour 4 minutes 3 seconds"
Starting at Fri Jun 2 10:50:28 BRT 2017
Finishing at Fri Jun 2 11:54:31 BRT 2017
$ ./the_script.sh "tomorrow 8:00am"
Starting at Fri Jun 2 10:50:39 BRT 2017
Finishing at Sat Jun 3 08:00:00 BRT 2017
$ ./the_script.sh "monday 8:00am"
Starting at Fri Jun 2 10:51:25 BRT 2017
Finishing at Mon Jun 5 08:00:00 BRT 2017
This is exactly what I was looking for,
here is a one line solution based on bsravanin's answer:
end=$((SECONDS+30)); of=$((end-SECONDS)) ; while [ $SECONDS -lt $end ]; do echo $((end-SECONDS)) seconds left of $of ; sleep 1 ; done;
For a more modern approach...
Bash
declare -ir MAX_SECONDS=5
declare -ir TIMEOUT=$SECONDS+$MAX_SECONDS
while (( $SECONDS < $TIMEOUT )); do
# foo
done
Korn
typeset -ir MAX_SECONDS=5
typeset -ir TIMEOUT=$SECONDS+$MAX_SECONDS
while (( $SECONDS < $TIMEOUT )); do
# bar
done

Custom format for time command

I'd like to use the time command in a bash script to calculate the elapsed time of the script and write that to a log file. I only need the real time, not the user and sys. Also need it in a decent format. e.g 00:00:00:00 (not like the standard output). I appreciate any advice.
The expected format supposed to be 00:00:00.0000 (milliseconds) [hours]:[minutes]:[seconds].[milliseconds]
I've already 3 scripts. I saw an example like this:
{ time { # section code goes here } } 2> timing.log
But I only need the real time, not the user and sys. Also need it in a decent format. e.g 00:00:00:00 (not like the standard output).
In other words, I'd like to know how to turn the time output into something easier to process.
You could use the date command to get the current time before and after performing the work to be timed and calculate the difference like this:
#!/bin/bash
# Get time as a UNIX timestamp (seconds elapsed since Jan 1, 1970 0:00 UTC)
T="$(date +%s)"
# Do some work here
sleep 2
T="$(($(date +%s)-T))"
echo "Time in seconds: ${T}"
printf "Pretty format: %02d:%02d:%02d:%02d\n" "$((T/86400))" "$((T/3600%24))" "$((T/60%60))" "$((T%60))""
Notes:
$((...)) can be used for basic arithmetic in bash – caution: do not put spaces before a minus - as this might be interpreted as a command-line option.
See also: http://tldp.org/LDP/abs/html/arithexp.html
EDIT:
Additionally, you may want to take a look at sed to search and extract substrings from the output generated by time.
EDIT:
Example for timing with milliseconds (actually nanoseconds but truncated to milliseconds here). Your version of date has to support the %N format and bash should support large numbers.
# UNIX timestamp concatenated with nanoseconds
T="$(date +%s%N)"
# Do some work here
sleep 2
# Time interval in nanoseconds
T="$(($(date +%s%N)-T))"
# Seconds
S="$((T/1000000000))"
# Milliseconds
M="$((T/1000000))"
echo "Time in nanoseconds: ${T}"
printf "Pretty format: %02d:%02d:%02d:%02d.%03d\n" "$((S/86400))" "$((S/3600%24))" "$((S/60%60))" "$((S%60))" "${M}"
DISCLAIMER:
My original version said
M="$((T%1000000000/1000000))"
but this was edited out because it apparently did not work for some people whereas the new version reportedly did. I did not approve of this because I think that you have to use the remainder only but was outvoted.
Choose whatever fits you.
To use the Bash builtin time rather than /bin/time you can set this variable:
TIMEFORMAT='%3R'
which will output the real time that looks like this:
5.009
or
65.233
The number specifies the precision and can range from 0 to 3 (the default).
You can use:
TIMEFORMAT='%3lR'
to get output that looks like:
3m10.022s
The l (ell) gives a long format.
From the man page for time:
There may be a shell built-in called time, avoid this by specifying /usr/bin/time
You can provide a format string and one of the format options is elapsed time - e.g. %E
/usr/bin/time -f'%E' $CMD
Example:
$ /usr/bin/time -f'%E' ls /tmp/mako/
res.py res.pyc
0:00.01
Use the bash built-in variable SECONDS. Each time you reference the variable it will return the elapsed time since the script invocation.
Example:
echo "Start $SECONDS"
sleep 10
echo "Middle $SECONDS"
sleep 10
echo "End $SECONDS"
Output:
Start 0
Middle 10
End 20
Not quite sure what you are asking, have you tried:
time yourscript | tail -n1 >log
Edit: ok, so you know how to get the times out and you just want to change the format. It would help if you described what format you want, but here are some things to try:
time -p script
This changes the output to one time per line in seconds with decimals. You only want the real time, not the other two so to get the number of seconds use:
time -p script | tail -n 3 | head -n 1
The accepted answer gives me this output
# bash date.sh
Time in seconds: 51
date.sh: line 12: unexpected EOF while looking for matching `"'
date.sh: line 21: syntax error: unexpected end of file
This is how I solved the issue
#!/bin/bash
date1=$(date --date 'now' +%s) #date since epoch in seconds at the start of script
somecommand
date2=$(date --date 'now' +%s) #date since epoch in seconds at the end of script
difference=$(echo "$((date2-$date1))") # difference between two values
date3=$(echo "scale=2 ; $difference/3600" | bc) # difference/3600 = seconds in hours
echo SCRIPT TOOK $date3 HRS TO COMPLETE # 3rd variable for a pretty output.

Print execution time of a shell command

Is is possible to print the execution time of a shell command with following combination?
root#hostname:~# "command to execute" && echo "execution time"
time is a built-in command in most shells that writes execution time information to the tty.
You could also try something like
start_time=`date +%s`
<command-to-execute>
end_time=`date +%s`
echo execution time was `expr $end_time - $start_time` s.
Or in bash:
start_time=`date +%s`
<command-to-execute> && echo run time is $(expr `date +%s` - $start_time) s
Don't forget that there is a difference between bash's builtin time (which should be called by default when you do time command) and /usr/bin/time (which should require you to call it by its full path).
The builtin time always prints to stderr, but /usr/bin/time will allow you to send time's output to a specific file, so you do not interfere with the executed command's stderr stream. Also, /usr/bin/time's format is configurable on the command line or by the environment variable TIME, whereas bash's builtin time format is only configured by the TIMEFORMAT environment variable.
$ time factor 1234567889234567891 # builtin
1234567889234567891: 142662263 8653780357
real 0m3.194s
user 0m1.596s
sys 0m0.004s
$ /usr/bin/time factor 1234567889234567891
1234567889234567891: 142662263 8653780357
1.54user 0.00system 0:02.69elapsed 57%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+215minor)pagefaults 0swaps
$ /usr/bin/time -o timed factor 1234567889234567891 # log to file `timed`
1234567889234567891: 142662263 8653780357
$ cat timed
1.56user 0.02system 0:02.49elapsed 63%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+217minor)pagefaults 0swaps
root#hostname:~# time [command]
It also distinguishes between real time used and system time used.
For a line-by-line delta measurement, try gnonom.
It is a command line utility, a bit like moreutils's ts, to prepend timestamp information to the standard output of another command. Useful for long-running processes where you'd like a historical record of what's taking so long.
Piping anything to gnomon will prepend a timestamp to each line, indicating how long that line was the last line in the buffer--that is, how long it took the next line to appear. By default, gnomon will display the seconds elapsed between each line, but that is configurable.
Adding to #mob's answer:
Appending %N to date +%s gives us nanosecond accuracy:
start=`date +%s%N`;<command>;end=`date +%s%N`;echo `expr $end - $start`
In zsh you can use
=time ...
In bash or zsh you can use
command time ...
These (by different mechanisms) force an external command to be used.
If I'm starting a long-running process like a copy or hash and I want to know later how long it took, I just do this:
$ date; sha1sum reallybigfile.txt; date
Which will result in the following output:
Tue Jun 2 21:16:03 PDT 2015
5089a8e475cc41b2672982f690e5221469390bc0 reallybigfile.txt
Tue Jun 2 21:33:54 PDT 2015
Granted, as implemented here it isn't very precise and doesn't calculate the elapsed time. But it's dirt simple and sometimes all you need.
If you are using zshell, you can have zshell print the time # the start and end of execution. You can accomplish this by adding the following in your ~/.zshrc:
# print time before & after every command
preexec() { eval THEDATE="`date +"[%D_%H:%M:%S] "`"; echo "<CMD STARTED> $THEDATE" }
precmd() { eval THEDATE="`date +"[%D_%H:%M:%S] "`"; echo "<CMD FINISHD> $THEDATE" }
and open a new terminal window to have the changes take effect in all future terminal sessions.
Just ps -o etime= -p "<your_process_pid>"

Resources