I have used time command to execute program (like "xxx$ time ./a.out"), with output as follows,
real 0m7.250s
usr 0m10.395s
sys 0m0.026s
What I want to get is 0 and 7.250 as in 0m7.250s. I have tried "awk '{print $2}'", but without success; nothing output there.
PS: I have tried put output of time command to a file by using ">", also without success.
Try:
pax$ tm=$((time sleep 1) 2>&1 | awk '/^real/{print $2}') ; echo $tm
0m1.002s
(substituting your own a.out command of course, sleep 1 was just used for an example).
It creates a subshell for the time command and ensures that its standard error is sent to standard output instead (time specifically outputs its information to standard error so that it's kept separate from normal output of the program it's timing).
The awk command the captures the line starting real and outputs the second argument (the time).
The time command sends its output to standard error, so as not to interfere with normal program output. You will want to use 2>&1 to redirect it where it can be captured.
Related
In bash, calling foo would display any output from that command on the stdout.
Calling foo > output would redirect any output from that command to the file specified (in this case 'output').
Is there a way to redirect output to a file and have it display on stdout?
The command you want is named tee:
foo | tee output.file
For example, if you only care about stdout:
ls -a | tee output.file
If you want to include stderr, do:
program [arguments...] 2>&1 | tee outfile
2>&1 redirects channel 2 (stderr/standard error) into channel 1 (stdout/standard output), such that both is written as stdout. It is also directed to the given output file as of the tee command.
Furthermore, if you want to append to the log file, use tee -a as:
program [arguments...] 2>&1 | tee -a outfile
$ program [arguments...] 2>&1 | tee outfile
2>&1 dumps the stderr and stdout streams.
tee outfile takes the stream it gets and writes it to the screen and to the file "outfile".
This is probably what most people are looking for. The likely situation is some program or script is working hard for a long time and producing a lot of output. The user wants to check it periodically for progress, but also wants the output written to a file.
The problem (especially when mixing stdout and stderr streams) is that there is reliance on the streams being flushed by the program. If, for example, all the writes to stdout are not flushed, but all the writes to stderr are flushed, then they'll end up out of chronological order in the output file and on the screen.
It's also bad if the program only outputs 1 or 2 lines every few minutes to report progress. In such a case, if the output was not flushed by the program, the user wouldn't even see any output on the screen for hours, because none of it would get pushed through the pipe for hours.
Update: The program unbuffer, part of the expect package, will solve the buffering problem. This will cause stdout and stderr to write to the screen and file immediately and keep them in sync when being combined and redirected to tee. E.g.:
$ unbuffer program [arguments...] 2>&1 | tee outfile
Another way that works for me is,
<command> |& tee <outputFile>
as shown in gnu bash manual
Example:
ls |& tee files.txt
If ‘|&’ is used, command1’s standard error, in addition to its standard output, is connected to command2’s standard input through the pipe; it is shorthand for 2>&1 |. This implicit redirection of the standard error to the standard output is performed after any redirections specified by the command.
For more information, refer redirection
You can primarily use Zoredache solution, but If you don't want to overwrite the output file you should write tee with -a option as follow :
ls -lR / | tee -a output.file
Something to add ...
The package unbuffer has support issues with some packages under fedora and redhat unix releases.
Setting aside the troubles
Following worked for me
bash myscript.sh 2>&1 | tee output.log
Thank you ScDF & matthew your inputs saved me lot of time..
Using tail -f output should work.
In my case I had the Java process with output logs. The simplest solution to display output logs and redirect them into the file(named logfile here) was:
my_java_process_run_script.sh |& tee logfile
Result was Java process running with output logs displaying and
putting them into the file with name logfile
You can do that for your entire script by using something like that at the beginning of your script :
#!/usr/bin/env bash
test x$1 = x$'\x00' && shift || { set -o pipefail ; ( exec 2>&1 ; $0 $'\x00' "$#" ) | tee mylogfile ; exit $? ; }
# do whaetever you want
This redirect both stderr and stdout outputs to the file called mylogfile and let everything goes to stdout at the same time.
It is used some stupid tricks :
use exec without command to setup redirections,
use tee to duplicates outputs,
restart the script with the wanted redirections,
use a special first parameter (a simple NUL character specified by the $'string' special bash notation) to specify that the script is restarted (no equivalent parameter may be used by your original work),
try to preserve the original exit status when restarting the script using the pipefail option.
Ugly but useful for me in certain situations.
Bonus answer since this use-case brought me here:
In the case where you need to do this as some other user
echo "some output" | sudo -u some_user tee /some/path/some_file
Note that the echo will happen as you and the file write will happen as "some_user" what will NOT work is if you were to run the echo as "some_user" and redirect the output with >> "some_file" because the file redirect will happen as you.
Hint: tee also supports append with the -a flag, if you need to replace a line in a file as another user you could execute sed as the desired user.
< command > |& tee filename # this will create a file "filename" with command status as a content, If a file already exists it will remove existed content and writes the command status.
< command > | tee >> filename # this will append status to the file but it doesn't print the command status on standard_output (screen).
I want to print something by using "echo" on screen and append that echoed data to a file
echo "hi there, Have to print this on screen and append to a file"
tee is perfect for this, but this will also do the job
ls -lr / > output | cat output
I have a script that stores the output of commands, functions, and other scripts in a log file.
I want to avoid capturing user input.
The line that is in charge of storing the output of the commands to a logfile is this one:
$command 2>&1 | tee /dev/tty | ruby -pe 'print Time.now.strftime("[%s] ") if !$stdin.tty?' >> "$tempfile"
If the command is a function or a script that asks for user input and prints out those data, that input is stored in temporary file. I would like to avoid that since I don't want to capture sensible data.
I can't modify the commands, functions that I'm wrapping.
Your command only saves program output, not user input. The problem you're seeing is that the command has chosen to output whatever the user inputs, merging it into its own output that is then obviously logged.
There is no good way around this. Please fix your command.
Anyways. Here's a bad, fragile, hacky way around it:
tempfile=test.txt
command='read -ep Enter_some_input: '
$command 2>&1 |
tee /dev/tty |
python3 -c $'import os\nwhile s:=os.read(0, 1024):\n if len(s) > 3: os.write(1, s)' |
ruby -pe 'print Time.now.strftime("[%s] ") if !$stdin.tty?' >> "$tempfile"
The Python command drops all reads of 3 bytes or less. This aims to remove character by character echo as would happen in the most basic cases of a user typing into readline and similar, while hopefully not removing too much intentional output.
I need to execute a command line tool within a shell script that requires user input. Usually, i would write something like:
echo $input | command
however, $input must be read from the standard error of command which I'm able to get e. g. using:
input=$(command 2>&1 > /dev/null | awk '/some.*regexp$/ {print $2}')
But I don't want to execute the command twice (once for extracting/saving the content of $input and once for feeding command with that input).
So, how can i get around that circular dependency and combine these two things to a single bash call with only one execution of command?
Many thanks in advance!
I was trying to write a script to execute a C program and measure the time for execution using time command in Unix.
The script looks like follows
cc /home/pop/Daa/abc.c
r = `time /home/pop/Daa/./a.out`
echo "recursion" $r >> log.txt
cc /home/pop/Daa/xyz.c
d = `time /home/pop/Daa/./a.out `
echo "dynamic" $d >> log.txt
But after executing the script the log.txt file contains only words recursion and dynamic. The time values seem to be missing. But executing time command on the commandline terminal gave the following output
real 0m0.001s
user 0m0.000s
sys 0m0.000s
How can get this output formatted to contain only the 'real' time of execution and written in the log file?
The output of time goes to stderr. You have to redirect it.
TIMEFORMAT=%R
r=$( { time /home/pop/Daa/./a.out; } 2>&1 )
And you can't have spaces around the equal sign. It is preferable to use $() instead of backticks since it's easier to read, there's no confusion with single quotes and it's easier to nest them.
When you assign values to a variable you shouldn't have spaces around the equals sign. For example:
r=`time /home/pop/Daa/./a.out`
If you only want the "real" time, use grep:
r=`time /home/pop/Daa/./a.out | grep real`
or use %e in the time command:
r=`time -f '%e' /home/pop/Daa/./a.out`
I tried to redirect the output of the time command, but I couldn't:
$time ls > filename
real 0m0.000s
user 0m0.000s
sys 0m0.000s
In the file I can see the output of the ls command, not that of time.
Please explain, why I couldn't and how to do this.
no need to launch sub shell. Use a code block will do as well.
{ time ls; } 2> out.txt
or
{ time ls > /dev/null 2>&1 ; } 2> out.txt
you can redirect the time output using,
(time ls) &> file
Because you need to take (time ls) as a single command so you can use braces.
The command time sends it's output to STDERR (instead of STDOUT). That's because the command executed with time normally (in this case ls) outputs to STDOUT.
If you want to capture the output of time, then type:
(time ls) 2> filename
That captures only the output of time, but the output of ls goes normal to the console. If you want to capture both in one file, type:
(time ls) &> filename
2> redirects STDERR, &> redirects both.
time is shell builtin and I'm not sure if there is way to redirect it. However you can use
/usr/bin/time instead, which definitely accept any output redirections.
The reason why redirection does not seem to work with time is that it's a bash reserved word (not a builtin!) when used in front of a pipeline. bash(1):
If the time reserved word precedes a pipeline, the elapsed as well as
user and system time consumed by its execution are reported when the
pipeline terminates.
So, to redirect output of time, either use curly braces:
{ time ls; } 2> filename
Or call /usr/bin/time:
/usr/bin/time ls 2> filename
If you don't want to mix output from time and the command.
With GNU time, you can use -o file like:
/usr/bin/time -o tim grep -e k /tmp 1>out 2>err
while tim is output of time, out and err are stdout and stderr from grep.
I use the redirection of stdout and stderr method with braces for testing.
The &>>rpt represents this >>rpt 2>&1 but shorter.
The braces will execute a command(s) in the current shell. See: man bash
{ time ls a*; } &>>rpt
Not the canonical use case, but another way to go.
Longer running simple tasks can be launched in a detached "screen" terminal with logged output. You could even give the log a unique name.
Primarily this method is good for something that will take hours and is invoked over SSH with a need to "check up on" from time to time. In preference to backgrounding and disowning.
screen -dmL time -v ./crackpassword
You get the same output a terminal would get, with the caveat that this is asynchronous. Of course it could be a script. The output may need tweaking.