My Makefile runs a program which, when it finds problem in input files, just dumps the error message to stderr but still returns zero. Here is the Makefile snippet:
target:
prog1 -i input1 input2 -o out.txt
prog2
"prog1" is the program and here is my fix:
target:
prog1 -i input1 input2 -o out.txt 2>&1 1>/dev/null | tee err_log
if [ -s err_log ]; then false ; fi
prog2
Basically I redirect stderr to file err_log and detect if err_log is not empty. I also want the error message to display in console.
These all work, but there is one minor annoyance: the false command, if [ -s err_log ]; then false ; fi shows in console and it is at the end of the console.
I wonder if in the place of false command I could just stop the make but with no message going to console? Basically I want the stderr message at the end.
You can tell make to not print the command it runs by prepending the recipe line with the # character:
target:
prog1 -i input1 input2 -o out.txt 2>&1 1>/dev/null | tee err_log
#if [ -s err_log ]; then false ; fi
prog2
FYI, you can just use:
#[ ! -s err_log ]
to get the same effect.
Use as few auxiliary files as possible. They are a source of additional problems you may need to develop workarounds to.
If you just want to test whether any output is being produced, you can use grep.
You also need to declare your output file to be the target and your input files to be dependencies.
So this should do the trick:
out.txt: input1 input2
prog1 -i $+ -o $# 2>&1 | grep -q .
prog2
However, you should really have a separate rule to run prog2 as it probably depends on out.txt. Something like
out.txt: input1 input2
prog1 -i $+ -o $# 2>&1 | grep -q .
whatever: out.txt
prog2
If prog2 has and output file and input files, make them the target and dependencies as well.
Related
I'm trying to redirect stdin to a FIFO with bash. This way, I will be able to use this stdin on an other part of the script.
However, it doesn't seem to work as I want
script.bash
#!/bin/bash
rm /tmp/in -f
mkfifo /tmp/in
cat >/tmp/in &
# I want to be able to reuse /tmp/in from an other process, for example :
xfce4-terminal --hide-menubar --title myotherterm --fullscreen -x bash -i -c "less /tmp/in"
Here I would expect , when I run ls | ./script.bash, to see the output of ls, but it doesn't work (eg the script exits, without outputing anything)
What am I misunderstanding ?
I am pretty sure that less need additional -f flag when reading from pipe.
test_pipe is not a regular file (use -f to see it)
If that does not help I would also recommend to change order between last two lines of your script:
#!/bin/bash
rm /tmp/in -f
mkfifo /tmp/in
xfce4-terminal --hide-menubar --title myotherterm --fullscreen -x bash -i -c "less -f /tmp/in" &
cat /dev/stdin >/tmp/in
In general, I avoid the use of /dev/stdin, because I get a lot of surprises from what is exactly /dev/stdin, especially when using redirects.
However, what I think that you're seeing is that less finishes before your terminal is completely started. When the less ends, so will the terminal and you won't get any output.
As an example:
xterm -e ls
will also not really display a terminal.
A solution might be tail -f, as in, for example,
#!/bin/bash
rm -f /tmp/in
mkfifo /tmp/in
xterm -e "tail -f /tmp/in" &
while :; do
date > /tmp/in
sleep 1
done
because the tail -f remains alive.
I wrote a program in c++ and now I have a binary. I have also generated a bunch of tests for testing. Now I want to automate the process of testing with bash. I want to save three things in one execution of my binary:
execution time
exit code
output of the program
Right now I am stack up with a script that only tests that binary does its job and returns 0 and doesn't save any information that I mentioned above. My script looks like this
#!/bin/bash
if [ "$#" -ne 2 ]; then
echo "Usage: testScript <binary> <dir_with_tests>"
exit 1
fi
binary="$1"
testsDir="$2"
for test in $(find $testsDir -name '*.txt'); do
testname=$(basename $test)
encodedTmp=$(mktemp /tmp/encoded_$testname)
decodedTmp=$(mktemp /tmp/decoded_$testname)
printf 'testing on %s...\n' "$testname"
if ! "$binary" -c -f $test -o $encodedTmp > /dev/null; then
echo 'encoder failed'
rm "$encodedTmp"
rm "$decodedTmp"
continue
fi
if ! "$binary" -u -f $encodedTmp -o $decodedTmp > /dev/null; then
echo 'decoder failed'
rm "$encodedTmp"
rm "$decodedTmp"
continue
fi
if ! diff "$test" "$decodedTmp" > /dev/null ; then
echo "result differs with input"
else
echo "$testname passed"
fi
rm "$encodedTmp"
rm "$decodedTmp"
done
I want save output of $binary in a variable and not send it into /dev/null. I also want to save time using time bash function
As you asked for the output to be saved in a shell variable, I tried answering this without using output redirection – which saves output in (temporary) text files (which then have to be cleaned).
Saving the command output
You can replace this line
if ! "$binary" -c -f $test -o $encodedTmp > /dev/null; then
with
if ! output=$("$binary" -c -f $test -o $encodedTmp); then
Using command substitution saves the program output of $binary in the shell variable. Command substitution (combined with shell variable assignment) also allows exit codes of programs to be passed up to the calling shell so the conditional if statement will continue to check if $binary executed without error.
You can view the program output by running echo "$output".
Saving the time
Without a more sophisticated form of Inter-Process Communication, there’s no way for a shell that’s a sub-process of another shell to change the variables or the environment of its parent process so the only way that I could save both the time and the program output was to combine them in the one variable:
if ! time-output=$(time "$binary" -c -f $test -o $encodedTmp) 2>&1); then
Since time prints its profiling information to stderr, I use the parentheses operator to run the command in subshell whose stderr can be redirected to stdout. The programming output and the output of time can be viewed by running echo "$time-output" which should return something similar to:
<program output>
<blank line>
real 0m0.041s
user 0m0.000s
sys 0m0.046s
You can get the process status in bash by using $? and print it out by echo $?.
And to catch the output of time, you could use sth like that
{ time sleep 1 ; } 2> time.txt
Or you can save the output of the program and execution time at once
(time ls) > out.file 2>&1
You can save output to a file using output redirection. Just change first /dev/null line:
if ! "$binary" -c -f $test -o $encodedTmp > /dev/null; then
to
if ! "$binary" -c -f $test -o $encodedTmp > prog_output; then
then change second and third /dev/null lines respectively:
if ! "$binary" -u -f $encodedTmp -o $decodedTmp >> prog_output; then
if ! diff "$test" "$decodedTmp" >> prog_output; then
To measure program execution put
start=$(date +%s)
on the first line
then
end=$(date +%s)
echo "Execution time in seconds: " $((end-start)) >> prog_output
on the end.
Specifically, I'm writing a script to make it easier to compile and run my C++ code. It's easy for it to tell if the compilation succeeded of failed, but I also want to add a state where it "compiled with warnings".
$out # to avoid an "ambiguous redirect"
g++ -Wall -Wextra $1 2> out
if [ $? == 0 ]
then
# this is supposed to test the length of the output string
# unless there are errors, $out should be length 0
if [ ${#out} == 0 ]
then
# print "Successful"
else
# print "Completed with Warnings"
fi
else
# print "Failed"
fi
As it is, the failure case check works fine, but $out is always an empty string, though stderr is no longer displaying on the screen, $out is never actually set. If possible, I would also like stderr to still go to the screen.
I hope what I've said makes sense. Cheers.
g++ -Wall -Wextra $1 2> out
This redirects stderr to a file named out, not a variable named $out.
If you want to run gcc and see stdout and stderr on screen as well as save stderr's output, you could use a named pipe (FIFO). It's a bit roundabout, but it'd get the job done.
mkfifo stderr.fifo
gcc -Wall -o /dev/null /tmp/warn.c 2> stderr.fifo &
tee stderr.log < stderr.fifo >&2
rm -f stderr.fifo
wait
After running these commands, the warnings will be available in stderr.log. Taking advantage of the fact that wait will return gcc's exit code, you could then do something like:
if wait; then
if [[ -s stderr.log ]]; then
# print "Completed with Warnings"
else
# print "Successful"
fi
else
# print "Failed"
fi
Annotated:
# Created a named pipe. If one process writes to the pipe, another process can
# read from it to see what was written.
mkfifo stderr.fifo
# Run gcc and redirect its stderr to the pipe. Do it in the background so we can
# read from the pipe in the foreground.
gcc -Wall -o /dev/null /tmp/warn.c 2> stderr.fifo &
# Read from the pipe and write its contents both to the screen (stdout) and to
# the named file (stderr.log).
tee stderr.log < stderr.fifo >&2
# Clean up.
rm -f stderr.fifo
# Wait for gcc to finish and retrieve its exit code. `$?` will be gcc's exit code.
wait
To capture in a variable and display on the screen, use tee:
out=$( g++ -Wall -Wextra "$1" 2>&1 >dev/null | tee /dev/stderr )
This throws out the standard output of g++ and redirects standard error to standard output. That output is piped to tee, which writes it to the named file (/dev/stderr, so that the messages go back to the original standard error) and standard output, which is captured in the variable out.
I am writing a short shell script which calls 'make all'. It's not critical, but is there a way I can suppress the message saying 'nothing to be done for all' if that is the case? I am hoping to find a flag for make which suppresses this (not sure there is one), but an additional line or 2 of code would work too.
FYI I'm using bash.
Edit: to be more clear, I only want to suppess messages that therer is nothing to be done. Otherwise, I want to display the output.
You can make "all" a PHONY target (if it isn't already) which has the real target as a prerequisite, and does something inconspicuous:
.PHONY: all
all: realTarget
#echo > /dev/null
I would like to improve on the previous solution, just to make it a little bit more efficient...:)
.PHONY: all
all: realTarget
#:
#true would also work but is a little slower than #: (I've done some performance tests). In any case both are quite faster than "echo > /dev/null"...
The flag -s silences make: make -s all
EDIT: I originally answered that the flag -q silenced make. It works for me, although the manpage specifies -s, --silent, --quiet as the valid flags.
The grep solution:
{ make all 2>&1 1>&3 | grep -v 'No rule to make target `all' >&2; } 3>&1
The construct 2>&1 1>&3 sends make's stdout to fd 3 and make's stderr to stdout. grep then reads from the previous command's stdout, removes the offending line and sends its stdout to stderr. Finally, fd 3 is returned to stdout.
2022-11-17, A response to #Pryftan's comment:
Ignoring the minor error that I used the wrong text.
Lets create a function that outputs some stuff
make() {
echo "this is stdout"
echo "this is stderr" >&2
printf 'oops, No rule to make target `%s`, not at all' "$1" >&2
}
Testing my solution:
$ { make foobar 2>&1 1>&3 | grep -v 'No rule to make target `all' >&2; } 3>&1
this is stdout
this is stderr
oops, No rule to make target `foobar`, not at all
$ { make all 2>&1 1>&3 | grep -v 'No rule to make target `all' >&2; } 3>&1
this is stdout
this is stderr
Looks good so far.
What about without the braces?
$ make all 2>&1 1>&3 | grep -v 'No rule to make target `all' >&2 3>&1
bash: 3: Bad file descriptor
In this case, we'd need to explicitly create fd 3
$ exec 3>&1; make all 2>&1 1>&3 | grep -v 'No rule to make target `all' >&2 3>&1
this is stdout
this is stderr
What is it about the braces? I think it's delaying evaluation of the contents, and that allows the trailing 3>&1 to be processed first. And that makes the inner 1>&3 valid.
I would like to replace all "no" by "on" in the console output of g++. I tried
$ g++ | sed -e 's/no/on/g'
But it shows
i686-apple-darwin9-g++-4.0.1: no input files
instead of
i686-apple-darwin9-g++-4.0.1: on input files
The message is arriving on the standard error, but the shell pipe operator connects the standard output of one process to the standard input of the next.
To reroute stderr, use
$ g++ 2>&1 | sed -e 's/no/on/g'
or
$ g++ |& sed -e 's/no/on/g'
to get
g++: on input files