G++ and sed pipeline - bash

I would like to replace all "no" by "on" in the console output of g++. I tried
$ g++ | sed -e 's/no/on/g'
But it shows
i686-apple-darwin9-g++-4.0.1: no input files
instead of
i686-apple-darwin9-g++-4.0.1: on input files

The message is arriving on the standard error, but the shell pipe operator connects the standard output of one process to the standard input of the next.
To reroute stderr, use
$ g++ 2>&1 | sed -e 's/no/on/g'
or
$ g++ |& sed -e 's/no/on/g'
to get
g++: on input files

Related

TeamCity: Process the internal build log in step

I'd like to grep the build log to count the number of compiler warnings. Is it possible to run a command on the build log from this or previous step?
TeamCity Enterprise 2022.04.4 (build 108763)
You could try to save compiler's output to a file and then grep it. For example
make 2>&1 | tee make.log
grep -o 'WARN' make.log | wc -l
In that case tee prints command output to both stdout and file

How to count md5sum of executed command in bash

I have a wrapper script for compiling command to count md5sum of executed command and time it (also some ither stuff). Point is I have to calculate md5sum inside wrapper script.
problem is that m5sum return same output for gcc main.c and gcc "main.c" but command is different.
Here is simple code.
$ cat sc.sh
#!/usr/bin/env bash
cmd="$#"
cmdh=$(echo "$cmd" | md5sum - | cut -f1 -d" ")
echo "CMD ${cmd}"
echo "MD5 ${cmdh}"
time $#
Here is one output:
$ ./sc.sh gcc -c main.c -o out
CMD gcc -c main.c -o out
MD5 b671a0f3b1235aa91e5f86011449c698
real 0m0.019s
user 0m0.009s
sys 0m0.010s
Here is second. I would like to have diffrent md5sum.
$ ./sc.sh gcc -c "main.c" -o out
CMD gcc -c main.c -o out
MD5 b671a0f3b1235aa91e5f86011449c698
real 0m0.017s
user 0m0.007s
sys 0m0.011s
Like here:
$ echo 'gcc -c "main.c" -o out' | md5sum - | cut -f1 -d" "
94d2bafbec690940d1b908678e9c9b7d
$ echo 'gcc -c main.c -o out' | md5sum - | cut -f1 -d" "
b671a0f3b1235aa91e5f86011449c698
Is such thing possible with bash? It would be awesome to not have it bound to any specific bash version, but if there is no other choice, then its also good.
Removing quotes when parsing the line that you have typed into the terminal is part of how the shell works. The commands you are typing are the same. Research how shell works and re-research a basic introduction to shell quoting.
Like here:
Then pass it "like here". Clearly ' quotes are missing from your commands, but they are present in the "like here" snippet.
$ ./sc.sh gcc -c main.c -o out
is exactly the same as
$ ./sc.sh gcc -c "main.c" -o out
is exactly the same as
$ ./sc.sh 'g''c''c' "-""c" 'main.c' '''''-o' 'ou't
and it happens to work the same way as the following, only because of your IFS and how you are using ="$#". Research what $# does and research IFS:
$ ./sc.sh 'gcc -c main.c -o out'
But the following command is different - the double quotes inside single quotes are preserved.
$ ./sc.sh 'gcc -c "main.c" -o out'
As a follow-up, research word splitting. Remember to check your scripts with https://shellcheck.net
Inside script gcc main.c and gcc "main.c" are the same command.
$0 = gcc and $1 = main.c in both variants.
You cannot see the difference internally, and the script cannot make different signs, so you have no reason to see that.

What is the best to detect stderr output and stop make

My Makefile runs a program which, when it finds problem in input files, just dumps the error message to stderr but still returns zero. Here is the Makefile snippet:
target:
prog1 -i input1 input2 -o out.txt
prog2
"prog1" is the program and here is my fix:
target:
prog1 -i input1 input2 -o out.txt 2>&1 1>/dev/null | tee err_log
if [ -s err_log ]; then false ; fi
prog2
Basically I redirect stderr to file err_log and detect if err_log is not empty. I also want the error message to display in console.
These all work, but there is one minor annoyance: the false command, if [ -s err_log ]; then false ; fi shows in console and it is at the end of the console.
I wonder if in the place of false command I could just stop the make but with no message going to console? Basically I want the stderr message at the end.
You can tell make to not print the command it runs by prepending the recipe line with the # character:
target:
prog1 -i input1 input2 -o out.txt 2>&1 1>/dev/null | tee err_log
#if [ -s err_log ]; then false ; fi
prog2
FYI, you can just use:
#[ ! -s err_log ]
to get the same effect.
Use as few auxiliary files as possible. They are a source of additional problems you may need to develop workarounds to.
If you just want to test whether any output is being produced, you can use grep.
You also need to declare your output file to be the target and your input files to be dependencies.
So this should do the trick:
out.txt: input1 input2
prog1 -i $+ -o $# 2>&1 | grep -q .
prog2
However, you should really have a separate rule to run prog2 as it probably depends on out.txt. Something like
out.txt: input1 input2
prog1 -i $+ -o $# 2>&1 | grep -q .
whatever: out.txt
prog2
If prog2 has and output file and input files, make them the target and dependencies as well.

Supress grep output but capture it in a variable

I'm trying to get the following line to work
WERRORS=`echo $VALPG | grep -q -s -o -m 1 '\<[0-9]* Errors'`
What I want is that the result of grep go into WERRORS variable but not echo in the terminal.
So i use -q, but then WERRORS is empty
If grep sends any error messages, they go to the error output, which is not captured by the backticks. If you need this output in a variable (which is somewhat problematic, because it's often localized), redirect it using 2>&1:
WERRORS=`echo $VALPG | grep -s -o -m 1 '\<[0-9]* Errors' 2>&1`
WERRORS=`echo $VALPG | grep -s -o -m 1 '\<[0-9]* Errors'`
kent$ val=abcpc
kent$ a=$(echo $val|grep -o -m 1 -s 'pc')
kent$ echo $a
pc

piping output from programs that do not output to STDOUT

Say I have a program that outputs to a file which is provided by an (required) argument:
./foo -o bar.txt
Is there any way to trick it into outputting to a pipe? This just outputs to a file named -.
./foo -o - | less
You can use /dev/stdout as the file name:
./foo -o /dev/stdout | less
This will feed the contents of bar.txt to filter as the file grows:
tail -f bar.txt | filter
other solution, create a named pipe:
mkfifo /tmp/myfifo; ./foo -o /tmp/myfifo
</tmp/myfifo cat -

Resources