I have a wrapper script for compiling command to count md5sum of executed command and time it (also some ither stuff). Point is I have to calculate md5sum inside wrapper script.
problem is that m5sum return same output for gcc main.c and gcc "main.c" but command is different.
Here is simple code.
$ cat sc.sh
#!/usr/bin/env bash
cmd="$#"
cmdh=$(echo "$cmd" | md5sum - | cut -f1 -d" ")
echo "CMD ${cmd}"
echo "MD5 ${cmdh}"
time $#
Here is one output:
$ ./sc.sh gcc -c main.c -o out
CMD gcc -c main.c -o out
MD5 b671a0f3b1235aa91e5f86011449c698
real 0m0.019s
user 0m0.009s
sys 0m0.010s
Here is second. I would like to have diffrent md5sum.
$ ./sc.sh gcc -c "main.c" -o out
CMD gcc -c main.c -o out
MD5 b671a0f3b1235aa91e5f86011449c698
real 0m0.017s
user 0m0.007s
sys 0m0.011s
Like here:
$ echo 'gcc -c "main.c" -o out' | md5sum - | cut -f1 -d" "
94d2bafbec690940d1b908678e9c9b7d
$ echo 'gcc -c main.c -o out' | md5sum - | cut -f1 -d" "
b671a0f3b1235aa91e5f86011449c698
Is such thing possible with bash? It would be awesome to not have it bound to any specific bash version, but if there is no other choice, then its also good.
Removing quotes when parsing the line that you have typed into the terminal is part of how the shell works. The commands you are typing are the same. Research how shell works and re-research a basic introduction to shell quoting.
Like here:
Then pass it "like here". Clearly ' quotes are missing from your commands, but they are present in the "like here" snippet.
$ ./sc.sh gcc -c main.c -o out
is exactly the same as
$ ./sc.sh gcc -c "main.c" -o out
is exactly the same as
$ ./sc.sh 'g''c''c' "-""c" 'main.c' '''''-o' 'ou't
and it happens to work the same way as the following, only because of your IFS and how you are using ="$#". Research what $# does and research IFS:
$ ./sc.sh 'gcc -c main.c -o out'
But the following command is different - the double quotes inside single quotes are preserved.
$ ./sc.sh 'gcc -c "main.c" -o out'
As a follow-up, research word splitting. Remember to check your scripts with https://shellcheck.net
Inside script gcc main.c and gcc "main.c" are the same command.
$0 = gcc and $1 = main.c in both variants.
You cannot see the difference internally, and the script cannot make different signs, so you have no reason to see that.
Related
I am trying to write a portable installation script for building the compiler for my programming language. You can see the script here:
mkdir ArithmeticExpressionCompiler
cd ArithmeticExpressionCompiler
if command -v wget &> /dev/null
then
wget https://flatassembler.github.io/Duktape.zip
else
curl -o Duktape.zip https://flatassembler.github.io/Duktape.zip
fi
unzip Duktape.zip
if command -v gcc &> /dev/null
then
gcc -o aec aec.c duktape.c -lm # The linker that comes with recent versions of Debian Linux insists that "-lm" is put AFTER the source files, or else it outputs some confusing error message.
else
clang -o aec aec.c duktape.c -lm
fi
./aec analogClock.aec
if command -v gcc &> /dev/null
then
gcc -o analogClock analogClock.s -m32
else
clang -o analogClock analogClock.s -m32
fi
./analogClock
However, when I run it on FreeBSD, it complains that wget is not found. But the script checks whether wget exists before calling it. wget is not supposed to be called on FreeBSD. Now, I know FreeBSD uses sh rather than bash, and I suppose my script is not actually POSIX-compliant. So, what am I doing wrong?
From the POSIX Spec:
If a command is terminated by the control operator ( '&'
), the shell shall execute the command asynchronously in a subshell.
This means that the shell shall not wait for the command to finish
before executing the next command.
In posix &> is not supported by posix instead it will see & as a background command indicator causing your command to be run asynchronously with the next part > /dev/null which is seen as a seperate command. This is basically if you were to run:
command -v wget & > /dev/null
Instead you have to redirect another way:
command -v wget >/dev/null 2>&1
I have a command line like this:
myscript constant/tap.txt -n base.dat -c normal/sta0.grs -o normal/brs0.opm
I have 100 .grs files and I need to generate 100 .opm files.
I want to put the command above into a loop that does the following:
myscript constant/tap.txt -n base.dat -c normal/sta0.grs -o normal/brs0.opm
myscript constant/tap.txt -n base.dat -c normal/sta1.grs -o normal/brs1.opm
myscript constant/tap.txt -n base.dat -c normal/sta2.grs -o normal/brs2.opm
myscript constant/tap.txt -n base.dat -c normal/sta3.grs -o normal/brs3.opm
myscript constant/tap.txt -n base.dat -c normal/sta4.grs -o normal/brs4.opm
.
.
.
myscript constant/tap.txt -n base.dat -c normal/sta100.grs -o normal/brs100.opm
I was trying to make it like below:
#!/bin/bash
# Basic until loop
counter=100
until [ $counter -gt 100 ]
do
myscript constant/tap.txt -n base.dat -c normal/sta100.grs -o normal/brs100.opm
done
echo All done
but I could not find a way to set the parameters changes during the loop
In the above command these are constant for each run:
myscript constant/tap.txt -n base.dat -c
The only thing that changes in each loop is the following input and output:
normal/sta100.grs
normal/brs100.opm
I have 100 of sta.grs in the normal folder and I want to create 100 of brs.opm in the normal folder.
#!/bin/bash
counter=0
until ((counter>100))
do
myscript constant/tap.txt -n base.dat -c normal/sta$counter.grs -o normal/brs$counter.opm
((++counter))
done
echo 'All done'
This is an excellent use case for GNU parallel:
find normal -name '*.grs' |
parallel myscript constant/tap.txt -n base.dat -c {} -o {.}.opm
The less code you write, the less errors you make. And this generalizes nicely to cases where your files are named in more complex patterns. And you get parallelization for free (you can get rid of it with -j1).
Instead of incrementing the counter manually, you could use a for loop like this:
for i in {0..100}; do
myscript constant/tap.txt -n base.dat -c normal/sta"$i".grs -o normal/"$i".opm
done
Also, consider that this will sort in an unintuitive way:
1.opm
10.opm
100.opm
11.opm
12.opm
so maybe use padded numbers everywhere with for i in {000..100}; do. This requires Bash 4.0 or newer; if you don't have that, you could do something like
for i in {0..100}; do
printf -v ipad '%03d' "$i"
myscript constant/tap.txt -n base.dat -c normal/sta"$ipad".grs \
-o normal/"$ipad".opm
done
where the printf line puts a padded version of the counter into the ipad variable.
(And if you have Bash older than 3.1, you can't use printf -v and have to do
ipad=$(printf '%03d' "$i")
instead.)
I invoked GNU parallel (on OS X Yosemite, installed using MacPorts, shell is bash 3.2.57) like this:
parallel mycommand -o A=5 -o ::: Y=1 Y=2
with the intent that it would run the following commands, in parallel:
mycommand -o A=5 -o Y=1
mycommand -o A=5 -o Y=2
But it actually runs this:
mycommand -o A=5 -o Y\=1
mycommand -o A=5 -o Y\=2
The backslash causes mycommand not to recognize that argument. This is a problem. And even after scanning the man page and reading the section of the tutorial on quoting, I can't figure out any way to get parallel to run the commands without the backslash getting in there. I've tried putting the Y= options in a file, I've tried single and double quotes with various levels of nesting, but the output of parallel --dry-run always shows Y\=. Is there some way I can get the backslash out?
This should do the trick:
parallel eval mycommand -o A=5 -o ::: Y=1 Y=2
Is there some similar option in dash shell corresponding to pipefail in bash?
Or any other way of getting a non-zero status if one of the commands in pipe fail (but not exiting on it which set -e would).
To make it clearer, here is an example of what I want to achieve:
In a sample debugging makefile, my rule looks like this:
set -o pipefail; gcc -Wall $$f.c -o $$f 2>&1 | tee err; if [ $$? -ne 0 ]; then vim -o $$f.c err; ./$$f; fi;
Basically it runs opens the error file and source file on error and runs the programs when there is no error. Saves me some typing. Above snippet works well on bash but my newer Ubunty system uses dash which doesn't seem to support pipefail option.
I basically want a FAILURE status if the first part of the below group of commands fail:
gcc -Wall $$f.c -o $$f 2>&1 | tee err
so that I can use that for the if statement.
Are there any alternate ways of achieving it?
Thanks!
I ran into this same issue and the bash options of set -o pipefail and ${PIPESTATUS[0]} both failed in the dash shell (/bin/sh) on the docker image I'm using. I'd rather not modify the image or install another package, but the good news is that using a named pipe worked perfectly for me =)
mkfifo named_pipe
tee err < named_pipe &
gcc -Wall $$f.c -o $$f > named_pipe 2>&1
echo $?
See this answer for where I found the info: https://stackoverflow.com/a/1221844/431296
The Q.'s sample problem requires:
I basically want a FAILURE status if the first part of the ... group of commands fail:
Install moreutils, and try the mispipe util, which returns the exit status of the first command in a pipe:
sudo apt install moreutils
Then:
if mispipe "gcc -Wall $$f.c -o $$f 2>&1" "tee err" ; then \
./$$f
else
vim -o $$f.c err
fi
While 'mispipe' does the job here, it is not an exact duplicate of the bash shell's pipefail; from man mispipe:
Note that some shells, notably bash, do offer a
pipefail option, however, that option does not
behave the same since it makes a failure of any
command in the pipeline be returned, not just the
exit status of the first.
I would like to replace all "no" by "on" in the console output of g++. I tried
$ g++ | sed -e 's/no/on/g'
But it shows
i686-apple-darwin9-g++-4.0.1: no input files
instead of
i686-apple-darwin9-g++-4.0.1: on input files
The message is arriving on the standard error, but the shell pipe operator connects the standard output of one process to the standard input of the next.
To reroute stderr, use
$ g++ 2>&1 | sed -e 's/no/on/g'
or
$ g++ |& sed -e 's/no/on/g'
to get
g++: on input files