I want to compute the time it takes to finish my command in makefile.
Here is what I tried, but it doesn't work:
FILE = some_file.6.txt
.ONESHELL:
another_file.txt: ${FILE}
#START=$$(date +%s.%N)
#END=$$(date +%s.%N)
#echo $$(($$END-$$START))
Here is the error I get:
$ make
/bin/sh: 3: arithmetic expression: expecting EOF: "1569658240.437512688-1569658240.436685866"
makefile:5: recipe for target 'another_file.txt' failed
make: *** [another_file.txt] Error 2
I've tried all combination of adding/removing both ( and $.
Please help, thanks.
This is not a makefile problem. It's a shell problem. This is trivially seen by running the command at a shell prompt rather than a makefile, and you'll get the same error:
$ /bin/sh -c 'echo $((1569658240.437512688-1569658240.436685866))'
/bin/sh: 1: arithmetic expression: expecting EOF: "1569658240.437512688-1569658240.436685866"
It's also an error in bash, so setting SHELL := /bin/bash won't help:
$ /bin/bash -c 'echo $((1569658240.437512688-1569658240.436685866))'
/bin/bash: 1569658240.437512688-1569658240.436685866: syntax error: invalid arithmetic operator (error token is ".437512688-1569658240.436685866")
If you check the documentation for your shell you'll see that arithmetic expressions only work on integer values, not floating point values as you're attempting above.
To perform more advanced math including on floating point values, you should investigate the bc program:
.ONESHELL:
another_file.txt: ${FILE}
#START=$$(date +%s.%N)
#END=$$(date +%s.%N)
#echo $$END-$$START | bc
Related
I'm a beginner in shell and linux in general, it's a Shell Arithmetic question and I can't figure out what to write in terminal to solve these three equations. I'm sorry if it seems a bad question, I tried echo command and expr, but all of them are wrong and with many different errors like, '(' , syntax error near..., E0F, and many other unfortunately. I hope someone will provide me with the correct commands. And I appreciate any help from you. I'll put down the terminal codes that I used which are wrong as I know.
$ x=8
$ y=21
$ echo $((2*x**3 + sqrt(y/2))
bash: unexpected EOF while looking for matching ')'
bash: syntax error: unexpected end of file
$ echo $((2*x**3) + (sqrt(y/2)))
bash: command substitution: line 1: syntax error near unexpected token +'
bash: command substitution: line 1: `(2*x**3) + (sqrt(y/2))'
$ echo $((2*x**3)+(sqrt(y/2))
bash: unexpected EOF while looking for matching )'
bash: syntax error: unexpected end of file
$ echo $((2*x**3)+(sqrt(y/2)))
bash: command substitution: line 1: syntax error near unexpected token +(sqrt(y/2))'
bash: command substitution: line 1: `(2*x**3)+(sqrt(y/2))'
$ echo $((2x**3)+(sqrt(y / 2)))
bash: command substitution: line 1: syntax error near unexpected token +(sqrt(y / 2))'
bash: command substitution: line 1: (2x**3)+(sqrt(y / 2))'
The shell is not the right tool to do floating point computations. It only does integer math and does not provide functions like square root.
However, the bc utility does both. It is an arbitrary-precision decimal arithmetic language and calculator.
$ bc
>>> scale=5
>>> sqrt(21)
4.58257
>>> scale=19
>>> sqrt(21)
4.5825756949558400065
>>> x=8
>>> y=21
>>> x+5
13
>>> x^2
64
>>> 2*x^2 - sqrt(y/2)
124.7596296507960698846
>>> Type Control-D to exit interactive bc.
$
Be sure to read the manual page for bc with man bc to understand all of its capabilities and limitations.
I wanted to determine the version of the Intel Fortran compiler in my makefile, so I added some script using GNU shell function as below for testing,
VERIFORT := $(shell ifort --version)
#VERIFORT := $(shell ifort --version | grep ^ifort) # error occurred too
.PHONY: test
test:
echo $(VERIFORT)
If you copy those code lines shown above, make sure there is a tab before the echo command.
which gives me some errors
/bin/sh: -c: line 0: syntax error near unexpected token `('
When I ran the command ifort --version or ifort --version | grep ^ifort in a terminal, it gave proper result and no error occurred.
My system: 64-bit CentOS 7
Appreciate any correction suggestions.
[EDIT]
Add more output details:
With the grep version of VERIFORT, the make command produced the following result,
echo ifort (IFORT) 18.0.2 20180210
/bin/sh: -c: line 0: syntax error near unexpected token `('
/bin/sh: -c: line 0: `echo ifort (IFORT) 18.0.2 20180210'
make: *** [test] Error 1
[SOLVED]
It turns out to be an echo-usage problem as mentioned by #MadScientist
I think you need to quote the value of the VERIFORT variable when you print it, so that the shell doesn't interpret special characters.
Quoting the VERIFORT variable produced the following result (the grep version)
echo 'ifort (IFORT) 18.0.2 20180210'
ifort (IFORT) 18.0.2 20180210
and no error occurred.
I also tested it by using echo in a terminal
echo ifort (IFORT) 18.0.2 20180210
Which generated the same error
bash: syntax error near unexpected token `('
It seems you didn't show the complete output of the make command. I think before this error message, make printed an echo line (unless the makefile you showed us isn't actually what you invoked, and your actual makefile adds a # before the echo... in which case you should remove it while you debug). If you'd shown us what that output was it would be more clear what the problem is. Also you didn't show what the output of the ifort --version command is when you run it from the command line, but I think it probably contains parentheses.
I think you need to quote the value of the VERIFORT variable when you print it, so that the shell doesn't interpret any special characters:
test:
echo '$(VERIFORT)'
From the docs:
.ONESHELL
If .ONESHELL is mentioned as a target, then when a target is
built all lines of the recipe will be given to a single invocation
of the shell rather than each line being invoked separately (*note
Recipe Execution: Execution.).
So, a makefile, like:
.ONESHELL :
all ::
echo 'foo
bar'
Running, I get:
$ make
echo 'foo
/bin/sh: 1: Syntax error: Unterminated quoted string
makefile:4: recipe for target 'all' failed
make: [all] Error 2 (ignored)
Trying, with almost the same makefile, but adding a - prefix to the recipe, to ignore errors, as documented:
To ignore errors in a recipe line, write a '-' at the beginning of
the line's text (after the initial tab). The '-' is discarded before
the line is passed to the shell for execution.
The makefile, like:
.ONESHELL :
all ::
-echo 'foo
bar'
Running, I get:
$ make
echo 'foo
/bin/sh: 1: Syntax error: Unterminated quoted string
makefile:4: recipe for target 'all' failed
make: [all] Error 2 (ignored)
bar'
/bin/sh: 1: Syntax error: Unterminated quoted string
makefile:4: recipe for target 'all' failed
make: [all] Error 2 (ignored)
Why?
GNU Make 3.81 does not support .ONESHELL, but 3.82 does.
$ /usr/gnu/bin/make --version
GNU Make 3.82
$ /usr/bin/gmake --version
GNU Make 3.81
$ cat gnu.mk
.ONESHELL:
all:
echo 'foo' "$$$$"
echo 'bar' "$$$$"
$ /usr/bin/gmake -f gnu.mk
echo 'foo' "$$"
foo 3100
echo 'bar' "$$"
bar 3101
$ /usr/gnu/bin/make -f gnu.mk
echo 'foo' "$$"
echo 'bar' "$$"
foo 3103
bar 3103
$
I deduce that you're using GNU Make 3.81 from 2006 (or possibly an earlier version, but 3.80 is from 2002); 3.82 is from 2010. The current version is 4.1 from 2014. The online documentation applies to the current version. While lots of the material also applies to older versions, not all of it does.
I also observe that the original makefile with the single single quote on the first line and another on the second seems to cause trouble even with GNU Make 3.82. However, when the lines are each OK, it does seem to work. That's a bit puzzling. And, now I've also installed GNU Make 4.1 (I'd already got it downloaded, but hadn't built it), it too has problems with the bust.mk file below.
$ cat bust.mk
.ONESHELL:
all:
echo 'foo
bar' "$$$$"
$ /usr/gnu/bin/make -f bust.mk
echo 'foo
/bin/sh: -c: line 0: unexpected EOF while looking for matching `''
/bin/sh: -c: line 1: syntax error: unexpected end of file
make: *** [all] Error 2
$
I'm not sure I understand how GNU Make gets confused by that. It might be worth filing a bug for it. OTOH, I'm not sure I'm going to get worried about it, either. But you'll need to take this odd behaviour into account in your future testing.
In a Makefile, I need to cycle through a list and write the current element of the list in a file.
The code is the following:
SHELL := /bin/bash
LIST = A B C
test:
for i in $(LIST) do \
echo $ii > file.txt \
done
I get the following error:
/bin/bash: -c: line 1: syntax error near unexpected token `>'
Do you know how to fix it?
You are missing a ; after $(LIST). You need to terminate the list of words you are giving to for before you can start the do block.
As indicated in the comments, you additionally need a : at the end of the echo line (the command that gets run has no newlines the way this works at the normal command line so you need to explicitly separate the commands from each other).
And further, to get the results you expect, you need to escape the $ in the shell command by using $$i (also pick one of $i or $ii as your variable name).
How long can be a command line that can be passed to sh -c ''? (in bash and in bourne shell)
The limit is much lower than that from the OS (in case of modern Linux).
For example:
$ /bin/true $(seq 1 100000)
$ /bin/sh -c "/bin/true $(seq 1 100000)"
bash: /bin/sh: Argument list too long
And how could I circumvent this problem?
Update
I want to note that getconf can't help here (because that is not a system limit):
$ seq 1 100000 | wc -c
588895
$ getconf ARG_MAX
2097152
Update #2
Now I've understood what is the point here. That is not a shell limit, that is a system limit but for the length of each argument, not for the entire arglist.
$ /bin/true $(seq 1 100000)
$ /bin/true "$(seq 1 100000)"
bash: /bin/true: Argument list too long
Thank you, CodeGnome, for the explanation.
TL;DR
A single argument must be shorter than MAX_ARG_STRLEN.
Analysis
According to this link:
And as additional limit since 2.6.23, one argument must not be longer than MAX_ARG_STRLEN (131072). This might become relevant if you generate a long call like "sh -c 'generated with long arguments'".
This is exactly the "problem" identified by the OP. While the number of arguments allowed may be quite large (see getconf ARG_MAX), when you pass a quoted command to /bin/sh the shell interprets the quoted command as a single string. In the OP's example, it is this single string that exceeds the MAX_ARG_STRLEN limit, not the length of the expanded argument list.
Implementation Specific
Argument limits are implementation specific. However, this Linux Journal article suggests several ways to work around them, including increasing system limits. This may not be directly applicable to the OP, but it nonetheless useful in the general case.
Do Something Else
The OP's issue isn't actually a real problem. The question is imposing an arbitrary constraint that doesn't solve a real-world problem.
You can work around this easily enough by using loops. For example, with Bash 4:
for i in {1..100000}; do /bin/sh -c "/bin/true $i"; done
works just fine. It will certainly be slow, since you're spawning a process on each pass through the loop, but it certainly gets around the command-line limit you're experiencing.
Describe Your Real Problem
If a loop doesn't resolve your issue, please update the question to describe the problem you're actually trying to solve using really long argument lists. Exploring arbitrary line-length limits is an academic exercise, and not on-topic for Stack Overflow.
I don't get that error message. My secret? Single quotes:
/bin/sh -c '/bin/true $(seq 1 100000)'
If I use double quotes, I get that error with every shell:
$ /bin/sh -c "/bin/true $(seq 1 100000)"
-bash: /bin/sh: Argument list too long
$ /bin/bash -c "/bin/true $(seq 1 100000)"
-bash: /bin/bash: Argument list too long
$ /bin/ksh -c "/bin/true $(seq 1 100000)"
-bash: /bin/ksh: Argument list too long
$ /bin/zsh -c "/bin/true $(seq 1 100000)"
-bash: /bin/zsh: Argument list too long
The argument list gets expanded in the current shell when double quotes are used as evidenced by the fact that Bash is the one issuing the error "-bash: ..." regardless of the shell being used to run the command. On my system sh is Dash, by the way.
This holds true even for other "host" shells:
$ dash
$ /bin/bash -c '/bin/true $(seq 1 100000)'
$ /bin/bash -c "/bin/true $(seq 1 100000)"
dash: /bin/bash: Argument list too long
Patient: Doctor, it hurts when I do this."
Doctor: Don't do that.
Make a file with #!/bin/sh as the first line, then the put rest of your command on subsequent lines? :)
More seriously, you can also read commands from STDIN using the -s option, so you can generate your long command line and pipe it in to /bin/sh -s
on my os it's the max length obtained by dichotomy
/bin/sh -c "/bin/true $(perl -e 'print"a"x131061')"
so it gives 131071
but there is no reason to have a line as long ; if it's due to a large number of arguments, "$#" can be used instead; for example
/bin/sh -c 'command "$#"' -- arg1 arg2 .. argn