I've got this Makefile that is presenting some odd behaviour:
$ javac -d Classes -sourcepath .. -classpath `for x in \`ls Classes/jars/*\`; do echo -n $x:; done` PCA/PCAClassifier.java
compiles the java just fine. But for somereason when I call
make PCA
I get:
Compiling PCAClassifier
javac -d Classes -sourcepath .. -classpath `for x in \`ls Classes/jars/*\`; do echo -n $x:; done` PCA/PCAClassifier.java
javac: invalid flag: Classes/jars/Jama.jar:
Usage: javac <options> <source files>
use -help for a list of possible options
make: * [Classes/RobotSuite/PCA/PCAClassifier.class] Error 2
I am so confused. Anyone have a solution?
Make version info:
GNU Make 3.81
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
This program built for i386-apple-darwin10.0
Background: Working on a group project. I'm running Ubuntu, my partner is running Mac OS X. For whatever reason, This makefile works just fine on my computer, but not on his. Even though the command works in his BASh terminal, somehow Make isn't sending it correctly.
You have given us almost no information to go on. Nonetheless, you are in luck!
On Mac OS X, bash has been built with --enable-strict-posix-default and hence the xpg_echo shell option defaults to being on in POSIX mode. POSIX mode is on when the shell has been invoked as /bin/sh, as it has been when it is invoked by Make, unless you instruct it otherwise by setting the $(SHELL) make variable (which you probably shouldn't).
This is the difference between Linux and Mac OS that is killing your makefile. When xpg_echo is on, the shell's built-in echo treats -n as just another argument to be printed (and hence also prints a newline). So the single classpath argument that you're trying to construct ends up as a bunch of separate arguments (half of which are "-n") and javac gets confused.
(This doesn't happen on the command line, even on Mac OS X, because then the shell has been invoked as /bin/bash, so it's not in POSIX mode and xpg_echo is off.)
So you have a number of options for fixing this:
Use /bin/echo -n in your shell snippet; unlike the built-in one, the real echo command these days mostly always understands -n;
Construct the classpath argument in a less roundabout way than your shell for loop; for example
... -classpath `ls Classes/jars/* | tr '\n' :` ...
or, if you are already assuming GNU Make, using make wildcards and functions instead of a shell snippet;
Have your colleague add shopt -u xpg_echo to an appropriate bash startup file on their machine (however this will just lead to future confusion when your next Mac OS-using colleague comes along and you've all long since forgotten how you fixed this this time).
Finally, a general note about debugging makefiles: when a javac ... command in a makefile recipe is giving incomprehensible error messages, change it to echo javac ... instead. Then you'll be able to see exactly how it's being invoked -- which, as seen here, may not be what you intended.
Related
According to gcc manual, the -E option only preprocesses the .c source file, without running the compiler and just giving an input file (.i). But what does the -E stand for?
Summary
This option was introduced on compilers much earlier than GCC, and GCC used the same naming for compatibility.
My best guess from the historical evidence is that -E stands for "expand macros".
The authors of those earlier compilers couldn't call it -P because there was already a -P option, which also ran only the preprocessor, but wrote the output to .i files instead of to standard output. -p was also taken.
Over time, -E became preferred to -P, and when GCC was written, it supported only -E and not -P.
Despite being supposedly off topic for Stack Overflow, I think a bit of history will help explain how this option got its name.
gcc(1) inherited its basic command line options from much earlier Unix C compilers. Many versions can be found in the Unix Tree archive.
It looks like the first version that supported -E was Research Unix V7, circa 1979: cc(1) source, man page source. There was also a -P option that also just ran the preprocessor, but sent the result to a file foo.i instead of to standard output. V6 had already supported -P but not -E: cc(1) source, man page source.
This at least answers why -E wasn't named -P instead: because -P was already in use. (And -p was also taken, it was used to request profiling.) The only hint I found as to why -E was chosen is that the corresponding flag variable in the source code is named exflag. I would hazard a guess that this stands for "expand", as what -E does is basically to expand macros.
It appears that -P was eventually deprecated in favor of -E. V8 still supported it, but omitted it from the man page. V10 (circa 1989) included two versions of the compiler, cc which compiled traditional C, and lcc for ANSI C. The man page says that cc supports -P with the preprocessor behavior, but for lcc, -P did something else ("write declarations for all defined globals on standard error"). They both supported -E. On other fronts, 32V and BSD, at least initially, supported both, but -P would emit a warning that it was obsolete and -E should be used instead.
It looks like gcc, from its earliest version, only supported -E and not -P. Since then, a -P option has been introduced, but it does something else ("inhibit generation of linemarkers").
I have a Makefile with a target that prepends an environment variable to the shell call (with the usual bash syntax). This is the gist of it:
mytest:
ANSWER=42 echo Hello!
(the real deal is a programme that does something with the ENV ANSWER, but that's irrelevant here)
This works as expected in a linux/bash environment. In windows/cmd.exe environments it works to my surprise in some machines, but fails in others with this error:
> make mytest
MYVAR=42 echo Hello!
'MYVAR' is not recognized as an internal or external command,
operable program or batch file.
make: *** [Makefile:332: mytest] Error 1
Which is what I'd normally expect, since ENVVAR=<value> <command> isn't valid syntax in the windows shell. Apparently Make does some magic that I don't understand.
If I pre-export the ENV this way, it works as expected:
mytest: export ANSWER:=42
mytest:
echo Hello!
But since it works on some windows environments, I'd like to know why and maybe adapt those instead of changing a lot of Makefiles.
All of the environments are using GNU Make version 4.3.
Running on Windows is complicated because GNU make can be built in different ways there. Sometimes it is built to use Windows cmd.exe as its shell. Sometimes it's built to use an installation of sh.exe as its shell (note, GNU make never comes with a shell: the shell is a separate facility provided on the system). And sometimes it's built to use sh.exe if it can find one, else use cmd.exe.
If you're seeing different behaviors on different systems, then the way make was built is different between those systems, and/or different systems have different extra software installed so that some have sh.exe and some don't.
I get /bin/sh: -c: line 1: syntax error: unexpected end of file for the if statement in the makefile below — what's wrong? (My prompt is /tmp >.)
/tmp > make test
set -e
if [[ -f /tmp/device ]] ; then
/bin/sh: -c: line 1: syntax error: unexpected end of file
make: *** [test] Error 2
/tmp > cat Makefile
.ONESHELL:
test:
set -e
if [[ -f /tmp/device ]] ; then
echo 'Do something'
fi
I am using GNU Make 3.81.
Using GNU Make 3.81, the .ONESHELL feature is demonstrably not supported — you showed that, and I showed that. I tested on Mac OS X 10.11.5, with the supplied /usr/bin/make as GNU Make 3.81. I demonstrated to my satisfaction by adding strategically placed echo PID=$$$$ lines before set -e, after it, and so on, and observed different PID values. GNU Make version 3.81 is from 2006 (there was also version 3.82 from 2010, before version 4.x was released, starting with 4.0 in 2013).
The current version of GNU Make is 4.2.1 (June 2016). That version does support the feature; your script works as expected when using a sufficiently recent version of GNU Make. It is a feature that has been around for a while — you probably don't have to upgrade to the latest to get the support, but why would you go with a down-version if you're going to upgrade anyway.
If you wish to use the .ONSHELL: feature, you'll have to ensure you are using a new enough version of GNU Make (newer than version 3.81). If that's not feasible, don't use the feature.
Reading the NEWS file from 4.2.1, it is clear that ONESHELL was added to GNU Make version 3.82:
Version 3.82 (28 Jul 2010)
…
* New special target: .ONESHELL instructs make to invoke a single instance
of the shell and provide it with the entire recipe, regardless of how many
lines it contains. As a special feature to allow more straightforward
conversion of makefiles to use .ONESHELL, any recipe line control
characters ('#', '+', or '-') will be removed from the second and
subsequent recipe lines. This happens _only_ if the SHELL value is deemed
to be a standard POSIX-style shell. If not, then no interior line control
characters are removed (as they may be part of the scripting language used
with the alternate SHELL).
I want to change the command so that command line flag(options) are placed before command line arguments, as which is done automatically by GNU getopt.
Mac use BSD getopt so that function is lacked. I want to tweak the bash so that upon executing of one command, I run a script that parse the flags and arguments reorder them and execute the reordered command.
In this way, both
ls -lh /tmp
ls /tmp -lh
will work in my Mac's terminal.
You can't safely write a general purpose tool to do the job of reordering arguments on a command line unless you know what the optstring argument to the getopt() function looks like for each command. Consider:
make something -f makefile
cp something -f makefile
In the first command, you have to move both the -f and makefile to the front to canonicalize the command invocation. In the second, you must only move the -f; if you move the following file name too, you rewrite the command and destroy your data.
Of course, you also have to know what the getopt_long() argument strings look like if the command takes long-form --force or --file=makefile style arguments too.
Frankly, you'd do better to use POSIXLY_CORRECT in your environment on Linux and forget about the insidious flexibility it provides, and learn to write your options before your arguments at all times. Your code will work across all Unix-like machines better if you do that.
You could install GNU software in some directory other than /bin and /usr/bin (e.g. /usr/gnu/bin and then ensure that you place /usr/gnu/bin on your PATH ahead of the system directories. There are pre-built systems like fink, too. However, that won't help with tools from Apple that don't have analogues from GNU. And there's a 'danger' that shell scripts you write will not be portable to other Macs that don't have the same setup that you do.
Given this very simple Makefile:
all:
#mkdir -pv test/{a,b}
I get this output on OS X 10.6.8 and CentOS 5.5:
mkdir: created directory `test'
mkdir: created directory `test/a'
mkdir: created directory `test/b'
But on Ubuntu 11.04 I get this:
mkdir: created directory `test'
mkdir: created directory `test/{a,b}'
Running the command mkdir -pv test/{a,b} manually in the shell on all platforms gives the expected result.
The version of GNU Make is the same on all platforms:
GNU Make 3.81
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
This program is built for [PLATFORM]
What's different under Ubuntu and why doesn't the shell expansion work there?
The problem is probably that Make spawns /bin/sh. It is usually a symlink to your system's default shell.
Option 1
You could make sure it points to bash (as this is a bashism). Probably, it is now /bin/dash or /bin/sh, depending on your version of Ubuntu.
Option 2
Easier option:
SHELL=/bin/bash
all:
#echo a{3,4}
#bash -c 'echo a{3,4}'
This prints the same output twice unless you comment-out the SHELL= line
Option 3
If you can't/don't want to modify the make file, you can invoke it like so:
make SHELL=/bin/bash
beware of interactions with sub-makefiles or includes. You might want to look at the make -e option and the make export keyword: http://www.gnu.org/s/hello/manual/make/Variables_002fRecursion.html
Make has a function to do this:
#mkdir -pv $(addprefix test/,a b)
gets expanded to
#mkdir -pv test/a test/b
The docs are very thorough.
It's been a long, long time since I've used Make...
There are several ways to specify a particular shell to use. The default shell for old Make was the original Bourne shell. If you wanted a different shell, you had to set it yourself.
You're using Linux and GNU, so I'll assume that you're using BASH as the default shell. Try this command in your Make:
echo "random = $RANDOM"
If this simply prints random = and doesn't include a random number, your Make is using Bourne shell as its default shell instead of BASH. (Which is weird because I didn't think there was a real Bourne shell in Linux...) To get around this:
You can add in a SHELL macro pointing to the BASH shell.
You can include the shell in your command.
Instead of:
#mkdir -pv test/{a,b}
Put this:
/bin/bash -c #mkdir -pv test/{a,b}
This specifies you want to use BASH and not the standard /bin/sh Bourne shell.
If the echo random = $RANDOM does print a random number, you're using BASH (or at least Kornshell), but the BRACE EXPANSION might not be set. Try using this in your Makefile:
set -o
And make sure braceexpand is on. It could be off when you run Make.