How to handle shell expansions in GNU Make under Ubuntu? - bash

Given this very simple Makefile:
all:
#mkdir -pv test/{a,b}
I get this output on OS X 10.6.8 and CentOS 5.5:
mkdir: created directory `test'
mkdir: created directory `test/a'
mkdir: created directory `test/b'
But on Ubuntu 11.04 I get this:
mkdir: created directory `test'
mkdir: created directory `test/{a,b}'
Running the command mkdir -pv test/{a,b} manually in the shell on all platforms gives the expected result.
The version of GNU Make is the same on all platforms:
GNU Make 3.81
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
This program is built for [PLATFORM]
What's different under Ubuntu and why doesn't the shell expansion work there?

The problem is probably that Make spawns /bin/sh. It is usually a symlink to your system's default shell.
Option 1
You could make sure it points to bash (as this is a bashism). Probably, it is now /bin/dash or /bin/sh, depending on your version of Ubuntu.
Option 2
Easier option:
SHELL=/bin/bash
all:
#echo a{3,4}
#bash -c 'echo a{3,4}'
This prints the same output twice unless you comment-out the SHELL= line
Option 3
If you can't/don't want to modify the make file, you can invoke it like so:
make SHELL=/bin/bash
beware of interactions with sub-makefiles or includes. You might want to look at the make -e option and the make export keyword: http://www.gnu.org/s/hello/manual/make/Variables_002fRecursion.html

Make has a function to do this:
#mkdir -pv $(addprefix test/,a b)
gets expanded to
#mkdir -pv test/a test/b
The docs are very thorough.

It's been a long, long time since I've used Make...
There are several ways to specify a particular shell to use. The default shell for old Make was the original Bourne shell. If you wanted a different shell, you had to set it yourself.
You're using Linux and GNU, so I'll assume that you're using BASH as the default shell. Try this command in your Make:
echo "random = $RANDOM"
If this simply prints random = and doesn't include a random number, your Make is using Bourne shell as its default shell instead of BASH. (Which is weird because I didn't think there was a real Bourne shell in Linux...) To get around this:
You can add in a SHELL macro pointing to the BASH shell.
You can include the shell in your command.
Instead of:
#mkdir -pv test/{a,b}
Put this:
/bin/bash -c #mkdir -pv test/{a,b}
This specifies you want to use BASH and not the standard /bin/sh Bourne shell.
If the echo random = $RANDOM does print a random number, you're using BASH (or at least Kornshell), but the BRACE EXPANSION might not be set. Try using this in your Makefile:
set -o
And make sure braceexpand is on. It could be off when you run Make.

Related

Makefile: writing filtering outputs to separate files [duplicate]

I found a difference of behaviour between GNU Make 4.1 and 3.81 and wonder whether my code is not POSIX compliant which 4 is enforcing more strictly, or whether something else is going on.
I distilled the failure case to this Makefile
.POSIX:
all: test-b
test-a:
cat a.txt b.txt c.txt >results.txt
test-b:
cat {a,b,c}.txt >results.txt
Assuming those files have been created with cat {a,b,c}.txt, target test-a always works, yet test-b works on Make 3.81 but fails on 4.1.
The output for 3.81:
$ make --version
GNU Make 3.81
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
This program built for i386-apple-darwin11.3.0
$ make
cat {a,b,c}.txt >results.txt
$ echo $?
0
The output for 4.1:
$ make --version
GNU Make 4.1
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
$ make
cat {a,b,c}.txt >results.txt
cat: {a,b,c}.txt: No such file or directory
Makefile:9: recipe for target 'test-b' failed
make: *** [test-b] Error 1
$ echo $?
2
It's possible the cat command is actually failing in 3.81 and it just isn't pointing it out, as later versions of GNU Make mention passing the -e flag to the shell when invoking target commands to make it more POSIX compliant, but I can't see how that command could be failing.
I assume the wildcards are handled solely by the shell, so I can't see how invoking the shell via a make target command should be any different.
Which is these behaviours are correct? If wildcards like that don't work in Makefiles, which other wildcards can I assume to work?
test-b still fails in 4.1 even if .POSIX: is removed from the file.
Recipes are sent to the shell. They are not interpreted by make. So your question is really, are curly-brace expansions supported by the shell?
That depends on which shell make uses. They are not supported by POSIX standard sh. They are supported by bash (and many other shells).
Make always invokes /bin/sh, regardless of what shell you personally use, unless you specifically set the make SHELL variable to something else. On some systems, /bin/sh is a symlink to /bin/bash so they are the same thing (bash runs in a "POSIX emulation" mode when invoked as /bin/sh but most bash features are still available). Other systems use different shells, such as dash, as /bin/sh which do not have extra bash features.
So, you can either (a) not have a portable makefile and assume /bin/sh is the same as /bin/bash, (b) set SHELL := /bin/bash in your makefile to force it to use bash always (but fail on systems that don't have bash installed), or (c) write your makefile recipes to use only POSIX sh features so it works regardless of which shell is used for /bin/sh.

Makefile internal command on linux functional but not macos

Within a Makefile on linux, we can excute shell/bash commands with it to move directories or excute another files. However when porting the same Makefile over to macOS, all the commands are not readible (therefore path and execution are broken). Is there a universal command or workflow that can work on both?
Example of Makefile
.ONESHELL:
COMMAND ?= none
GIT_HASH ?= githash
alpine:
#cd images/alpine
#make ${COMMAND} GIT_HASH=${GIT_HASH} ALPINE_VERSION=3.6.5 TAG=3.6
so in a linux box both #CD and #MAKE are executed but not for macOS Catalina. I would like to make it universal so that both system will respect the appropriate command that follows.
Chances are that your MacOS box uses its default GNU make version (3.81). .ONESHELL was introduced with 3.82. Upgrade with Homebrew or MacPort. Anyway, better avoid make in recipes, prefer $(MAKE), and instead of cd; make you can use GNU make's -C option: $(MAKE) -C images/alpine ...

Bash is not recognized?

I am trying to build the open source drivers for the c1000a to fix my router. However, I run into a weird issue:
$ make PROFILE=963268BGW
make version is 3.81
kernel version is 3.13.0-29-generic
shell is /bin/sh. Bash version is
***************************************************
ERROR: /bin/sh does not invoke bash shell
***************************************************
make: *** [prebuild_checks] Error 1
Can anyone explain what this might mean?
The answer is that someone wrote a script or Makefile which depends on /bin/sh being Bash, and went out of their way to write code to detect that this is the case, rather than to write code which finds the shell that is required to run the script, like /bin/bash or /usr/bin/bash.
It is easy to tell GNU Make what shell to use for executing build "recipes" (bodies of rules), namely the SHELL variable. The default is /bin/sh/ but if you put
SHELL = /usr/bin/bash # or whatever else
then GNU Make will use that.
I'd gut the Makefile of the logic which tests /bin/sh and bails out, and set up the SHELL variable to point to Bash.
This error might happen when you are doing compilation in Debian flavor distributions such as Ubuntu. Use the command below to compile:
$> make PROFILE=963268BGW SHELL=/bin/bash

Bash: Is it possible to change command before execution

I want to change the command so that command line flag(options) are placed before command line arguments, as which is done automatically by GNU getopt.
Mac use BSD getopt so that function is lacked. I want to tweak the bash so that upon executing of one command, I run a script that parse the flags and arguments reorder them and execute the reordered command.
In this way, both
ls -lh /tmp
ls /tmp -lh
will work in my Mac's terminal.
You can't safely write a general purpose tool to do the job of reordering arguments on a command line unless you know what the optstring argument to the getopt() function looks like for each command. Consider:
make something -f makefile
cp something -f makefile
In the first command, you have to move both the -f and makefile to the front to canonicalize the command invocation. In the second, you must only move the -f; if you move the following file name too, you rewrite the command and destroy your data.
Of course, you also have to know what the getopt_long() argument strings look like if the command takes long-form --force or --file=makefile style arguments too.
Frankly, you'd do better to use POSIXLY_CORRECT in your environment on Linux and forget about the insidious flexibility it provides, and learn to write your options before your arguments at all times. Your code will work across all Unix-like machines better if you do that.
You could install GNU software in some directory other than /bin and /usr/bin (e.g. /usr/gnu/bin and then ensure that you place /usr/gnu/bin on your PATH ahead of the system directories. There are pre-built systems like fink, too. However, that won't help with tools from Apple that don't have analogues from GNU. And there's a 'danger' that shell scripts you write will not be portable to other Macs that don't have the same setup that you do.

Makefile odd behaviour

I've got this Makefile that is presenting some odd behaviour:
$ javac -d Classes -sourcepath .. -classpath `for x in \`ls Classes/jars/*\`; do echo -n $x:; done` PCA/PCAClassifier.java
compiles the java just fine. But for somereason when I call
make PCA
I get:
Compiling PCAClassifier
javac -d Classes -sourcepath .. -classpath `for x in \`ls Classes/jars/*\`; do echo -n $x:; done` PCA/PCAClassifier.java
javac: invalid flag: Classes/jars/Jama.jar:
Usage: javac <options> <source files>
use -help for a list of possible options
make: * [Classes/RobotSuite/PCA/PCAClassifier.class] Error 2
I am so confused. Anyone have a solution?
Make version info:
GNU Make 3.81
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
This program built for i386-apple-darwin10.0
Background: Working on a group project. I'm running Ubuntu, my partner is running Mac OS X. For whatever reason, This makefile works just fine on my computer, but not on his. Even though the command works in his BASh terminal, somehow Make isn't sending it correctly.
You have given us almost no information to go on. Nonetheless, you are in luck!
On Mac OS X, bash has been built with --enable-strict-posix-default and hence the xpg_echo shell option defaults to being on in POSIX mode. POSIX mode is on when the shell has been invoked as /bin/sh, as it has been when it is invoked by Make, unless you instruct it otherwise by setting the $(SHELL) make variable (which you probably shouldn't).
This is the difference between Linux and Mac OS that is killing your makefile. When xpg_echo is on, the shell's built-in echo treats -n as just another argument to be printed (and hence also prints a newline). So the single classpath argument that you're trying to construct ends up as a bunch of separate arguments (half of which are "-n") and javac gets confused.
(This doesn't happen on the command line, even on Mac OS X, because then the shell has been invoked as /bin/bash, so it's not in POSIX mode and xpg_echo is off.)
So you have a number of options for fixing this:
Use /bin/echo -n in your shell snippet; unlike the built-in one, the real echo command these days mostly always understands -n;
Construct the classpath argument in a less roundabout way than your shell for loop; for example
... -classpath `ls Classes/jars/* | tr '\n' :` ...
or, if you are already assuming GNU Make, using make wildcards and functions instead of a shell snippet;
Have your colleague add shopt -u xpg_echo to an appropriate bash startup file on their machine (however this will just lead to future confusion when your next Mac OS-using colleague comes along and you've all long since forgotten how you fixed this this time).
Finally, a general note about debugging makefiles: when a javac ... command in a makefile recipe is giving incomprehensible error messages, change it to echo javac ... instead. Then you'll be able to see exactly how it's being invoked -- which, as seen here, may not be what you intended.

Resources