What's the difference between #! /usr/bin/env ruby and #! ruby?
(I found many other questions discussing the difference between #! /usr/bin/env ruby and #! /usr/bin/ruby, but that's not my question.)
#! ruby
...is not guaranteed to work on UNIXlike systems (and does not work on any I personally know of) at all; a valid shebang must have a fully qualified path. It may suffice to tell your editor which programming language you're using, but that doesn't mean that the kernel will successfully use it to select an interpreter with which to run a program.
The kernel's execve syscall doesn't do PATH lookups -- that's added by C-standard-library wrappers such as execlp and execvp, but parsing shebangs is done directly by the kernel, so your C-library nicities don't happen there.
#!/usr/bin/env ruby
...uses the PATH to look up the location of the ruby executable. Because the path to the env executable is fully specified, this is a valid shebang line (which #! ruby is not).
env has other purposes as well -- you can run, for instance, env -i someprog to run someprog with a completely empty environment, or env FOO=bar someprog to run someprog with the environment variable FOO set to the value bar (which FOO=bar someprog would also do if running through a shell, but the env approach also works with no shell involved).
However, the relevant use case in this context is forcing a PATH lookup.
Related
Or, why doesn't this work, assuming you correctly set the file as executable? (On my system it just hangs, but can be killed with ^C.)
#!/usr/bin/env TEST=TEST python
print('hello')
While this does:
#!/usr/bin/env python
print('hello')
As does this:
[fred#pc build]$ /usr/bin/env TEST=TEST python hello.py
hello
Since version 8.30 env has the option -S, which make things
work also in linux:
Use
#!/usr/bin/env -S TEST=TEST python
On some systems, including Linux, the effect of the
#!/usr/bin/env TEST=TEST python
line in 'hello.py' if you run it with './hello.py' is the same as running
/usr/bin/env 'TEST=TEST python' ./hello.py
Notice that 'TEST=TEST python' is all one argument. This causes 'env' to set the TEST environment variable to 'TEST python' and then run ./hello.py with exec. Then the shebang line is processed again, and the process repeats recursively. The overall effect is to have env execed over and over again until the process is interrupted. If you run top on the system while the execing is going on you'll see a very busy process.
See the Shebang (Unix) Wikipedia article for information about how the #! mechanism works. It includes some details of differences between how it is handled on different systems.
It's not possible to set environment variables on the shebang line on Linux, and many other systems. You'll need to either set them in the program itself or, if that is not appropriate, use a wrapper program (e.g. a trivial shell program) to run the Python program.
I typically put a shebang for bash at the top of my shell scripts, e.g.:
#!/usr/bin/bash
However I see many other variants of this, like #!/bin/bash or #!/usr/local/bin/bash etc.
It seems to me these different conventions would result in compatibility or portability issues. If my bash is at another location than someone else's, my script won't work on their machine and vice versa.
If a shell interpreter like bash is apparently not always at the same location, isn't it plain WRONG to explicitly use a hardcoded path in a script?
I understood you can use a somewhat more flexible or less system-dependent approach like this:
#!/usr/bin/env bash
Which results in the (or a?) local version of bash, wherever that may be installed.
Does the latter variant always work? Or is there a better approach that has the highest chance of referring to any system's bash regardless of where it's installed?
I would recommend either "#!/bin/bash" or "#!/usr/bin/bash". On a modern Linux distro, bash should be installed in both places.
Apparently, that isn't true for OpenBSD ... which uses ksh as the default shell. But on an OpenBSD system, you are liable to find that bash isn't installed at all. It is apparently an optional package, and the admin may have not installed it.
So, if you want to maximize portability, use "/bin/sh" and restrict yourself to standard POSIX shell syntax and commands. "/bin/sh" is typically a link to bash or ksh, and runs in POSIX compliant mode.
Other variations:
"#!/usr/local/bin/bash" typically won't work on Linux. If it does, it may give you a locally built / modified version of bash.
"#!/usr/bin/env bash" should work, with a couple of caveats:
This will give you whatever version of bash is first on the user's command search path (i.e. $PATH).
It is conceivable that the path to env may be different, or that it may not exist. (The env command wasn't in the first version of the POSIX specs.)
I've seen in a number of places, including recommendations on this site (What is the preferred Bash shebang?), to use #!/usr/bin/env bash in preference to #!/bin/bash. I've even seen one enterprising individual suggest using #!/bin/bash was wrong and bash functionality would be lost by doing so.
All that said, I use bash in a tightly controlled test environment where every drive in circulation is essentially a clone of a single master drive. I understand the portability argument, though it is not necessarily applicable in my case. Is there any other reason to prefer #!/usr/bin/env bashover the alternatives and, assuming portability was a concern, is there any reason using it could break functionality?
#!/usr/bin/env searches PATH for bash, and bash is not always in /bin, particularly on non-Linux systems. For example, on my OpenBSD system, it's in /usr/local/bin, since it was installed as an optional package.
If you are absolutely sure bash is in /bin and will always be, there's no harm in putting it directly in your shebang—but I'd recommend against it because scripts and programs all have lives beyond what we initially believe they will have.
The standard location of bash is /bin, and I suspect that's true on all systems. However, what if you don't like that version of bash? For example, I want to use bash 4.2, but the bash on my Mac is at 3.2.5.
I could try reinstalling bash in /bin but that may be a bad idea. If I update my OS, it will be overwritten.
However, I could install bash in /usr/local/bin/bash, and setup my PATH to:
PATH="/usr/local/bin:/bin:/usr/bin:$HOME/bin"
Now, if I specify bash, I don't get the old cruddy one at /bin/bash, but the newer, shinier one at /usr/local/bin. Nice!
Except my shell scripts have that !# /bin/bash shebang. Thus, when I run my shell scripts, I get that old and lousy version of bash that doesn't even have associative arrays.
Using /usr/bin/env bash will use the version of bash found in my PATH. If I setup my PATH, so that /usr/local/bin/bash is executed, that's the bash that my scripts will use.
It's rare to see this with bash, but it is a lot more common with Perl and Python:
Certain Unix/Linux releases which focus on stability are sometimes way behind with the release of these two scripting languages. Not long ago, RHEL's Perl was at 5.8.8 -- an eight year old version of Perl! If someone wanted to use more modern features, you had to install your own version.
Programs like Perlbrew and Pythonbrew allow you to install multiple versions of these languages. They depend upon scripts that manipulate your PATH to get the version you want. Hard coding the path means I can't run my script under brew.
It wasn't that long ago (okay, it was long ago) that Perl and Python were not standard packages included in most Unix systems. That meant you didn't know where these two programs were installed. Was it under /bin? /usr/bin? /opt/bin? Who knows? Using #! /usr/bin/env perl meant I didn't have to know.
And Now Why You Shouldn't Use #! /usr/bin/env bash
When the path is hardcoded in the shebang, I have to run with that interpreter. Thus, #! /bin/bash forces me to use the default installed version of bash. Since bash features are very stable (try running a 2.x version of a Python script under Python 3.x) it's very unlikely that my particular BASH script will not work, and since my bash script is probably used by this system and other systems, using a non-standard version of bash may have undesired effects. It is very likely I want to make sure that the stable standard version of bash is used with my shell script. Thus, I probably want to hard code the path in my shebang.
There are a lot of systems that don't have Bash in /bin, FreeBSD and OpenBSD just to name a few. If your script is meant to be portable to many different Unices, you may want to use #!/usr/bin/env bash instead of #!/bin/bash.
Note that this does not hold true for sh; for Bourne-compliant scripts I exclusively use #!/bin/sh, since I think pretty much every Unix in existence has sh in /bin.
For invoking bash it is a little bit of overkill. Unless you have multiple bash binaries like your own in ~/bin but that also means your code depends on $PATH having the right things in it.
It is handy for things like python though. There are wrapper scripts and environments which lead to alternative python binaries being used.
But nothing is lost by using the exact path to the binary as long as you are sure it is the binary you really want.
#!/usr/bin/env bash
is definitely better because it finds the bash executable path from your system environment variable.
Go to your Linux shell and type
env
It will print all your environment variables.
Go to your shell script and type
echo $BASH
It will print your bash path (according to the environment variable list) that you should use to build your correct shebang path in your script.
I would prefer wrapping the main program in a script like below to check all bash available on system. Better to have more control on the version it uses.
#! /usr/bin/env bash
# This script just chooses the appropriate bash
# installed in system and executes testcode.main
readonly DESIRED_VERSION="5"
declare all_bash_installed_on_this_system
declare bash
if [ "${BASH_VERSINFO}" -ne "${DESIRED_VERSION}" ]
then
found=0
all_bash_installed_on_this_system="$(\
awk -F'/' '$NF == "bash"{print}' "/etc/shells"\
)"
for bash in $all_bash_installed_on_this_system
do
versinfo="$( $bash -c 'echo ${BASH_VERSINFO}' )"
[ "${versinfo}" -eq "${DESIRED_VERSION}" ] && { found=1 ; break;}
done
if [ "${found}" -ne 1 ]
then
echo "${DESIRED_VERSION} not available"
exit 1
fi
fi
$bash main_program "$#"
Normally #!path/to/command will trigger bash to prepend the command path to the invoking script when executed. Example,
# file.sh
#!/usr/bin/bash
echo hi
./file.sh will start a new process and the script will get executed like /bin/bash ./file.sh
Now
# file.sh
#!/usr/bin/env bash
echo hi
will get executed as /usr/bin/env bash ./file.sh which quoting from the man page of env describes it as:
env - run a program in a modified environment
So env will look for the command bash in its PATH environment variable and execute in a separate environment where the environment values can be passed to env like NAME=VALUE pair.
You can test this with other scripts using different interpreters like python, etc.
#!/usr/bin/env python
# python commands
Your question is biased because it assumes that #!/usr/bin/env bash is superior to #!/bin/bash. This assumption is not true, and here's why:
env is useful in two cases:
when there are multiple versions of the interpreter that are incompatible.
For example python 2/3, perl 4/5, or php 5/7
when the location depends on the PATH, for instance with a python virtual environment.
But bash doesn't fall under any of these two cases because:
bash is quite stable, especially on modern systems like Linux and BSD which form the vast majority of bash installations.
there's typically only one version of bash installed under /bin.
This has been the case for the past 20+ years, only very old unices (that nobody uses any longer) had a different location.
Consequently going through the PATH variable via /usr/bin/env is not useful for bash.
Add to these three good resons to use #!/bin/bash:
for system scripts (when not using sh) for which the PATH variable may not contain /bin.
For example cron defaults to a very strict PATH of /usr/bin:/bin which is fine, sure, but other context/environments may not include /bin for some peculiar reason.
when the user screwed-up his PATH, which is very common with beginners.
for security when for example you're calling a suid program that invokes a bash script. You don't want the interpreter to be found via the PATH variable which is entirely under the user's control!
Finally, one could argue that there is one legitimate use case of env to spawn bash: when one needs to pass extra environment variables to the interpreter using #!/usr/bin/env -S VAR=value bash.
But this is not a thing with bash because when you're in control of the shebang, you're also in control of the whole script, so just add VAR=value inside the script instead and avoid the aforementioned problems introduced by env with bash scripts.
Is it possible to specify a shebang line without knowing the path of the program you want to do the executing?
maybe don't specify the path
#!node
or specify several options
#!/usr/local/bin/node
#!/usr/bin/node
Extra points for cross platform solution (various flavors of linux, BSD, OSX etc...)
/usr/bin/env is specifically thought of for cross-platform solutions.
env executes utility after modifying the environment as specified on
the command line. The option name=value specifies an environmental
variable, name, with a value of value. The option `-i' causes env
to completely ignore the environment it inherits.
If no utility is specified, env prints out the names and values of
the variables in the environment, with one name=value pair per line.
so something in lines of:
#!/usr/bin/env node
Will be cross-platform and "the right way to go".
Contrary to what people may think there is not standard location for env so we can only grab some info regarding it's location:
/usr/bin/env - MacOS (10.12)
both /bin/env, /usr/bin/env - Fedora (25)
I am sure others will be able to extend the list.
Put a space after the shebang. If the program is in environment variable PATH, it should go.
#! perl
Of course, a special case for Perl would be
:
eval 'exec perl -S $0 ${1+"$#"}'
if 0;
This works on unix and OSX, even when there is no /usr/bin/env as noted by #Jens
Dont use shebang.
node <<eof
your node program here
eof
I am trying to run commands from ruby via system (or by using backticks), but am running into problems. When I try to call a command, the shell is unable to find it, even though I know it works if I call it straight. For example:
`zip`
>> sh: zip: command not found
The problem seems to be that ruby is using the sh shell, in which $PATH is not set correctly, rather than bash, and I am not sure why. The user my application is running under is set up to use bash by default.
Is there any way to tell ruby to use bash instead of sh?
As far as I know, the only way to do that is to explicitly invoke the shell, e.g.
`bash -c zip`
or
`#{ ENV['SHELL'] } -c zip`
Or with system: system("bash", "-c", command)
However ruby (and all processes spawned by ruby) should inherit the parent processes' environment and thus have $PATH set correctly even when using another shell. Do you maybe run ruby from a cron job or init script or the like, where PATH is simply not set correctly?
i'm guessing that Ruby uses the system() C function, which uses /bin/sh. In most systems, /bin/sh is just a symlink to other shell, so you can change to whatever you want
Why not just specify the full path the the zip executable.