I've noticed echo behaves slightly differently when called directly
root$echo "line1\nline2"
and when called via a script:
#! /bin/sh
echo "line1\nline2"
[...]
The first case would print:
line1\nline2
, while the latter would print
line1
line2
So echo is always assumed to have flag -e when used in a script?
Your script has a shebang line of #! /bin/sh, while your interactive shell is probably Bash. If you want Bash behavior, change the script to #! /bin/bash instead.
The two programs sh and bash are different shells. Even on systems where /bin/sh and /bin/bash are the same program, it behaves differently depending on which command you use to invoke it. (Though echo doesn't act like it has -e turned on by default in Bash's sh mode, so you're probably on a system whose sh is really dash.)
If you type sh at your prompt, you'll find that echo behaves the same way as in your script. But sh is not a very friendly shell for interactive use.
If you're trying to write maximally portable shell scripts, you should stick to vanilla sh syntax, but if you aren't writing for distribution far and wide, you can use Bash scripts - just make sure to tell the system that they are, in fact, Bash scripts, by putting bash in the shebang line.
The echo command is probably the most notoriously inconsistent command across shells, by the way. So if you are going for portability, you're better off avoiding it entirely for anything other than straight up, no-options, no-escapes, always-a-newline output. One pretty portable option is printf.
Related
In the same script, I want to use some CSH commands and some BASH commands.
Invoking one after the other giving me problems despite I am following the different syntax for respective shells. I want to know where is mistake in my code
Your suggestions are appreciated!!
I am a beginner to shell, especially so for CSH. but the code I got has been written in CSH entirely. Since I have some familiarity with CSH, I wanted to tweak the existing CSH code by including BASH commands, which I am comfortable using it. When I tried BASH commands after CSH by invoking !#/bin/bash, it is giving some errors. I want to know if I am missing any options!!
#!/bin/csh
----
----
----
#!/bin/bash
dir2in="/nethome/achandra/NCEI/CCSM4_Historical/Forecasts"
filin2 ="ccsm4_0_cfsrr_Fcst.${ENS}.cam2.h1.${yyear[${iimonth}]}-${mmon[${iimonth}]}-${ssday}-00000.nc"
cp $dirin/$filin /nethome/achandra/NCEI/CCSM4_Historical_Forecasts/
ln -s /nethome/achandra/NCEI/CCSM4_Historical/Forecasts/$filin /nethome/achandra/NCEI/CCSM4_Historical_Forecasts/"${$filin%.nc.cdo}.nc"
#!/bin/csh
I am getting errors such as
"dirin: Undefined variable."
You are asking here for "embedding one language into another", which, as #Bayou already explained, is not supported directly. Maybe you were spoiled from the HTML-world, where you can squeeze CSS and Javascript in between and maybe use some server side PHP or Ruby stuff too.
The closest to this are HERE-documents. If you write inside your bash script a
csh <<CSH_END
your ...
csh ....
commands ...
go here ...
CSH_END
these commands are executed in a child process driven by csh. It works the other way around with bash in the same way. Make sure that the terminator symbol (CSH_END in my example) starts in column 1.
Whether this will work for your application, I can't say, because things which run in the same process in your original script, now run in different processes.
You can't mix them up like you're suggesting. It's like asking "can I use PHP code in a Python script". However, most of the shells have options to run commands (-c), just as csh does. For using Bash within a sh script:
#! /bin/sh
CONDITION=$(/bin/bash -c "[[ 1 > 2 ]] || echo no")
echo $CONDITION
exit 0
Otherwise you could create separate files and execute them.
#! /bin/sh
CONDITION=$(./bash-script.sh)
echo $CONDITION
exit 0
You, of course, should use csh instead of sh. Both of my scripts will output the following text.
$ ./test.sh
no
I am using Bash
$ echo $SHELL
/bin/bash
and starting about a year ago I stopped using Shebangs with my Bash scripts. Can
I benefit from using #!/bin/sh or #!/bin/bash?
Update: In certain situations a file is only treated as a script with the
Shebang, example
$ cat foo.sh
ls
$ cat bar.sh
#!/bin/sh
ls
$ file foo.sh bar.sh
foo.sh: ASCII text
bar.sh: POSIX shell script, ASCII text executable
On UNIX-like systems, you should always start scripts with a shebang line. The system call execve (which is responsible for starting programs) relies on an executable having either an executable header or a shebang line.
From FreeBSD's execve manual page:
The execve() system call transforms the calling process into a new
process. The new process is constructed from an ordinary file, whose
name is pointed to by path, called the new process file.
[...]
This file is
either an executable object file, or a file of data for an interpreter.
[...]
An interpreter file begins with a line of the form:
#! interpreter [arg]
When an interpreter file is execve'd, the system actually execve's the
specified interpreter. If the optional arg is specified, it becomes the
first argument to the interpreter, and the name of the originally
execve'd file becomes the second argument
Similarly from the Linux manual page:
execve() executes the program pointed to by filename. filename must be
either a binary executable, or a script starting with a line of the
form:
#! interpreter [optional-arg]
In fact, if a file doesn't have the right "magic number" in it's header, (like an ELF header or #!), execve will fail with the ENOEXEC error (again from FreeBSD's execve manpage):
[ENOEXEC] The new process file has the appropriate access
permission, but has an invalid magic number in its
header.
If the file has executable permissions, but no shebang line but does seem to be a text file, the behaviour depends on the shell that you're running in.
Most shells seem to start a new instance of themselves and feed it the file, see below.
Since there is no guarantee that the script was actually written for that shell, this can work or fail spectacularly.
From tcsh(1):
On systems which do not understand the `#!' script interpreter conven‐
tion the shell may be compiled to emulate it; see the version shell
variable. If so, the shell checks the first line of the file to see if
it is of the form `#!interpreter arg ...'. If it is, the shell starts
interpreter with the given args and feeds the file to it on standard
input.
From FreeBSD's sh(1):
If the program is not a normal executable file (i.e., if it
does not begin with the “magic number” whose ASCII representation is
“#!”, resulting in an ENOEXEC return value from execve(2)) but appears to
be a text file, the shell will run a new instance of sh to interpret it.
From bash(1):
If this execution fails because the file is not in executable format,
and the file is not a directory, it is assumed to be a shell script, a
file containing shell commands. A subshell is spawned to execute it.
You cannot always depend on the location of a non-standard program like bash. I've seen bash in /usr/bin, /usr/local/bin, /opt/fsf/bin and /opt/gnu/bin to name a few.
So it is generally a good idea to use env;
#!/usr/bin/env bash
If you want your script to be portable, use sh instead of bash.
#!/bin/sh
While standards like POSIX do not guarantee the absolute paths of standard utilities, most UNIX-like systems seem to have sh in /bin and env in /usr/bin.
Scripts should always begin with a shebang line. If a script doesn't start with this, then it may be executed by the current shell. But that means that if someone who uses your script is running a different shell than you do, the script may behave differently. Also, it means the script can't be run directly from a program (e.g. the C exec() system call, or find -exec), it has to be run from a shell.
You might be interested in an early description by Dennis M Ritchie (dmr) who invented the #! :
From uucp Thu Jan 10 01:37:58 1980
.>From dmr Thu Jan 10 04:25:49 1980 remote from research
The system has been changed so that if a file
being executed begins with the magic characters #! , the rest of the
line is understood to be the name of an interpreter for the executed
file. Previously (and in fact still) the shell did much of this job;
it automatically executed itself on a text file with executable mode
when the text file's name was typed as a command. Putting the facility
into the system gives the following benefits.
1) It makes shell scripts more like real executable files, because
they can be the subject of 'exec.'
2) If you do a 'ps' while such a command is running, its real name
appears instead of 'sh'. Likewise, accounting is done on the basis of
the real name.
3) Shell scripts can be set-user-ID.
4) It is simpler to have alternate shells available; e.g. if you like
the Berkeley csh there is no question about which shell is to
interpret a file.
5) It will allow other interpreters to fit in more smoothly.
To take advantage of this wonderful opportunity, put
#! /bin/sh
at the left margin of the first line of your shell scripts. Blanks
after ! are OK. Use a complete pathname (no search is done). At the
moment the whole line is restricted to 16 characters but this limit
will be raised.
Hope this helps
If you write bash scripts, i.e. non portable scripts containing bashisms, you should keep using the #!/bin/bash shebang just to be sure the correct interpreter is used. You should not replace the shebang by #!/bin/sh as bash will run in POSIX mode so some of your scripts might behave differently.
If you write portable scripts, i.e. scripts only using POSIX utilities and their supported options, you might keep using #!/bin/sh on your system (i.e. one where /bin/sh is a POSIX shell).
It you write stricly conforming POSIX scripts to be distributed in various platforms and you are sure they will only be launched from a POSIX conforming system, you might and probably should remove the shebang as stated in the POSIX standard:
As it stands, a strictly conforming application must not use "#!" as the first two characters of the file.
The rationale is the POSIX standard doesn't mandate /bin/sh to be the POSIX compliant shell so there is no portable way to specify its path in a shebang. In this third case, to be able to use the 'find -exec' syntax on systems unable to run a shebangless still executable script, you can simply specify the interpreter in the find command itself, eg:
find /tmp -name "*.foo" -exec sh -c 'myscript "$#"' sh {} +
Here, as sh is specified without a path, the POSIX shell will be run.
The header is useful since it specifies which shell to use when running the script. For example, #!/bin/zsh would change the shell to zsh instead of bash, where you can use different commands.
For example, this page specifies the following:
Using #!/bin/sh, the default Bourne shell in most commercial variants
of UNIX, makes the script portable to non-Linux machines, though you
sacrifice Bash-specific features ...
TL;DR: always in scripts; please not in source'd scripts
Always in your parent
FYI: POSIX compliant is #!/bin/bash, not #!/bin/sh
You want to clarify this so that nothing else overrides the interpreter your script is made for.
You don't want a user at the terminal using zsh to have trouble if your script was written for POSIX bash scripts.
You don't want to run source in your #!/bin/bash unrecognized by #!/bin/sh, someone in an sh terminal have it break the script because it is expecting the simple/POSIX . for including source'd files
You don't want e.g. zsh features - not available in other interpreters - to make their way into your bash code. So, put #!/bin/bash in all your script headers. Then, any of your zsh habits in your script will break so you know to remove them before your roll-out.
It's probably best, especially so POSIX-compliant scripts don't break in a terminal like zsh.
Not expected for included source scripts
FYI: POSIX compliant for sourcing text in a BASH script is ., not source
You can use either for sourcing, but I'll do POSIX.
Standard "shebanging" for all scripting:
parent.sh:
#!/bin/bash
echo "My script here"
. sourced.sh # child/source script, below
sourced.sh:
echo "I am a sourced child script"
But, you are allowed to do this...
sourced.sh: (optional)
#!/bin/bash
echo "I am a sourced child script"
There, the #!/bin/bash "shebang" will be ignored. The main reason I would use it is for syntax highlighting in my text editor. However, in the "proper" scripting world, it is expected that your rolled-out source'd script will not contain the shebang.
In addition to what the others said, the shebang also enables syntax highlighting in some text editors, for example vim.
$SHELL and #!/bin/bash or #!/bin/sh are different.
To start, #!/bin/sh is a symlink to /bin/bash on most Linux systems (on Ubuntu it is now /bin/dash)
But on whether to start with /bin/sh or /bin/bash:
Bash and sh are two different shells. Basically bash is sh, with more
features and better syntax. Most commands work the same, but they are
different.
Just assume if you're writing a bash script, stick with /bin/bash and not /sh because problems can arise.
$SHELL does not necessarily reflect the currently running shell.
Instead, $SHELL is the user's preferred shell, which is typically the
one set in /etc/passwd. If you start a different shell after logging
in, you can not necessarily expect $SHELL to match the current shell
anymore.
This is mine for example, but it could also be /root:/bin/dash or /root:/bin/sh depending on which shell you have input in passwd. So to avoid any problems, keep the passwd file at /bin/bash and then using $SHELL vs. #!/bin/bash wouldn't matter as much.
root#kali:~/Desktop# cat /etc/passwd
root:x:0:0:root:/root:/bin/bash
Sources:
http://shebang.mintern.net/bourne-is-not-bash-or-read-echo-and-backslash/
https://unix.stackexchange.com/questions/43499/difference-between-echo-shell-and-which-bash
http://man.cx/sh
http://man.cx/bash
I've recently switched to the ksh93 shell. I did this by adding the following two lines to my .profile file
export SHELL=/usr/local/bin/ksh93
exec $SHELL
Since I did that some simple scripts have started misbehaving in a way I don't understand. I narrowed it down to the following simple script called say test.sh
#!/bin/ksh
echo $0 $1
If I type the command test.sh fred I would expect to see the same output test.sh fred. Instead I see test.sh noglob. If I remove the shebang or if I change it to read #!/usr/local/bin/ksh93 then the script works as expected.
Can anyone explain what's going on, or what to do about it? I'm stumped.
I'm using Solaris 5.9 if it makes any difference.
I notice from the comments that your .kshrc has a set noglob. The set command with no options will set the command-line parameters, which is why $1 is "noglob", it should be set -o noglob.
By the way, setting noglob is weird, are you sure you want that?
I suspect (as others have mentioned) that /bin/ksh is Korn shell 88.
There is an important difference between ksh88 and ksh93 with regards to .kshrc. On ksh88 .kshrc is executed for every korn shell process, even non-interactive ones (scripts). In ksh93 .kshrc is not executed for shell scripts, only for interactive login shells.
When you do exec $SHELL that is not a login shell, it is better to change your entry in /etc/passwd. By the way, using variable SHELL is a bad idea, since that is set by the login shell.
There's probably an alias on ksh in your system with noglob set as an option, or noglob is being passed as a default parameter by default in your old shell. You should also check what ksh you're really calling (check if there's a link to another shell in from /bin/ksh). ksh --version should give some insight as well.
As a last point, instead of calling the shell directly i'd recommend to use
#!/usr/bin/env ksh
I've a file new.sh
i=5
i=$[i+1]
echo $i
when I execute ./new.sh, it shows 6. But when I execute "sh new.sh", it shows
$[i+1]
as output. I just want to know why, and I need a code which will work on both.
Many Linux distributions use dash as their standard shell for scripts and therefore /bin/sh is only a symlink to /bin/dash, which is more lightweight, but lags some functionality compared with bash. Check that with:
ls -l /bin/sh
If you want to write POSIX compatible scripts, you should use foo=$n; $((n=n+1)) instead of foo=$((n++)) and foo=$((n=n+1)) instead of foo=$((++n)). The form $[] is deprecated and should be avoided.
The OS decides what type of file it is based on the first line. Make it:
#!/bin/bash
The behaviour is different because when bash is run as sh, it turns off bash-specific features and hews more closely to the POSIX shell standard.
If you want your code to work the same under both bash and sh, you will need to write your code in the sh subset of the bash syntax.
Why do all script files start with
#!/bin/sh
or with
#!/bin/csh
Is that required? What's the purpose of this? And what's the difference between the two?
This is known as a Shebang:
http://en.wikipedia.org/wiki/Shebang_(Unix)
#!interpreter [optional-arg]
A shebang is only relevant when a script has the execute permission (e.g. chmod u+x script.sh).
When a shell executes the script it will use the specified interpreter.
Example:
#!/bin/bash
# file: foo.sh
echo 1
$ chmod u+x foo.sh
$ ./foo.sh
1
The #! line tells the kernel (specifically, the implementation of the execve system call) that this program is written in an interpreted language; the absolute pathname that follows identifies the interpreter. Programs compiled to machine code begin with a different byte sequence -- on most modern Unixes, 7f 45 4c 46 (^?ELF) that identifies them as such.
You can put an absolute path to any program you want after the #!, as long as that program is not itself a #! script. The kernel rewrites an invocation of
./script arg1 arg2 arg3 ...
where ./script starts with, say, #! /usr/bin/perl, as if the command line had actually been
/usr/bin/perl ./script arg1 arg2 arg3
Or, as you have seen, you can use #! /bin/sh to write a script intended to be interpreted by sh.
The #! line is only processed if you directly invoke the script (./script on the command line); the file must also be executable (chmod +x script). If you do sh ./script the #! line is not necessary (and will be ignored if present), and the file does not have to be executable. The point of the feature is to allow you to directly invoke interpreted-language programs without having to know what language they are written in. (Do grep '^#!' /usr/bin/* -- you will discover that a great many stock programs are in fact using this feature.)
Here are some rules for using this feature:
The #! must be the very first two bytes in the file. In particular, the file must be in an ASCII-compatible encoding (e.g. UTF-8 will work, but UTF-16 won't) and must not start with a "byte order mark", or the kernel will not recognize it as a #! script.
The path after #! must be an absolute path (starts with /). It cannot contain space, tab, or newline characters.
It is good style, but not required, to put a space between the #! and the /. Do not put more than one space there.
You cannot put shell variables on the #! line, they will not be expanded.
You can put one command-line argument after the absolute path, separated from it by a single space. Like the absolute path, this argument cannot contain space, tab, or newline characters. Sometimes this is necessary to get things to work (#! /usr/bin/awk -f), sometimes it's just useful (#! /usr/bin/perl -Tw). Unfortunately, you cannot put two or more arguments after the absolute path.
Some people will tell you to use #! /usr/bin/env interpreter instead of #! /absolute/path/to/interpreter. This is almost always a mistake. It makes your program's behavior depend on the $PATH variable of the user who invokes the script. And not all systems have env in the first place.
Programs that need setuid or setgid privileges can't use #!; they have to be compiled to machine code. (If you don't know what setuid is, don't worry about this.)
Regarding csh, it relates to sh roughly as Nutrimat Advanced Tea Substitute does to tea. It has (or rather had; modern implementations of sh have caught up) a number of advantages over sh for interactive usage, but using it (or its descendant tcsh) for scripting is almost always a mistake. If you're new to shell scripting in general, I strongly recommend you ignore it and focus on sh. If you are using a csh relative as your login shell, switch to bash or zsh, so that the interactive command language will be the same as the scripting language you're learning.
This defines what shell (command interpreter) you are using for interpreting/running your script. Each shell is slightly different in the way it interacts with the user and executes scripts (programs).
When you type in a command at the Unix prompt, you are interacting with the shell.
E.g., #!/bin/csh refers to the C-shell, /bin/tcsh the t-shell, /bin/bash the bash shell, etc.
You can tell which interactive shell you are using the
echo $SHELL
command, or alternatively
env | grep -i shell
You can change your command shell with the chsh command.
Each has a slightly different command set and way of assigning variables and its own set of programming constructs. For instance the if-else statement with bash looks different that the one in the C-shell.
This page might be of interest as it "translates" between bash and tcsh commands/syntax.
Using the directive in the shell script allows you to run programs using a different shell. For instance I use the tcsh shell interactively, but often run bash scripts using /bin/bash in the script file.
Aside:
This concept extends to other scripts too. For instance if you program in Python you'd put
#!/usr/bin/python
at the top of your Python program