I've made a little bash script which clean some files (*~, *#, etc.) and the terminal but I've seen that the history command is disabled by default in non-interactive shells by bash (I wanted to add "history -c").
I'm just curious of why ?
In General
For an overwhelming majority of scripts, reading dotfiles for history would be pure startup-time overhead, with that content never used at runtime. This would make all shell scripts take longer to start up, with no compensating benefit.
If noninteractive scripts did support history expansion, this would make their behavior dependent on prior interactive actions, thus harder to predict and different between invocations. This is, in particular, a compelling reason not to have set -H on by default in noninteractive script invocation.
History is specified in the User Portability Utilities section of the POSIX sh standard, which is focused around support for interactive scripting. Quoting from the standard, with emphasis added:
When the sh utility is being used interactively, it shall maintain a list of commands previously entered from the terminal in the file named by the HISTFILE environment variable. The type, size, and internal format of this file are unspecified. Multiple sh processes can share access to the file for a user, if file access permissions allow this; see the description of the HISTFILE environment variable.
Finally, some context:
history -c is needed in an interactive script because that interactive shell will be writing a new copy of HISTFILE when it exits; if you didn't use a special command, it would potentially write back content that you instead want to clear. In a noninteractive shell, it won't be writing any history anyhow, so you might as well simply delete or truncate the history file.
Related
I'm thinking of writing my own ls command. Mostly as a learning experience, but I also think I can make it a bit more useful (for me) than the default.
I'm worried though that if I alias ls, this also interferes with any bash/sh scripts that use ls as it's output.
Is there a way to override ls, but only when it's not used in scripts (or pipes?)
You're worried that aliasing your version of ls will interfere with other processes.
Let's have a look at the POSIX standard.
From the man page of alias:
Historical versions of the KornShell have allowed aliases to be
exported to scripts that are invoked by the same shell. This is
triggered by the alias −x flag; it is allowed by this volume of
POSIX.1‐2008 only when an explicit extension such as −x is used. The
standard developers considered that aliases were of use primarily to
interactive users and that they should normally not affect shell
scripts called by those users; functions are available to such
scripts.
So, what does "normally" mean for bash? For instance, which version of ls would be used inside a shell script?
From the man page of bash:
Aliases are not expanded when the shell is not interactive, unless the
expand_aliases shell option is set using shopt (see the description of
shopt under SHELL BUILTIN COMMANDS below).
That means, you don't have to worry about shell scripts - they will use the unaliased version of ls.
But what about pipes? Again, we can combine those two man pages for great good:
From the man page of bash:
Each command in a pipeline is executed as a separate process (i.e., in
a subshell).
From the man page of alias:
An alias definition shall affect the current shell execution
environment and the execution environments of the subshells of the
current shell. When used as specified by this volume of POSIX.1‐2008,
the alias definition shall not affect the parent process of the
current shell nor any utility environment invoked by the shell
That is, while your alias will not be used inside shell scripts, it will be used in pipes.
I have been working on a few scripts on CentOS 7 and sometimes I see:
#!/bin/sh -
on the first line. Looking at the man page for sh I see the following under the Special Parameters
- Expands to the current option flags as specified upon invocation,
by the set builtin command, or those set by the shell
itself (such as the -i option).
What exactly does this mean? When do I need to use this special parameter option??
The documentation you are reading has nothing to do with the command line you're looking at: it's referring to special variables. In this case, if you run echo $- you will see "the current option flags as specified upon invocation...".
If you take a look at the OPTIONS part of the bash man page, you will find:
-- A -- signals the end of options and disables further option processing.
Any arguments after the -- are treated as filenames and arguments. An
argument of - is equivalent to --.
In other words, an argument of - simply means "there are no other options after this argument".
You often see this used in situation in which you want to avoid filenames starting with - accidentally being treated as command options: for example, if there is a file named -R in your current directory, running ls * will in fact behave as ls -R and produce a recursive listing, while ls -- * will not treat the -R file specially.
The single dash when used in the #! line is meant as a security precaution. You can read more about that here.
/bin/sh is an executable representing the system shell. Actually, it is usually implemented as a symbolic link pointing to the executable for whichever shell is the system shell. The system shell is kind of the default shell that system scripts should use. In Linux distributions, for a long time this was usually a symbolic link to bash, so much so that it has become somewhat of a convention to always link /bin/sh to bash or a bash-compatible shell. However, in the last couple of years Debian (and Ubuntu) decided to switch the system shell from bash to dash - a similar shell - breaking with a long tradition in Linux (well, GNU) of using bash for /bin/sh. Dash is seen as a lighter, and much faster, shell which can be beneficial to boot speed (and other things that require a lot of shell scripts, like package installation scripts).
Dash is fairly well compatible with bash, being based on the same POSIX standard. However, it doesn't implement the bash-specific extensions. There are scripts in existence that use #!/bin/sh (the system shell) as their shebang, but which require bash-specific extensions. This is currently considered a bug that should be fixed by Debian and Ubuntu, who require /bin/sh to be able to work when pointed to dash.
Even though Ubuntu's system shell is pointing to dash, your login shell as a user continues to be bash at this time. That is, when you log in to a terminal emulator anywhere in Linux, your login shell will be bash. Speed of operation is not so much a problem when the shell is used interactively, and users are familiar with bash (and may have bash-specific customization in their home directory).
I am already using a makefile, and I was hoping to be able to use it to store a few useful aliases that the user could then invoke. I know that I can make a bash file with the aliases already built in, so and run it with source, so I can do something like:
# File: aliases.sh
alias useful="command to run"
alias also-useful="another command -to run"
Then I can run this in the current terminal session with:
source ./aliases.sh
Using a Makefile
So I was hoping to achieve something similar with a makefile, I was hoping to have a simple aliases entry, so the user could just run:
make aliases
I prefer to avoid adding an extra file if this is at all possible, because I don't want to add extra files for such simple tasks. If you have any suggestions that would be better, I'd be open to hearing them too.
If what you're asking is for make aliases to create aliases that you can then invoke at your shell prompt, something like:
$ make aliases
$ useful
then that is impossible and the reason has nothing to do with make.
In a UNIX/POSIX system the process hierarchy is strict: a process starts one or more sub-processes, and each of those can start more, etc. So a login manager process starts your shell (or your window manager), your shell starts make, which is another process, and make will run another shell as a subprocess to run each recipe, and each shell will run programs like compilers, commands like rm which are also processes, etc.
It is a fundamental rule of all processes that they cannot modify the environment (memory) of their parents (and they can only modify the environment of their children before they are started). So, if you start a new shell and change your working directory then exit that shell, the parent's shell is not changed. If you set an environment variable in the child process, the variable is not set in the parent. Etc.
Shell aliases are part of a particular shell's memory. So a program you start cannot create aliases in its parent shell. It doesn't matter if that program is make or anything else.
That's why you have to use the special command source to load those into your shell: instead of running a new shell, the source command tells the current shell to run the commands in the script as if you'd typed them in at the command line... so no new process is created and the current shell's environment and memory is modified. If you ran your aliases file as a shell script, via aliases rather than source aliases, then a new shell would be created, the aliases would be defined, then the shell would exit and all the aliases would be gone again.
So, all that to say it's not possible for make to define aliases in the shell that invokes it: the operating system won't allow it.
my scripts are developed using a shebang line as "#!/bin/ksh" and the default shell is
$ echo $SHELL
/bin/ksh
i am moving all these scripts without changing the shebang line to a new machine where the default shell is
$ echo $SHELL
/bin/bash
Should i worry about this ?
I am guessing there should not be any issue as the shebang line will override the interpreter and use ksh as defined in the scripts and as i want it to be.
Please share your thoughts ..
The default shell does not affect how scripts are executed (unless you're using a shell that does something very strange).
An executable script with no #! line will be executed with /bin/sh. Actually that doesn't appear to be correct, but in any case you don't have to worry about that.
As long as your scripts start with #!/bin/ksh and you execute them normally, the system will execute them by passing them to /bin/ksh.
One thing you might have to worry about is whether /bin/ksh exists, and if it does, just what it is. On my system (Linux Mint 17), /bin/ksh is a symlink to /etc/alternatives/ksh, which in turn is a symlink to /bin/ksh93.
Scripts with #!/bin/ksh are probably common enough that almost all UNIX-like systems will cater to them, and will install something that behaves like ksh at that location.
Note that what you call the "default shell", specified by $SHELL, is not a system-wide default. It's just the value of a particular environment variable. That variable is set for each user on login based on the shell specified in /etc/passwd or equivalent; thus different users can have different default shells. You can change the value of $SHELL after logging in. The entry in /etc/passwd or equivalent is set when the account is created, and can be changed later. Most systems have a default user shell that's set for new accounts if no shell is specified (for example, most Linux systems user /bin/bash).
The supposition given is correct: The shebang line is honored on any execve() call. Only if your scripts are sourced (. yourscript or source yourscript) or lack a valid shebang do you need to care which interpreter they're called from.
If this were not true, scripts in non-shell languages wouldn't work as expected (as the Python interpreter, for instance, is never a system's default shell).
The kernel will use the shebang line to select the appropriate interpreter to use when the script is executed in the default manner, whether it is sh, bash, ksh, expect, python, or whatever. The only real issue to be wary of is scripts written on a system where sh is one specific shell (e.g. bash) that are then moved to another system where sh is a different shell (e.g. dash) since they may use shell features found in the former that do not exist in the latter.
Parentheses are used in shell to group commands, executing them in a subshell, so that they don't affect parent shell environment.
Now, I wonder if this spawned subshells do read init files, like any other shells.
From direct experience I'd say they don't.
But I don't find any place where that is stated.
Also, is this different for different types of shells?
In general, the behaviour of shells is not particularly well-defined since the only applicable standard was essentially the result of reverse-engineering the behaviour of various commonly-used shells. Nonetheless, there is an expectation that shells will converge to the standard, albeit with extensions.
Having said that, here's what Posix says about (...):
(compound-list)
Execute compound-list in a subshell environment.
And a subshell environment:
A subshell environment shall be created as a duplicate of the shell environment, except that signal traps that are not being ignored shall be set to the default action. Changes made to the subshell environment shall not affect the shell environment. Command substitution, commands that are grouped with parentheses, and asynchronous lists shall be executed in a subshell environment. Additionally, each command of a multi-command pipeline is in a subshell environment; as an extension, however, any or all commands in a pipeline may be executed in the current environment. All other commands shall be executed in the current shell environment.
The take-away here is that the subshell environment is a "duplicate of the shell environment", and not a new shell; the only difference is the specific exception for signal traps. So it is pretty clearly not expected that the subshell will undergo reinitialization, such as rereading startup files.
Posix only provides one requirement for start-up files, which is documented in Section 4 in the description of the sh utility:
ENV
This variable, when and only when an interactive shell is invoked, shall be subjected to parameter expansion by the shell, and the resulting value shall be used as a pathname of a file containing shell commands to execute in the current environment.
Most shells implement a richer set of start-up files with specific names, so that the ENV variable may not be necessary. So the fact that Posix states "when and only when an interactive shell is invoked" is only indicative, but I think it is a good indication.
When a subshell is started, it is just a child resulting from a fork(), thus it inherits all from the father and doesn't need to read the config files, whose it already knows.
Conversely, when a shell is exec()-uted, it looses everything except PIDs and redirections, thus it has to read again config files.