I've recently switched to the ksh93 shell. I did this by adding the following two lines to my .profile file
export SHELL=/usr/local/bin/ksh93
exec $SHELL
Since I did that some simple scripts have started misbehaving in a way I don't understand. I narrowed it down to the following simple script called say test.sh
#!/bin/ksh
echo $0 $1
If I type the command test.sh fred I would expect to see the same output test.sh fred. Instead I see test.sh noglob. If I remove the shebang or if I change it to read #!/usr/local/bin/ksh93 then the script works as expected.
Can anyone explain what's going on, or what to do about it? I'm stumped.
I'm using Solaris 5.9 if it makes any difference.
I notice from the comments that your .kshrc has a set noglob. The set command with no options will set the command-line parameters, which is why $1 is "noglob", it should be set -o noglob.
By the way, setting noglob is weird, are you sure you want that?
I suspect (as others have mentioned) that /bin/ksh is Korn shell 88.
There is an important difference between ksh88 and ksh93 with regards to .kshrc. On ksh88 .kshrc is executed for every korn shell process, even non-interactive ones (scripts). In ksh93 .kshrc is not executed for shell scripts, only for interactive login shells.
When you do exec $SHELL that is not a login shell, it is better to change your entry in /etc/passwd. By the way, using variable SHELL is a bad idea, since that is set by the login shell.
There's probably an alias on ksh in your system with noglob set as an option, or noglob is being passed as a default parameter by default in your old shell. You should also check what ksh you're really calling (check if there's a link to another shell in from /bin/ksh). ksh --version should give some insight as well.
As a last point, instead of calling the shell directly i'd recommend to use
#!/usr/bin/env ksh
Related
In multiple different shells, the value of SHELL remains mostly constant:
bash$ echo $SHELL
/bin/bash
bash$ csh
csh$ echo $SHELL
/bin/bash
csh$ exec tcsh
csh$ echo $SHELL
/bin/bash
csh$ exec ksh
$ echo $SHELL
/bin/bash
$ exec dash
$ echo $SHELL
/bin/bash
$ exec zsh
zsh$ echo $SHELL
banana
zsh$ exec bash
bash$ echo $SHELL
banana
Since $SHELL is not useful for determining the currently running shell, what is its purpose?
As pointed out in the question, SHELL is (almost) completely worthless for determining the currently running shell. Although there is some correlation between the value of $SHELL and the user's login shell, that relationship is tenuous at best and $SHELL cannot be used to reliably determine which shell you are currently running.
Instead, the purpose of SHELL is to allow the user to communicate a preference to the system, similar to the use of PAGER or EDITOR. If a process needs the user to edit a file, it should examine EDITOR and open an editor that the user likes. If a process needs to present textual information to the user, it should check the value of PAGER to determine which program to use. If a process needs to invoke a SHELL to execute commands, it should execute the shell specified in SHELL. SHELL should probably not be used in quite the same way as PAGER or EDITOR, which really are just user preferences. It is more likely to be used to work around certain behaviors. For instance, if /bin/sh is a bloated shell, or some not-strictly-standards conforming shell that causes some obscure error, or even if /bin/sh actually fixes bugs that older scripts rely on for correct behavior, the user (or the system, via default startup files) may prefer to set SHELL to something like /usr/xpg4/bin/sh to side-step those issues.
Note that ksh documents a slightly different usage of SHELL, and states "The pathname of the shell is kept in the environment." Inspecting the code (https://github.com/ksh2020/ksh/blob/master/src/lib/libast/path/pathshell.c#L104), we see that the documentation is not quite accurate, as reflected in the behavior seen above. The bash documentation states "This variable expands to the full pathname to the shell. If it is not set when the shell starts, bash assigns to it the full pathname of the current user's login shell"
It is very likely that most users will set SHELL to the value of their login shell, so bash's behavior is reasonable. After all, if you have a favorite shell, it makes sense to use it as your login shell and to set it in SHELL. On the other hand, if you are using
tclsh or python as your login shell, it would make sense to set SHELL to /bin/sh or similar. Since many users will set SHELL to their login shell there is a correlation between the value of SHELL and the shell you are currently using, but this relationship is certainly not guaranteed.
In the question, note the value of $SHELL in zsh is set to banana, and that value persists into the next invocation of bash. This is a bit pathological, but might be instructive. What is happening here is simply that the value of SHELL in the $HOME/.zshrc was set to the string banana. When bash was invoked, that value was retained. It is the prerogative of the user to set SHELL to any value they like, and it does not need to be in any way related to the current shell nor even to make any sense.
Original Title: Indirect parameter substitution breaks when the script is sourced (zsh)
zsh 5.7.1 (x86_64-apple-darwin19.0)
GNU bash, version 4.4.20(1)-release (x86_64-pc-linux-gnu)
I’m developing a shell script on a Mac and I’m trying to keep it portable between bash & zsh, so array indexing is a consideration. I know that I can set KSH_ARRAYS to get indexing to start at 0, but I decided to query the OS for the shell that’s in use and set the start index accordingly, which led to the issue described below.
It made sense (to me anyway!) to use indirect expansion, which is what led to the problem. Consider the script indirect.sh:
#! /bin/bash
declare -r ARRAY_START_BASH=0
declare -r ARRAY_START_ZSH=1
declare -r SHELL_BASH=0
declare -r SHELL_ZSH=1
# Indirect expansion is used to reference the values of the variables declared
# in this case statement e.g. ${!ARRAY_START}
case $(basename $SHELL) in
"bash" )
declare -r SHELL_ID=SHELL_BASH
declare -r ARRAY_START=ARRAY_START_BASH
;;
"zsh" )
declare -r SHELL_ID=SHELL_ZSH
declare -r ARRAY_START=ARRAY_START_ZSH
;;
* )
return 1
;;
esac
echo "Shell ID: ${!SHELL_ID} Index arrays from: ${!ARRAY_START}"
It works fine when run from the command line while in the same directory:
<my home> ~ % echo "$(./indirect.sh)"
Shell ID: 1 Index arrays from: 1
Problems arise when I source the script:
<my home> ~ % echo "$(. ~/indirect.sh)"
/Users/<me>/indirect.sh:28: bad substitution
I don’t understand why sourcing the script changes the behavior of the parameter expansion.
Is this expected behavior? If so, I’d be grateful if someone could explain it and hopefully, offer a work around.
The problem described in the original post has nothing to do with indirect expansion. The difference in behavior is a result of different shells being invoked depending on whether the script is “executed” or “sourced”. These differences reveal the basic flaw in deriving the shell from the $SHELL variable that underpins the script's design. If the shell defined in $SHELL does not match the shebang, the script will fail either when sourced or executed. An explanation follows.
Indirect expansion doesn’t offer value in the given scenario because values could just as easily be assigned directly. They’ll have to be assigned that way regardless given the different syntax used for indirect expansion between shells. In fact, other syntax differences between shells makes the entire premise for detecting the shell moot! However, putting that aside, the difference in behavior is a result of different shells being invoked based on whether the script is “executed” or “sourced”. The behavior of sourcing is well documented with numerous explanations on the web, but for context here’s how it works:
Executing a Script
Use the “./“ syntax to execute a script.
When run this way, the script executes in a sub-shell. Any changes the
script makes to it’s shell are applied to the sub-shell, not the shell
in which the script was launched, so those changes are lost when the
shell exits because the sub-shell in which it executed is destroyed as
well. For example, if the script changes the working directory, it
does so in the sub-shell. The working directory of the main shell that
launched the script is unchanged when the script terminates. If you
want to make changes to the shell in which the script was launched, it
must be sourced.
Sourcing a Script
Use the “source “ syntax to source a
script. When run this way, the script essentially becomes an argument
for the source command, which handles invoking the appropriate
execution. Some shells (e.g. ksh) use a single period “.” instead of
“source”.
When a script is executed with the “./“ syntax, the shebang at the top of the file is used to determine which shell to use. When a script is sourced, the shebang is ignored and the shell in which the script is launched is used instead. Also note that the period that appears in the “./“ command syntax used to execute a script, is not related to the period that’s occasionally used as an alias for the source command.
The script in the post uses bash in the shebang statement, so it works when executed because it’s run using bash. When it’s sourced from zsh, it encounters the incorrect indirect expansion syntax:
“${!A_VAR}"
The correct syntax is:
"${(P)A_VAR}"
However, correcting the syntax won’t help because it will then fail when executed. The shebang will invoke bash and the syntax will be wrong again. That renders indirection useless for accessing a variable designed to indicate the shell in use. More importantly, a design based on querying an environment variable for the shell is flawed due to differences in the shell that’s ultimately used depending on whether the script is executed or sourced.
To add to your answer (what I'm going to say is too long for a comment), I can not think of any application, why your script could be useful if not sourced. Actually, I came accross the need of such a script by myself in exactly one occasion:
Since I use as interactive shell not only zsh, but also sometimes bash, so I have written my .zshrc and .bashrc to set up everything (including defining variables and shell functions for interactive use). In order to safe work,
I try to put code which works under both bash and zsh into a single file (say: .commonrc), and my .zshrc and .bashrc have inside them a
source .commonrc
While many things are so different in bash and zsh, that I can't put them into .commonrc, some can, provided I do some tweaking. One reason for headache is obviously the different indexing of arrays, which you seemingly try to solve. So I have also a similar feature. However, I don't nee ca case construct for this. Instead, my .bashrc looks like this (using your naming of the variables):
...
declare -r ARRAY_START=0
source .commonrc
...
and my .zshrc looks like this:
...
declare -r ARRAY_START=1
source .commonrc
...
Since it does not happen that the .bashrc is run from a zsh and vice versa, I don't need to query what kind of shell I have.
In the same script, I want to use some CSH commands and some BASH commands.
Invoking one after the other giving me problems despite I am following the different syntax for respective shells. I want to know where is mistake in my code
Your suggestions are appreciated!!
I am a beginner to shell, especially so for CSH. but the code I got has been written in CSH entirely. Since I have some familiarity with CSH, I wanted to tweak the existing CSH code by including BASH commands, which I am comfortable using it. When I tried BASH commands after CSH by invoking !#/bin/bash, it is giving some errors. I want to know if I am missing any options!!
#!/bin/csh
----
----
----
#!/bin/bash
dir2in="/nethome/achandra/NCEI/CCSM4_Historical/Forecasts"
filin2 ="ccsm4_0_cfsrr_Fcst.${ENS}.cam2.h1.${yyear[${iimonth}]}-${mmon[${iimonth}]}-${ssday}-00000.nc"
cp $dirin/$filin /nethome/achandra/NCEI/CCSM4_Historical_Forecasts/
ln -s /nethome/achandra/NCEI/CCSM4_Historical/Forecasts/$filin /nethome/achandra/NCEI/CCSM4_Historical_Forecasts/"${$filin%.nc.cdo}.nc"
#!/bin/csh
I am getting errors such as
"dirin: Undefined variable."
You are asking here for "embedding one language into another", which, as #Bayou already explained, is not supported directly. Maybe you were spoiled from the HTML-world, where you can squeeze CSS and Javascript in between and maybe use some server side PHP or Ruby stuff too.
The closest to this are HERE-documents. If you write inside your bash script a
csh <<CSH_END
your ...
csh ....
commands ...
go here ...
CSH_END
these commands are executed in a child process driven by csh. It works the other way around with bash in the same way. Make sure that the terminator symbol (CSH_END in my example) starts in column 1.
Whether this will work for your application, I can't say, because things which run in the same process in your original script, now run in different processes.
You can't mix them up like you're suggesting. It's like asking "can I use PHP code in a Python script". However, most of the shells have options to run commands (-c), just as csh does. For using Bash within a sh script:
#! /bin/sh
CONDITION=$(/bin/bash -c "[[ 1 > 2 ]] || echo no")
echo $CONDITION
exit 0
Otherwise you could create separate files and execute them.
#! /bin/sh
CONDITION=$(./bash-script.sh)
echo $CONDITION
exit 0
You, of course, should use csh instead of sh. Both of my scripts will output the following text.
$ ./test.sh
no
I am using Windows Linux Subsystem (Ubuntu). When I try to set a newline-delimiter, I lose my 'n'-characters. My simplified script;
#!/bin/sh
echo $HOME #gives /home/hennio
IFS=$'\n'
echo $HOME #gives /home/he io
IFS=$'\n\b' didnt solve the problem. I checked my shebang with $(which sh), it is correct (although using zsh).
Searching on internet didnt give any results. Can someone please tell me whats going on? It driving me nuts..
To maintain compatibility, a shell invoked as /bin/sh usually tries to emulate a POSIX shell or some variant of a Bourne shell. Neither POSIX nor Bourne support $'...'. There are two possible solutions:
Method 1: Use the shebang of a shell, like bash, that supports $'...'.
Or,
Method 2: Use a POSIX method to assigning a newline to IFS:
IFS='
'
(Hat tip: Gordon Davisson)
Documentation
From man zsh:
Zsh tries to emulate sh or ksh when it is invoked as sh or ksh
respectively
From man bash:
If bash is invoked with the name sh, it tries to mimic the
startup behavior of historical versions of sh as closely as possible,
while conforming to the POSIX standard as well.
I've noticed echo behaves slightly differently when called directly
root$echo "line1\nline2"
and when called via a script:
#! /bin/sh
echo "line1\nline2"
[...]
The first case would print:
line1\nline2
, while the latter would print
line1
line2
So echo is always assumed to have flag -e when used in a script?
Your script has a shebang line of #! /bin/sh, while your interactive shell is probably Bash. If you want Bash behavior, change the script to #! /bin/bash instead.
The two programs sh and bash are different shells. Even on systems where /bin/sh and /bin/bash are the same program, it behaves differently depending on which command you use to invoke it. (Though echo doesn't act like it has -e turned on by default in Bash's sh mode, so you're probably on a system whose sh is really dash.)
If you type sh at your prompt, you'll find that echo behaves the same way as in your script. But sh is not a very friendly shell for interactive use.
If you're trying to write maximally portable shell scripts, you should stick to vanilla sh syntax, but if you aren't writing for distribution far and wide, you can use Bash scripts - just make sure to tell the system that they are, in fact, Bash scripts, by putting bash in the shebang line.
The echo command is probably the most notoriously inconsistent command across shells, by the way. So if you are going for portability, you're better off avoiding it entirely for anything other than straight up, no-options, no-escapes, always-a-newline output. One pretty portable option is printf.