I need to extract part of the string before the last opt in the string:
NDS=/opt/novell/opt/eDirectory/opt/abc
I want just the /opt/novell/opt/eDirectory/ part
I used NDSHOME=${NDS%opt*}
but I am getting bad substitution on Solaris 10, that is using the Bourne shell,
although this script works fine on Linux and AIX.
Can somebody find a solution to this??
Solaris' /bin/sh is notoriously not Posix compatible.
You could try the following workaround with sed
NDSHOME=`echo "$NDS" | sed 's/^\(.*\)\<opt\>.*$/\1/'`
Note: Normally I would use $() instead of backtics for command substitution, but I don't think those work on Solaris either.
Edit
Changed it so that opt will not match intra-word boundaries
$ echo $NDS
/opt/novell/opt/eDirectory/opt/helicopter
$ echo "$NDS" | sed 's/^\(.*\)\<opt\>.*$/\1/'
/opt/novell/opt/eDirectory/
The dirname command prints all but the last level of the path name given as an argument.
$ NDS=/opt/novell/opt/eDirectory/opt/abc
$ NDSHOME=`dirname $NDS`
$ echo $NDSHOME
/opt/novell/opt/eDirectory/opt
(Add export as needed.)
EDIT :
The above doesn't work in this case. The OP wants to remove all components of the path following the last opt, not just the last component. I'll leave it here in case it's useful to someone else.
You could use dirname in a loop, stopping when the last component (as determined by basename) is `opt':
NDSHOME="$NDS"
while [ "`basename $NDSHOME`" != opt ] ; do
NDSHOME="`dirname $NDSHOME`"
done
but that's more complex than it needs to be; sed is a better solution.
SiegeX's answer is good, but it treats opt as a string, not just as a path component. For example, it will turn
/opt/novell/opt/eDirectory/opt/helicopter
into
/opt/novell/opt/eDirectory/opt/helic
Try this:
NDSHOME=`echo "$NDS" | sed 's,\(.*\)/opt/.*$,\1,'`
(Note the use of , rather than / as a delimiter, so the / characters in the pattern don't have to be escaped.)
Solaris is definitely POSIX (and several related standards like xpg4 and susv3) compliant but it also takes compatibility with older versions quite seriously.
The side effect is that when both these features conflict in some way, the affected commands are, by default, not POSIX compliant not to break legacy scripts.
Should you want POSIX compatibility and have no legacy script issues, just set your PATH like this:
PATH=/usr/xpg6/bin:/usr/xpg4/bin:/usr/ccs/bin:$PATH
and set your login shell to /usr/xpg4/bin/sh
Alternatively, if you do not need strict compliance, you might just use /bin/ksh as your default shell.
With Solaris 10 and older, you shouldn't really use /bin/sh at all in new scripts.
For reference, here is the Solaris 10 standards manual page and the one for Solaris 11 is here.
Use the Bourne-Again shell instead.
Related
I have been working on a few scripts on CentOS 7 and sometimes I see:
#!/bin/sh -
on the first line. Looking at the man page for sh I see the following under the Special Parameters
- Expands to the current option flags as specified upon invocation,
by the set builtin command, or those set by the shell
itself (such as the -i option).
What exactly does this mean? When do I need to use this special parameter option??
The documentation you are reading has nothing to do with the command line you're looking at: it's referring to special variables. In this case, if you run echo $- you will see "the current option flags as specified upon invocation...".
If you take a look at the OPTIONS part of the bash man page, you will find:
-- A -- signals the end of options and disables further option processing.
Any arguments after the -- are treated as filenames and arguments. An
argument of - is equivalent to --.
In other words, an argument of - simply means "there are no other options after this argument".
You often see this used in situation in which you want to avoid filenames starting with - accidentally being treated as command options: for example, if there is a file named -R in your current directory, running ls * will in fact behave as ls -R and produce a recursive listing, while ls -- * will not treat the -R file specially.
The single dash when used in the #! line is meant as a security precaution. You can read more about that here.
/bin/sh is an executable representing the system shell. Actually, it is usually implemented as a symbolic link pointing to the executable for whichever shell is the system shell. The system shell is kind of the default shell that system scripts should use. In Linux distributions, for a long time this was usually a symbolic link to bash, so much so that it has become somewhat of a convention to always link /bin/sh to bash or a bash-compatible shell. However, in the last couple of years Debian (and Ubuntu) decided to switch the system shell from bash to dash - a similar shell - breaking with a long tradition in Linux (well, GNU) of using bash for /bin/sh. Dash is seen as a lighter, and much faster, shell which can be beneficial to boot speed (and other things that require a lot of shell scripts, like package installation scripts).
Dash is fairly well compatible with bash, being based on the same POSIX standard. However, it doesn't implement the bash-specific extensions. There are scripts in existence that use #!/bin/sh (the system shell) as their shebang, but which require bash-specific extensions. This is currently considered a bug that should be fixed by Debian and Ubuntu, who require /bin/sh to be able to work when pointed to dash.
Even though Ubuntu's system shell is pointing to dash, your login shell as a user continues to be bash at this time. That is, when you log in to a terminal emulator anywhere in Linux, your login shell will be bash. Speed of operation is not so much a problem when the shell is used interactively, and users are familiar with bash (and may have bash-specific customization in their home directory).
I was writing some code, navigating my computer (OSX 10.11.6) via the command line, like I always do, and I made a typo! Instead of typing:
cd USB
I typed
Cd USB
Nothing happened, but it didn't register as an invalid command. Perplexed by this, I did some investigating: I checked the man entry. There was no entry. I found the source file (/usr/bin/Cd) using which Cd, and then cated it:
#!/bin/sh
# $FreeBSD: src/usr.bin/alias/generic.sh,v 1.2 2005/10/24 22:32:19 cperciva Exp $
# This file is in the public domain.
builtin `echo ${0##*/} | tr \[:upper:] \[:lower:]` ${1+"$#"}
What is this, and why is it here? How does it relate to freeBSD?
Any help would be amazing, thanks!
macOS uses a case-insensitive filesystem by default[1]
, which can be misleading at times:
which Cd is effectively the same as which cd and which CD in terms of returning the (effectively) same file path.
Confusingly, even though all 3 command refer to the same file, they do so in a case-preserving manner, misleadingly suggesting that the actual case of the filename is whatever you specified.
As a workaround, you can see the true case of the filename if you employ globbing (filename expansion):
$ ls "$(which Cd)"* # could match additional files, but the one of interest is among them
/usr/bin/cd # true case of the filename
Bash (the macOS default shell) is internally case-sensitive.
That is, it recognizes cd as builtin cd (its built-in directory-changing command).
By contrast, it does NOT recognize Cd as that, due to the difference in case.
Given that it doesn't recognize Cd as a builtin, it goes looking for an external utility (in the $PATH), and that is when it finds /usr/bin/cd.
/usr/bin/cd is implemented as a shell script, which is mostly useless, because as an external utility it cannot affect the shell's state, so its attempts to change the directory are simply quietly ignored.
(Keith Thompson points out in a comment that you can use it as test whether a given directory can be changed to, because the script's exit code will reflect that).
Matt's answer provides history behind the inclusion of the script in FreeBSD and OSX (which mostly builds on FreeBSD), but it's worth taking a closer look at the rationale (emphasis mine):
From the POSIX spec:
However, all of the standard utilities, including the regular built-ins in the table, but not the special built-ins described in Special Built-In Utilities, shall be implemented in a manner so that they can be accessed via the exec family of functions as defined in the System Interfaces volume of POSIX.1-2008 and can be invoked directly by those standard utilities that require it (env, find, nice, nohup, time, xargs).
In essence, the above means: regular built-ins must (also) be callable stand-alone, as executables (whether as scripts or binaries), nut just as built-ins from within the shell.
The cited regular built-ins table comprises these utilities:
alias bg cd command false fc fg getopts jobs kill newgrp pwd read true umask unalias wait
Note: special built-in utilities are by definition shell-internal only, and their behavior differs from regular built-in utilities.
As such, to be formally POSIX-compliant an OS must indeed provide cd as an external utility.
At the same time, the POSIX spec. does have awareness that at least some of these regular built-ins - notably cd - only makes sense as a built-in:
"Since cd affects the current shell execution environment, it is always provided as a shell regular built-in." - http://pubs.opengroup.org/onlinepubs/9699919799/utilities/cd.html
Among the regular built-in utilities listed, some make sense both as a built-in and as an external utility:
For instance kill needs to be a built-in in order to kill jobs (which are a shell-internal concept), but it is also useful as an external utility, so as to kill processes by PID.
However, among the regular built-in utilities listed, the following never make sense as external utilities, as far as I can tell Do tell me if you disagree
, even though POSIX mandates their presence:
alias bg cd command fc fg getopts jobs read umask unalias
Tip of the hat to Matt for helping to complete the list; he also points that the hash built-in, even though it's not a POSIX utility, also has a pointless script implementation.
[1] As Dave Newton points out in a comment, it is possible to format HFS+, the macOS filesystem, in a case-sensitive manner (even though most people stick with the case-insensitive default). Based on the answer Dave links to, the following command will tell you whether your macOS filesystem is case-insensitive or not:
diskutil info / | grep -iq '^\s*Name.*case-sensitive*' && echo "case-SENSITIVE" || echo "case-INsensitive"
What is this?
The script itself is a portable way to convert a command, even with random upper casing, into the equivalent shell builtin based on the exec paths file name, that is any part of the string after the final / in the $0 variable). The script then runs the builtin command with the same arguments.
As OSX file systems are case insensitive by default, /usr/bin/cd converts running Cd, CD, cD and any form of cd with a / fs path (like /usr/bin/cd) back to the shell builtin command cd. This is largely useless in a script as cd only affects the current shell it is running in, which immediately closes when the script ends.
How does it relate to freeBSD?
A similar file exists in FreeBSD, which Apple adapted to do case conversion. Mac file systems by default are case insensitive (but case preserving).
The $FreeBSD: src/usr.bin/alias/generic.sh,v 1.2 2005/10/24 22:32:19 cperciva Exp $ header is the source information in the file.
Most of the underlying OSX system comes directly from FreeBSD or was based on it. The Windowing system on top of this and the Cocoa app layer is where OSX becomes truly Apple. Some of the lower level Apple bits have even made it back into FreeBSD like Clang and LLVM compiler.
Why is it here?
The earlier FreeBSD svn commits shed a bit of light:
A little bit more thought has resulted in a generic script which can
implement any of the useless POSIX-required ``regular shell builtin''
utilities...
Although most builtins aren't very useful when run in a new shell via a script, this compliance script was used for the commands alias bg cd command fc fg getopts hash jobs read type ulimit umask unalias wait. POSIX compliance is fun!
As I recall, MacOS uses a case-insensitive file system by default. The command you saw as /usr/bin/Cd is actually /usr/bin/cd, but it can be referred to by either name.
You can see this by typing
ls /usr/bin/ | grep -i cd
Normally cd is a builtin command in the shell. As you know, it changes the current directory. An external cd command is nearly useless -- but it still exists.
It can be used to detect whether it's possible to change to a specified directory without actually affecting the working directory of your current process.
Your shell (probably bash) tends to assume case-sensitive command names. The builtin command can only be referred to as cd, but since it's able to open the script file named /usr/bin/Cd, it can find and execute it.
I'm trying to get last executed command from command line from a script to be saved for a later reference:
Example:
# echo "Hello World!!!"
> Hello World!!!
# my_script.sh
> echo "Hello World!!!"
and the content of the script would be :
#!/usr/bin/ksh
fc -nl -1 | sed -n 1p
Now as you notices using here ksh and fc is a built in command which if understood correctly should be implemented by any POSIX compatible shells. [I understand that this feature is interactive and that calling same fc again will give different result but this is not the concern do discuss about]
Above works so far so good only if my_script.sh is being called from the shell which is as well ksh, or if calling from bash and changing 1st line of script as #!/bin/bash then it works too and it doesn't if shells are different.
I would like to know if there is any good way to achieve above without being constrained by the shell your script is called from. I understand that fc is a built in command and it works per shell thus most probably my approach is not good at all from what I want to achieve. Any better suggestions?
I actually attempted this, but it cannot be done between different shells consistently.
While
fc -l`,
is the POSIX standard command for showing $SHELL history, implementation details may be different.
At least bash and ksh93 both will report the last command with
fc -n -l -1 -1
However, POSIX does not guarantee that shell history will be carried over to a new instance of the shell, as this requires the presence of a $HISTFILE. If none is
present, the shell may default to $HOME/.sh_history.
However, this history file or Command History List is not portable between different shells.
The POSIX Shell description of the
Command History List says:
When the sh utility is being used interactively, it shall maintain a list of commands
previously entered from the terminal in the file named by the HISTFILE environment
variable. The type, size, and internal format of this file are unspecified.
Emphasis mine
What this means is that for your script needs to know which shell wrote that history.
I tried to use $SHELL -c 'fc -nl -1 -1', but this did not appear to work when $SHELL refers to bash. Calling ksh -c ... from bash actually worked.
The only way I could get this to work is by creating a function.
last_command() { (fc -n -l -1 -1); }
In both ksh and bash, this will give the expected result. Variations of this function can be used to write the result elsewhere. However, it will break whenever it's called
from a different process than the current.
The best you can do is to create these functions and source them into your
interactive shell.
fc is designed to be used interactively. I tested your example on cygwin/bash and the result was different. Even with bash everywhere the fc command didn't work in my case.
I think fc displays the last command of the current shell (here I don't speak about the shell interpretor, but shell as the "process box". So the question is more why it works for you.
I don't think there is a clean way to achieve what you want because (maybe I miss something) you want two different process (bash and your magic command [my_script.sh]) and by default OS ensure isolation between them.
You can rely on what you observe (not portable, depends on the shell interpretor etc.)
You cannot rely on BASH historic because it's in-memory (the file is updated only on exit).
You can use an alias or a function (edited: #Charles Duffy is right). In this case you won't be able to use your "magic command" from another terminal, but for an interactive use it does the job.
Edited:
Or you can provide two commands: one to save and another to look for. In this case you manage your own historic but you have to save explicitly each command that is painful...
So I look for a hook. And I found this other thread : https://superuser.com/questions/175799/does-bash-have-a-hook-that-is-run-before-executing-a-command
# At the beginning of the Shell (.bashrc for example)
save(){ history 1 >>"$HOME"/myHistory ; }
trap 'save' DEBUG
# An example of use
rm -f "$HOME"/myHistory
echo "1 2 3"
cat "$HOME"/myHistory
14 echo "1 2 3"
15 cat "$HOME"/myHistory
But I observe it slows down the interpretor...
Little convoluted, but I was able to use this command to get the most recent command in zsh, bash, ksh, and tcsh on Linux:
history | tail -2 | head -1 | sed -r 's/^[ \t]*[0-9]*[ \t]+([^ \t].*$)/\1/'
Caveats: this uses GNU sed, so you'll need to install that if you're using BSD, OS X, etc; and also, tcsh will display the time of the command before the command itself. Regular csh doesn't seem to having a functioning history command when I tried it.
I'm writing a bash script which uses parameter expansion like these a few times throughout:
${pathvariable%/*}
${pathvariable##*/}
I'll need to run this script on some boxes which may have older versions of bash installed. I know it works in bash 4.1+ but I'm wondering how portable it is?
Is this kind of construction going to fail in older versions of bash? If so, which versions?
Is there a more portable construct I should consider?
They are defined in POSIX (see here) so you should be safe.
EDIT: In case you are paranoid and don't trust in bash's compliance:
$ ./bash --posix
$ echo ${BASH_VERSION}
2.05a.0(1)-release
$ foo="foo/bar"
$ echo "${foo%/*}"
foo
$ echo "${foo##*/}"
bar
2.05a was released in ~2001.
This should not fail in any version of bash since its really a korn shell construct. As long as your sure you will be using bash (any version) your fine.
I was writing a script when I decided to move the functions to a lib file, but when I open the lib file all the $( and the consecutive ) are red highlighted, here are some examples of the script
TAB="$(printf '\t')"
percent=$(echo "scale=2; $number/$total*100" | bc | sed -e 's/\.[[:digit:]]*//g')
if [[ -z $(grep $site/post $max_lim) ]];then
The filetype is conf but I've set it as sh syntax in .vimrc
Any idea of what is happenning?
Thank you
Edit: Thanks for the quick answers, I found that this line makes vim match the files with the extension specified behind the * with the syntax sh
au BufReadPost * set syntax=sh
I've also thought that using shebang in the libraries was not allowed, but is a nice solution
Anyway using g:is_bash in .vimrc returns an error of pattern not found
So what I would like to do is as I only write in bash, to vim recognize any file without extension as bash
The syntax file for sh actually handles several different kinds of shell syntax: bash, ksh, and plain old sh. Since your conf file isn't recognized as bash or ksh, it falls back to sh. $(...) isn't a valid construct in sh, so it is highlighted as an error.
To fix this, you can make sure "g:is_bash" is set for the file, so that the sh syntax script will know your file should be highlighted as bash code. Please edit your question to include what you added to your .vimrc to make the file use sh syntax highlighting. This will make it easier to suggest the correct way of setting "g:is_bash".
UPDATE: As Alok commented, you should be able to add the following to the file
#!/bin/bash
to let vim know the correct syntax highlighting to use as well.
In my case, I wanted to preserve #!/bin/sh as the shebang line because not every system has /bin/bash available.
Whereas the original Bourne shell may have not supported the $(...) syntax, most sh shells nowadays are POSIX-compliant, and the POSIX spec supports this syntax. For example,
On Ubuntu, /bin/sh is /bin/dash.
On MacOS, /bin/sh is /bin/bash.
On Alpine, /bin/sh is /bin/ash.
All of which satisfy the POSIX spec. Traditionally, if we'd like to write portable Shell, we should leave the shebang line as #!/bin/sh. We shouldn't change it to #!/bin/bash just for syntax highlighting if we're not going to use any Bashisms.
Okay, but what about the erroneous red highlighting? The problem is with Vim interpreting #!/bin/sh as a reference to the original Bourne shell from 1979 with no support for $(...). Maybe this is a testament to Vim's backwards compatibility, or maybe not enough people care. Here's a related GitHub issue describing the same behavior.
In any case, the best solution for me was to set let g:is_posix = 1 in my config. Interestingly, if you look through Vim's runtime files, it's equivalent to setting let g:is_kornshell = 1.
A brief interesting history on how the Bourne shell was bourne, bourne again as bash as a substitute for /bin/sh on Ubuntu, and eventually replaced in favor of dash can be found at https://askubuntu.com/a/976504.