For sh/bash/zsh there is https://github.com/koalaman/shellcheck however there won't be support for fish with it https://github.com/koalaman/shellcheck/issues/209 - is there any linters for fish?
To my knowledge, there is not (and obviously this is impossible to prove).
And if someone were to create such a thing, there'd need to be consensus about what the "typical beginner's syntax issues" and "semantic problems that cause a shell to behave strangely and counter-intuitively" are.
Fish doesn't have many of POSIX sh's warts (as it was written as a reaction to them). Some examples from the shellcheck README:
echo $1 # Unquoted variables
Fish's quoting behavior is quite different - in particular, there is no word splitting on variables, so unquoted variables usually do what you want.
v='--verbose="true"'; cmd $v # Literal quotes in variables
This is presumably an (unsuccessful) attempt to defeat word splitting, which isn't necessary.
This example nicely illustrates the issue - there are multiple decades worth of sh scripts. The flaws and unintuitive behaviors are really well known. So well known in fact, that the common-but-incorrect workarounds are known as well. That's just not the case for fish.
(Obviously, other examples do apply to fish as well, especially the "Frequently misused commands" section.)
Some things in fish that I know new users often trip over:
Unquoted variables expand to one argument per element in the list (since every variable is one). That includes zero if the list is empty, which is an issue with test - e.g. test -n $var will return 0 because fish's test builtin is one of the few parts that are POSIX-compatible (since POSIX demands test with one argument returns 0). Double-quote if you always need one argument.
{} expands to nothing and {x} expands to "x", which means find -exec needs quoting, as do some git commit-ishes (HEAD#{4}). (edit: This has since been changed, {} expands to {} and {x} expands to {x} unless x has a comma or other expansion, so HEAD#{4} works)
fish -n or --no-execute "does not execute any commands, only performs syntax checking", so you could do something like what I am doing here:
for f in **/*.fish; do fish -n "$f"; done
Related
Here is the code at the bash shell. How is the file mask supposed to be specified, if not this way? I expected both commands to find the search expression, but it's not happening. In this example, I know in advance that I prefer to restrict the search to python source code files only, because unqualified searches are silly time wasters.
So, this works as expected:
grep -rni '/home/ga/projects' -e 'def Pr(x,u,v)'
/home/ga/projects/anom/anom.py:27:def Pr(x,u,v): blah, blah, ...
but this won't work:
grep --include=\*.{py} -rni '/home/ga/projects' -e 'def Pr(x,u,v)'
I'm using GNU grep version 2.16.
--include=\*.{py} looks like a broken attempt to use brace expansion (an unquoted {...} expression).
However, for brace expansion
to occur in bash (and ksh and zsh), you must either have:
a list of at least 2 items, separated with ,; e.g. {py,txt}, which expands to 2 arguments, py and txt.
or, a range of items formed from two end points, separated with ..; e.g., {1..3}, which expands to 3 arguments, 1, 2, and 3.
Thus, with a single item, simply do not use brace expansion:
--include=\*.py
If you did have multiple extensions to consider, e.g., *.py as well as *.pyc files, here's a robust form that illustrates the underlying shell features:
'--include=*.'{py,pyc}
Here:
Brace expansion is applied, because {...} contains a 2-item list.
Since the {...} directly follows the literal (single-quoted) string --include=*., the results of the brace expansion include the literal part.
Therefore, 2 arguments are ultimately passed to grep, with the following literal content:
--include=*.py
--include=*.pyc
Your command fails because of the braces '{}'. It will search for it in the file name. You can create a file such as 'myscript.{py}' to convince yourself. You'll see it will appear in the results.
The correct option parameter would be '*.py' or the equivalent \*.py. Either way will protect it from being (mis)interpreted by the shell.
On the other side, I can only advise to use the command find for such jobs :
find /home/ga/projects -regex '.*\.py$' -exec grep -e "def Pr(x,u,v)" {} +
That will protect you from hard to understand shell behaviour.
Try like this (using quotes to be safe; also better readability than backslash escaping IMHO):
grep --include='*.py' ...
your \*.{py} brace expansion usage isn't supported at all by grep. Please see the comments below for the full investigation regarding this. For the record, blame this answer for the resulting brace wars ;)
By the way, the brace expansion works generally fine in Bash. See mklement0 answer for more details.
Ack. As an alternative, you might consider switching to ack instead from now on. It's a tool just like grep, but fully optimized for programmers.
It's a great fit for what you are doing. A nice quote about it:
Every once in a while something comes along that improves an idea so much, you can't ignore it. Such a thing is ack, the grep replacement.
According to POSIX:
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sh.html
there are some cases where it not obvious. For example:
If the file is not in the current working directory,
the implementation may perform a search for an executable
file using the value of PATH, as described in Command Search and Execution.
My Bash 4.x doesn't follow this optional rule (due to security concern??) so I can't test how it be in real life...
What platform independent way to find directory of shell executable in shell script?
PS. Also dirname $0 case fail with:
#!/bin/sh
echo $0
dirname $0
when you:
$ sh runme.sh
runme.sh
.
So you need something like:
CMDPATH=`cd $(dirname $0); echo $PWD`
To made code dependent only on built-in shell capabilities I rewrite code to:
PREVPWD=$PWD
cd ${0%${0##*/}}.
CMDPATH=$PWD
cd $PREVPWD
This look ugly but doesn't require fork any executables...
EDIT3:
Though not strictly POSIX yet, realpath is a GNU core app since 2012. Full disclosure: never heard of it before I noticed it in the info coreutils TOC and immediately thought of this question, but using the following function as demonstrated should reliably, (soon POSIXLY?), and, I hope, efficiently
provide its caller with an absolutely sourced $0:
% _abs_0() {
> o1="${1%%/*}"; ${o1:="${1}"}; ${o1:=`realpath -s "${1}"`}; eval "$1=\${o1}";
> }
% _abs_0 ${abs0:="${0}"} ; printf %s\\n "${abs0}"
/no/more/dots/in/your/path2.sh
EDIT4: It may be worth highlighting that this solution uses POSIX parameter expansion to first check if the path actually needs expanding and resolving at all before attempting to do so. This should return an absolutely sourced $0via a messenger variable (with the notable exception that -s will preserve symlinks) as efficiently as I could imagine it could be done whether or not the path is already absolute.
EDIT2:
Now I believe I understand your problem much better which, unfortunately, actually renders most of the below irrelevant.
(minor edit: before finding realpath in the docs, I had at least pared down my version of this not to depend on the time field, but, fair warning, after testing some I'm less convinced ps is fully reliable in its command path expansion capacity)
On the other hand, you could do this:
ps ww -fp $$ | grep -Eo '/[^:]*'"${0#*/}"
eval "abs0=${`ps ww -fp $$ | grep -Eo ' /'`#?}"
I need to fix it to work better with fields instead of expecting the time field to come just before the process's path and relying on its included colon as a reference, especially because this will not work with a colon in your process's path, but that's trivial and will happen soon, I think. The functionality is otherwise POSIX compliant, I believe. Probably parameter expansion alone can do what is necessary, I think.
Not strictly relevant (or correct):
This should work in every case that conforms to POSIX guidelines:
echo ${0%/*}
EDIT:
So I'll confess that, at least at first blush, I don't fully understand the issue you describe. Obviously in your question you demonstrate some familiarity with POSIX standards for variable string manipulation via parameter expansion (even if your particular implementation seems slightly strained at a glance), so it's likely I'm missing some vital piece of information in my interpretation of your question and perhaps, at least in its current form, this is not the answer you seek.
I have posted before on parameter expansion for inline variable null/set tests which may or may not be of use to you as you can see at the "Portable Way to Check Emptiness of a Shell Variable" question. I mention this mainly because my answer there was in large part copied/pasted from the POSIX guidelines on parameter expansion, includes an anchored link to the guidelines coverage on this subject, and a few examples from both the canonical documentation and my own perhaps less expertly demonstrated constructs.
I will freely admit however, that while I do not yet fully understand what it is you ask, I don't believe that you will find a specific answer there. Instead I suspect you may have forgotten, as I do occasionally, that the # and % operators in POSIX string manipulation are used to specify the part of the string you want to remove, not that you wish to retain as some might find more intuitive. What I mean is any string slice you search for in this way is designed to disappear from your output, which will then be only what your remains of your original string after your specified search string is removed.
So here's a bit of an overview:
Whereas a single instance of either operator will remove only as little as possible to fully satisfy your search, but when doubly instanced the search is called in a greedy form and removes as much of the original string as your search could possibly allow.
Other than that you need only know some basic regex and remember that # begins its search for your removal string from the left and scans through to the right, and that % begins instead its search from the right and scans through to the left.
## short example before better learning if I'm on the right track
## demonstrating path manipulation with '#' and '%'
% _path_one='/one/two/three/four.five'
% _path_two='./four.five'
## short searching from the right with our wildcard to the right
## side of a single character removes everything to the right of
## of the specified character and the character itself
## this is a very simple means of stripping extensions and paths
% echo ${_path_one%.*} ${_path_one%/*}
/one/two/three/four /one/two/three
## long searching from the left with the wildcard to the left of
## course produces opposite results
% echo ${_path_one##*.} ${_path_one##*/}
five four.five
## will soon come back to show more probably
I believe you can get it using readlink:
scriptPath=$(readlink -f -- "$0")
scriptDirectory=${scriptPath%/*}
I just found out about the bash extglob shell option here:-
How can I use inverse or negative wildcards when pattern matching in a unix/linux shell?
All the answers that used shopt -s extglob also mentioned shopt -u extglob to turn it off.
Why would I want to turn something so useful off? Indeed why isn't it on by default?
Presumably it has the potential for giving some nasty surprises.
What are they?
No nasty surprises -- default-off behavior is only there for compatibility with traditional, standards-compliant pattern syntax.
Which is to say: It's possible (albeit unlikely) that someone writing fo+(o).* actually intended the + and the parenthesis to be treated as literal parts of the pattern matched by their code. For bash to interpret this expression in a different manner than what the POSIX sh specification calls for would be to break compatibility, which is right now done by default in very few cases (echo -e with xpg_echo unset being the only one that comes immediately to mind).
This is different from the usual case where bash extensions are extending behavior undefined by the POSIX standard -- cases where a baseline POSIX shell would typically throw an error, but bash instead offers some new and different explicitly documented behavior -- because the need to treat these characters as matching themselves is defined by POSIX.
To quote the relevant part of the specification, with emphasis added:
An ordinary character is a pattern that shall match itself. It can be any character in the supported character set except for NUL, those special shell characters in Quoting that require quoting, and the following three special pattern characters. Matching shall be based on the bit pattern used for encoding the character, not on the graphic representation of the character. If any character (ordinary, shell special, or pattern special) is quoted, that pattern shall match the character itself. The shell special characters always require quoting.
When unquoted and outside a bracket expression, the following three characters shall have special meaning in the specification of patterns:
? - A question-mark is a pattern that shall match any character.
* - An asterisk is a pattern that shall match multiple characters, as described in Patterns Matching Multiple Characters.
[ - The open bracket shall introduce a pattern bracket expression.
Thus, the standard explicitly requires any non-NUL character other than ?, * or [ or those listed elsewhere as requiring quoting to match themselves. Bash's behavior of having extglob off by default allows it to conform with this standard in its default configuration.
However, for your own scripts and your own interactive shell, unless you're making a habit of running code written for POSIX sh with unusual patterns included, enabling extglob is typically worth doing.
Being a Kornshell person, I have extglob on in my .bashrc by default because that's the way it is in Kornshell, and I use it a lot.
For example:
$ find !(target) -name "*.xml"
In Kornshell, this is no problem. In BASH, I need to set extglob. I also set lithist and set -o vi. This allows me to use VI commands in using my shell history, and when I hit v, it shows my code as a bunch of lines.
Without lithist set:
for i in *;do;echo "I see $i";done
With listhist set:
for i in *
do
echo "I see $i"
done
Now, only if BASH had the print statement, I'd be all set.
In my bash scripts, I regularly use file paths which may contain spaces:
FOO=/path\ with\ spaces/
Later, if I want to use FOO, I have to wrap it in quotes ("$FOO") or it will be interpreted as a list (/path, with, spaces/). Is there a better way to force a variable never to be interpreted as a list? It is cumbersome to have to constantly quote-wrap.
No. You must always use quotes or bash will word-split (except in [[, but that is a special case).
You can also change the internal field separator, IFS, as in:
ORIGIFS="$IFS"
IFS=$(echo -en "\n\b")
# do stuff...
IFS="$ORIGIFS"
However, this affects all situations where bash looks to do field splitting, which might be more broad than you'd like.
When using a POSIX shell, the following
touch {quick,man,strong}ly
expands to
touch quickly manly strongly
Which will touch the files quickly, manly, and strongly, but is it possible to dynamically create the expansion? For example, the following illustrates what I want to do, but does not work because of the order of expansion:
TEST=quick,man,strong #possibly output from a program
echo {$TEST}ly
Is there any way to achieve this? I do not mind constricting myself to Bash if need be. I would also like to avoid loops. The expansion should be given as complete arguments to any arbitrary program (i.e. the program cannot be called once for each file, it can only be called once for all files). I know about xargs but I'm hoping it can all be done from the shell somehow.
... There is so much wrong with using eval. What you're asking is only possible with eval, BUT what you might want is easily possible without having to resort to bash bug-central.
Use arrays! Whenever you need to keep multiple items in one datatype, you need (or, should use) an array.
TEST=(quick man strong)
touch "${TEST[#]/%/ly}"
That does exactly what you want without the thousand bugs and security issues introduced and concealed in the other suggestions here.
The way it works is:
"${foo[#]}": Expands the array named foo by expanding each of its elements, properly quoted. Don't forget the quotes!
${foo/a/b}: This is a type of parameter expansion that replaces the first a in foo's expansion by a b. In this type of expansion you can use % to signify the end of the expanded value, sort of like $ in regular expressions.
Put all that together and "${foo[#]/%/ly}" will expand each element of foo, properly quote it as a separate argument, and replace each element's end by ly.
In bash, you can do this:
#!/bin/bash
TEST=quick,man,strong
eval echo $(echo {$TEST}ly)
#eval touch $(echo {$TEST}ly)
That last line is commented out but will touch the specified files.
Zsh can easily do that:
TEST=quick,man,strong
print ${(s:,:)^TEST}ly
Variable content is splitted at commas, then each element is distributed to the string around the braces:
quickly manly strongly
Taking inspiration from the answers above:
$ TEST=quick,man,strong
$ touch $(eval echo {$TEST}ly)