I am reading a shell scripts from github :script
It has two lines of code confused me. I have never seen ## used in bash like this before.
could anyone explain this to me, how does it work? thanks.
branch_name=$(git symbolic-ref -q HEAD)
branch_name=${branch_name##refs/heads/}
Note:The first line produces something like 'refs/heads/master'
and the next line remove the leading refs/heads make the branch_name becomes master.
From the bash(1) man page, EXPANSION section, Parameter Expansion subsection:
${parameter#word}
${parameter##word}
Remove matching prefix pattern. The word is expanded to produce
a pattern just as in pathname expansion. If the pattern matches
the beginning of the value of parameter, then the result of the
expansion is the expanded value of parameter with the shortest
matching pattern (the ``#'' case) or the longest matching pat‐
tern (the ``##'' case) deleted.
Also available in the manual, of course (but it doesn't seem to support linking to this exact text; search the page for ##).
Have a look here where a lot other string manipulation tricks are described. In short
${string##substring}
Deletes longest match of $substring from front of $string.
Related
To rephrase - I want to use Bash command substitution and string substitution in the same line.
My actual commands are longer, but the ridiculous use of echo here is just a "substitution" for shortness and acts the same - with same errors ;)
I know we can use a Bash command to produce it's output string as a parameter for another command like this:
echo "$(echo "aahahah</ddd>")"
aahahah</ddd>
I also know we can remove last known part of a string like this:
var="aahahah</ddd>"; echo "${var%</ddd>}"
aahahah
I am trying to write a command where one command gives a string output, where I want to remove last part, which is known.
echo "${$(echo "aahahah</ddd>")%</ddd>}"
-bash: ${$(echo "aahahah</ddd>")%</ddd>}: bad substitution
It might be the order of things happening or substitution only works on variables or hardcoded strings. But I suspect just me missing something and it is possible.
How do I make it work?
Why doesn't it work?
When a dollar sign as in $word or equivalently ${word} is used, it asks for word's content. This is called parameter expansion, as per man bash.
You may write var="aahahah</ddd>"; echo "${var%</ddd>}": That expands var and performs a special suffix operation before returning the value.
However, you may not write echo "${$(echo "aahahah</ddd>")%</ddd>}" because there is nothing to expand once $(echo "aahahah</ddd>") is evaluated.
From man bash (my emphasis):
${parameter%word}
Remove matching suffix pattern. The word is expanded to produce a
pattern just as in pathname expansion. If the pattern
matches a trailing portion of the expanded value of parameter, then
the result of the expansion is the expanded value of parameter
with the shortest matching pattern (the ''%'' case) or the longest matching pattern (the ''%%'' case) deleted.
Combine your commands like this
var=$(echo "aahahah</ddd>")
echo ${var/'</ddd>'}
I know that .* means fetch all files regardless of the extensions (I hope I'm not wrong). However, I can't for the love of my life seem to figure out what does that extra % sign mean!
Here's two code snippets that might help describe the situation a bit more :
img_files=${img_files}' '$(ls ${TRAINING_DIR}/*.exp${exposure}.tif)
for img_file in ${img_files}; do
run_command tesseract ${img_file} ${img_file%.*} \
${box_config} ${config} &
For those who need even more details, here's the full script.
The expression ${img_file%.*} will remove the rightmost dot and any character after it in the variable img_file. From man bash:
${parameter%word}
${parameter%%word}
Remove matching suffix pattern. The word is expanded to produce
a pattern just as in pathname expansion. If the pattern matches
a trailing portion of the expanded value of parameter, then the
result of the expansion is the expanded value of parameter with
the shortest matching pattern
Example:
>var="word1 word2"
>echo ${var%word2}
word1
>echo ${var%word1}
word1 word2
% here means removal from right edge. For example
consider a variable img_file="racecar"
${img_file%c*} will return race.
${img_file%%c*} = ra
I have searched a lot, and while I see a couple of examples of these used, specifically from here:
scale=${scale##*[!0-9]*}
[ -z "${scale//[0-9]}" ]
There is no explanation for what these symbols do, how they work or when to use them scripting. I have not found them explained elsewhere when special symbols are discussed. Looks like they could be useful. Can anyone explain how the ## and // work in the script examples on the page linked above? Thanks.
They're part of shell parameter expansion syntax, used to modify the value of the variable. # and % are used to delete a prefix or suffix of the variable, and // is used to substitute one string for another.
${parameter#word}
${parameter##word}
The word is expanded to produce a pattern just as in filename expansion (see Filename Expansion). If the pattern matches the beginning of the expanded value of parameter, then the result of the expansion is the expanded value of parameter with the shortest matching pattern (the ‘#’ case) or the longest matching pattern (the ‘##’ case) deleted.
So ${scale##*[!0-9]*} means to remove the beginning of the string that matches anything followed by a non-digit followed by anything. So foobar becomes an empty string (because everything is removed), while 123 is left alone because [!0-9] never matches anything.
${parameter/pattern/string}
The pattern is expanded to produce a pattern just as in filename expansion. Parameter is expanded and the longest match of pattern against its value is replaced with string. If pattern begins with ‘/’, all matches of pattern are replaced with string. Normally only the first match is replaced. If pattern begins with ‘#’, it must match at the beginning of the expanded value of parameter. If pattern begins with ‘%’, it must match at the end of the expanded value of parameter. If string is null, matches of pattern are deleted and the / following pattern may be omitted.
So ${scale//[0-9]} simply removes all digits from the the value of the variable, then test -z is used to test if this is an empty string (meaning the original string only had digits).
From: http://tldp.org/LDP/abs/html/string-manipulation.html
${string##substring}
Deletes longest match of $substring from front of $string.
${string//substring/replacement}
Replace all matches of $substring with $replacement.
I have a Bash script that takes in a directory as a parameter, and after some processing will do some output based on the files in that directory.
The command would be like the following, where dir is a directory with the following structure inside
dir/foo
dir/bob
dir/haha
dir/bar
dir/sub-dir
dir/sub-dir/joe
> myscript ~/files/stuff/dir
After some processing, I'd like the output to be something like this
foo
bar
sub-dir/joe
The code I have to remove the path passed in is the following:
shopt -s extglob
for file in $files ; do
filename=${file#${1}?(/)}
This gets me to the following, but for some reason the optional / is not being taken care of. Thus, my output looks like this:
/foo
/bar
/sub-dir/joe
The reason I'm making it optional is because if the user runs the command
> myscript ~/files/stuff/dir/
I want it to still work. And, as it stands, if I run that command with the trailing slash, it outputs as desired.
So, why does my ?(/) not work? Based on everything I've read, that should be the right syntax, and I've tried a few other variations as well, all to no avail.
Thanks.
that other guy's helpful answer solves your immediate problem, but there are two things worth nothing:
enumerating filenames with an unquoted string variable (for file in $files) is ill-advised, as sjsam's helpful answer points out: it will break with filenames with embedded spaces and filenames that look like globs; as stated, storing filenames in an array is the robust choice.
there is no strict need to change global shell option shopt -s extglob: parameter expansions can be nested, so the following would work without changing shell options:
# Sample values:
file='dir/sub-dir/joe'
set -- 'dir/' # set $1; value 'dir' would have the same effect.
filename=${file#${1%/}} # -> '/sub-dir/joe'
The inner parameter expansion, ${1%/}, removes a trailing (%) / from $1, if any.
I suggested you change files to an array which is a possible workaround for non-standard filenames that may contain spaces.
files=("dir/A/B" "dir/B" "dir/C")
for filename in "${files[#]}"
do
echo ${filename##dir/} #replace dir/ with your param.
done
Output
A/B
B
C
Here's the documentation from man bash under "Parameter Expansion":
${parameter#word}
${parameter##word}
Remove matching prefix pattern. The word is
expanded to produce a pattern just as in pathname
expansion. If the pattern matches the beginning of
the value of parameter, then the result of the
expansion is the expanded value of parameter with
the shortest matching pattern (the ``#'' case) or
the longest matching pattern (the ``##'' case)
deleted.
Since # tries to delete the shortest match, it will never include any trailing optional parts.
You can just use ## instead:
filename=${file##${1}?(/)}
Depending on what your script does and how it works, you can also just rewrite it to cd to the directory to always work with paths relative to .
I am trying to become more familiar with using the builtin string matching stuff available in shells in linux. I came across this guys posting, and he showed an example
a="abc|def"
echo ${a#*|} # will yield "def"
echo ${a%|*} # will yield "abc"
I tried it out and it does what its advertised to do, but I don't understand what the $,{},#,*,| are doing, I tried looking for some reference online or in the manuals but I couldn't find anything. Can anyone explain to me what's going on here?
This article in the Linux Journal says that the # operator deletes the shortest possible match on the left, while the % operator deletes the shortest possible match on the right.
So ${a#*|} returns everything after the |, and ${a%|*} returns everything before the |.
If you had a situation that called for greedy matching, you'd use ## or %%.
Take a look at this.
${string%substring}
Deletes shortest match of $substring
from back of $string.
${string#substring}
Deletes shortest match of $substring
from front of $string.
EDIT:
I don't understand what the $,{},#,*,|
are doing
I recommend reading this
Typically, ${somename} will substitute the contents of a defined parameter:
mystring="1234567"
echo ${mystring} # produces '1234567'
The % and # symbols are allowing you to add commands that modify the default behavior.
The asterisk '*' is a wildcard; while the pipe '|' is simply a matching character. Let me do the same thing using the matching character of '4'.
mystring="1234567"
echo ${mystring#*4} # produces '567'
Those features and other similarly useful ones are documented in the Shell Parameter Expansion section of the Bash Reference Manual. Here's another really good reference.