What does a caret do when appended to a bash variable but within braces? I'm trying to decipher this within a bash script:
readonly TEST=${USER^}
When I don't know the meaning of some syntax in bash/sh I use my browsers find function in bash's manual and sh's specification. This is pretty effective as both contain the entire manual in a single page.
From bash's manual:
${parameter^pattern}
[...]
The ‘^’ operator converts lowercase letters matching pattern to uppercase
[...]
If pattern is omitted, it is treated like a ‘?’, which matches every character.
So ${variable^} expands to the value of $variable with the first letter converted to its uppercase variant.
Related
this is my first post here (maybe more later ^^)
So, here we are :
I'm using Gitlab-CI, docker, makefile, firebase.
So in my Makefile :
firebase-deploy: ## Deploy the application
$(FIREBASE) deploy -m "$(comment)"
in my CI :
variables:
DOCKER_DRIVER: overlay2
MESSAGE: $CI_COMMIT_MESSAGE
deploy-firebase:
stage: deploy
script:
- make install
- make deploy comment="$MESSAGE"
The problem is actually, i have
docker-compose exec -T js node_modules/firebase-tools/lib/bin/firebase.js deploy -m "test ci ?
/bin/sh: syntax error: unterminated quoted string
So, as you can see, the double quote at the end is ... not here.
I tested a few things
\"Comment\"
\'Comment\'
"Comment\"
And more, but none of them work.
One thing interesting is that if i commit FROM gitlab.com it work as expected, but from my computer on push : nop.
Do you have any advice, or something to look for ?
Thanks!
If your commit message contains end of lines you cannot pass it to your script as it is. Try, maybe, to substitute end of lines by another character, e.g. -. Of course the way to do it depends on the shell that executes your CI scripts. If it is bash, for instance, the following should work:
make deploy comment="${MESSAGE//$'\n'/-}"
If you must also escape other characters you can chain substitutions. Example if you want to substitute end of lines and double quotes by - and ':
c="${MESSAGE//$'\n'/-}"; c="${c//\"/\'}"; make deploy comment="$c"
To learn more about these bash substitutions just type man bash and read section EXPANSION, sub-section Parameter Expansion. The parameter expansion I used is:
${parameter/pattern/string}
that replaces pattern by string in the value of variable parameter (in bash parlance parameter is another name for variable). When the pattern begins with a / all matches of pattern are replaced with string, else only the first match is replaced. So the first parameter expansion:
c="${MESSAGE//$'\n'/-}"
substitutes all end-of-line characters in the value of variable MESSAGE by - (more on $'\n' later) and assigns the result to variable c. Note the enclosing double quotes, they are needed because you can have spaces in the message. And the second:
c="${c//\"/\'}"
substitutes all " in the value of variable c by ' and assigns again the result to variable c.
In this case the most complicated is to put a end-of-line character in the first pattern, a double quote in the second and a single quotes in the second string. The $'string' construct is explained in the QUOTING section of the bash manual:
Words of the form $'string' are treated specially. The word expands to
string, with backslash-escaped characters replaced as specified by the
ANSI C standard.
So $'\n' is replaced by the end-of-line character. In the second pattern and string you just need to escape the double and single quotes with a backslash to have them interpreted literally.
Finally the 3 commands of this little shell script are chained by ;, the bash sequence operator. They are executed one after the other in the same bash invocation.
I simply want a function (or just a 1-liner) to find/replace a string inside a variable, and not worry if the variables contain crazy characters.
Pseudo-code:
findReplace () {
#what goes here?
}
myLongVar="some long \crazy/ text my_placeholder bla"
replace="my_placeholder"
replaceWith="I like hamburgers/fries"
myFinalVar=$(findReplace $myLongVar $replace $replaceWith)
All similar questions seem complicated and use files
You can define the function like this:
findReplace1() {
printf "%s" "${1/"$2"/$3}"
}
And then run it like this:
myFinalVar=$(findReplace "$myLongVar" "$replace" "$replaceWith")
Note the double-quotes -- they're very important, because without them bash will split the variables' values into separate words (e.g. "some long \crazy/ text..." -> "some" "long" "\crazy/" "text...") and also try to expand anything that looks like a wildcard into a list of matching filenames. It's ok to leave them off on the right side of an assignment (myFinalVar=...), but that's one of the few places where it's ok. Also, note that within the function I put double-quotes around $2 -- in that case again it's to keep it from being treated as a wildcard pattern, but here it'd a string-match wildcard rather than filenames. Oh, and I used printf "%s" instead of echo because some versions of echo do weird things with strings that contain backslashes and/or start with "-".
And, of course, you can just skip the function and do the replacement directly:
myFinalVar=${myLongVar/"$replace"/$replaceWith}
Try:
myFinalVar=${myLongVar/$replace/$replaceWith}
If your want to replace all occurrences of $replace, not just the first, use:
myFinalVar=${myLongVar//$replace/$replaceWith}
Documentation
From man bash:
${parameter/pattern/string}
Pattern substitution. The pattern is expanded to produce a pattern
just as in pathname expansion. Parameter is expanded and the longest
match of pattern against its value is replaced with
string. If pattern begins with /, all matches of pattern are
replaced with string. Normally only the first match is replaced. If
pattern begins with #, it must match at the beginning of
the expanded value of parameter. If pattern begins with %, it must
match at the end of the expanded value of parameter. If string is
null, matches of pattern are deleted and the / following pattern may
be omitted. If the nocasematch shell option is enabled, the match is
performed without regard to the case of alphabetic
characters. If parameter is # or *, the substitution operation is
applied to each positional parameter in turn, and the
expansion is the resultant list. If parameter is an array variable
subscripted with # or *, the substitution operation is applied to each
member of the array in turn, and the expansion is the
resultant list.
This question already has answers here:
What is the meaning of the ${0##...} syntax with variable, braces and hash character in bash?
(4 answers)
Closed 2 years ago.
While looking online on how to get a file's extension and name, I found:
filename=$(basename "$fullfile")
extension="${filename##*.}"
filename="${filename%.*}
What is the ${} syntax...? I know regular expressions but "${filename##*.}" and "${filename%.*} escape my understanding.
Also, what's the difference between:
filename=$(basename "$fullfile")
And
filename=`basename "$fullfile"`
...?
Looking in Google is a nightmare, because of the strange characters...
The ${filename##*.} expression is parameter expansion ("parameters" being the technical name for the shell feature that other languages call "variables"). Plain ${varname} is the value of the parameter named varname, and if that's all you're doing, you can leave off the curly braces and just put $varname. But if you leave the curly braces there, you can put other things inside them after the name, to modify the result. The # and % are some of the most basic modifiers - they remove a prefix or suffix of the string that matches a wildcard pattern. # removes from the beginning, and % from the end; in each case, a single instance of the symbol removes the shortest matching string, while a double symbol matches the longest. So ${filename##*.} is "the value of filename with everything from the beginning to the last period removed", while ${filename%.*} is "the value of filename with everything from the last period to the end removed".
The backticks syntax (`...`) is the original way of doing command substitution in the Bourne shell, and has since been borrowed by languages like Perl and Ruby to incorporate calling out to system commands. But it doesn't deal well with nesting, and its attempt to even allow nesting means that quoting works differently inside them, and it's all very confusing. The newer $(...) syntax, originally introduced in the Korn shell and then adopted by Bash and zsh and codified by POSIX, lets quoting work the same at all levels of a nested substitution and makes for a nice symmetry with the ${...} parameter expansion.
As #e0k states in a comment on the question the ${varname...} syntax is Bash's parameter (variable) expansion. It has its own syntax that is unrelated to regular expressions; it encompasses a broad set of features that include:
specifying a default value
prefix and postfix stripping
string replacement
substring extraction
The difference between `...` and $(...) (both of which are forms of so-called command substitutions) is:
`...` is the older syntax (often called deprecated, but that's not strictly true).
$(...) is its modern equivalent, which facilitates nested use and works more intuitively when it comes to quoting.
See here for more information.
I'm fairly new to Bash and I'm having trouble working out what is happening to my input as it is interpreted. Specifically, when escaping occurs relative to the other expansion steps.
From what I've read, bash does the following (in order):
brace expansion
tilde expansion
parameter and variable expansion
command substitution
arithmetic expansion
word splitting
filename expansion
But this list doesn't include when it converts all escape sequences e.g. '\\' into their meanings e.g. '\'. That is, if I want to print a backslash character. The command to run is
echo \\
not
echo \
So the syntax required for the semantics of a backslash character is two backslashes. This must be converted into a single slash representation internally.
It seems to be sometime before command substitution as I found out with a small test program.
So, my question is: When does this step take place? (or a complete list of the bash interpretation loop would be perfect)
and also, are there any other subtleties in the interpreter that are likely to catch me out? (related to knowing the complete list i guess)
From the man page's Expansion section, just before the Redirection section.
Quote Removal
After the preceding expansions, all unquoted occurrences of the characters \, ', and " that did not result from one of the above expansions
are removed.
Quote removal is one final process after the seven expansions you list.
According to bash manual:
control operator
A token that performs a control function. It is a newline or one of the following: ‘||’, ‘&&’, ‘&’, ‘;’, ‘;;’, ‘|’, ‘|&’, ‘(’, or ‘)’.
metacharacter
A character that, when unquoted, separates words. A metacharacter is a blank or one of the following characters: ‘|’, ‘&’, ‘;’, ‘(’, ‘)’, ‘<’, or ‘>’.
Many characters are both control operator and metacharacter.
So how could I konw the syntax category of e.g. a ;?
Take if COND ; then CMD ; fi as an example.
; seems like a control operator in the context, for it can be substituted by newline.
However removing pre and post spaces around ; still works ok.
Isn't it supposed to be separated by sapces if it's an operator?
According to the bash manual, an operator is:
A control operator or a redirection operator. See Redirections,
for a list of redirection operators. Operators contain at least one
unquoted metacharacter.
The metacharacter is basically any character that cannot be part of a word.
Definition of word:
A sequence of characters treated as a unit by the shell. Words may not include unquoted metacharacters.
There is no need for spaces around operators because they always contain metacharacters, which makes the parser know it is not part of the word.
An exception is redirection, where e.g.
ls 2>&1
requires a space prior to the redirection statement since the operator has a parameter 2, and requires the parameter to be next to the operator (otherwise it will be a parameter to ls).