I simply want a function (or just a 1-liner) to find/replace a string inside a variable, and not worry if the variables contain crazy characters.
Pseudo-code:
findReplace () {
#what goes here?
}
myLongVar="some long \crazy/ text my_placeholder bla"
replace="my_placeholder"
replaceWith="I like hamburgers/fries"
myFinalVar=$(findReplace $myLongVar $replace $replaceWith)
All similar questions seem complicated and use files
You can define the function like this:
findReplace1() {
printf "%s" "${1/"$2"/$3}"
}
And then run it like this:
myFinalVar=$(findReplace "$myLongVar" "$replace" "$replaceWith")
Note the double-quotes -- they're very important, because without them bash will split the variables' values into separate words (e.g. "some long \crazy/ text..." -> "some" "long" "\crazy/" "text...") and also try to expand anything that looks like a wildcard into a list of matching filenames. It's ok to leave them off on the right side of an assignment (myFinalVar=...), but that's one of the few places where it's ok. Also, note that within the function I put double-quotes around $2 -- in that case again it's to keep it from being treated as a wildcard pattern, but here it'd a string-match wildcard rather than filenames. Oh, and I used printf "%s" instead of echo because some versions of echo do weird things with strings that contain backslashes and/or start with "-".
And, of course, you can just skip the function and do the replacement directly:
myFinalVar=${myLongVar/"$replace"/$replaceWith}
Try:
myFinalVar=${myLongVar/$replace/$replaceWith}
If your want to replace all occurrences of $replace, not just the first, use:
myFinalVar=${myLongVar//$replace/$replaceWith}
Documentation
From man bash:
${parameter/pattern/string}
Pattern substitution. The pattern is expanded to produce a pattern
just as in pathname expansion. Parameter is expanded and the longest
match of pattern against its value is replaced with
string. If pattern begins with /, all matches of pattern are
replaced with string. Normally only the first match is replaced. If
pattern begins with #, it must match at the beginning of
the expanded value of parameter. If pattern begins with %, it must
match at the end of the expanded value of parameter. If string is
null, matches of pattern are deleted and the / following pattern may
be omitted. If the nocasematch shell option is enabled, the match is
performed without regard to the case of alphabetic
characters. If parameter is # or *, the substitution operation is
applied to each positional parameter in turn, and the
expansion is the resultant list. If parameter is an array variable
subscripted with # or *, the substitution operation is applied to each
member of the array in turn, and the expansion is the
resultant list.
Related
What does a caret do when appended to a bash variable but within braces? I'm trying to decipher this within a bash script:
readonly TEST=${USER^}
When I don't know the meaning of some syntax in bash/sh I use my browsers find function in bash's manual and sh's specification. This is pretty effective as both contain the entire manual in a single page.
From bash's manual:
${parameter^pattern}
[...]
The ‘^’ operator converts lowercase letters matching pattern to uppercase
[...]
If pattern is omitted, it is treated like a ‘?’, which matches every character.
So ${variable^} expands to the value of $variable with the first letter converted to its uppercase variant.
To rephrase - I want to use Bash command substitution and string substitution in the same line.
My actual commands are longer, but the ridiculous use of echo here is just a "substitution" for shortness and acts the same - with same errors ;)
I know we can use a Bash command to produce it's output string as a parameter for another command like this:
echo "$(echo "aahahah</ddd>")"
aahahah</ddd>
I also know we can remove last known part of a string like this:
var="aahahah</ddd>"; echo "${var%</ddd>}"
aahahah
I am trying to write a command where one command gives a string output, where I want to remove last part, which is known.
echo "${$(echo "aahahah</ddd>")%</ddd>}"
-bash: ${$(echo "aahahah</ddd>")%</ddd>}: bad substitution
It might be the order of things happening or substitution only works on variables or hardcoded strings. But I suspect just me missing something and it is possible.
How do I make it work?
Why doesn't it work?
When a dollar sign as in $word or equivalently ${word} is used, it asks for word's content. This is called parameter expansion, as per man bash.
You may write var="aahahah</ddd>"; echo "${var%</ddd>}": That expands var and performs a special suffix operation before returning the value.
However, you may not write echo "${$(echo "aahahah</ddd>")%</ddd>}" because there is nothing to expand once $(echo "aahahah</ddd>") is evaluated.
From man bash (my emphasis):
${parameter%word}
Remove matching suffix pattern. The word is expanded to produce a
pattern just as in pathname expansion. If the pattern
matches a trailing portion of the expanded value of parameter, then
the result of the expansion is the expanded value of parameter
with the shortest matching pattern (the ''%'' case) or the longest matching pattern (the ''%%'' case) deleted.
Combine your commands like this
var=$(echo "aahahah</ddd>")
echo ${var/'</ddd>'}
I have searched a lot, and while I see a couple of examples of these used, specifically from here:
scale=${scale##*[!0-9]*}
[ -z "${scale//[0-9]}" ]
There is no explanation for what these symbols do, how they work or when to use them scripting. I have not found them explained elsewhere when special symbols are discussed. Looks like they could be useful. Can anyone explain how the ## and // work in the script examples on the page linked above? Thanks.
They're part of shell parameter expansion syntax, used to modify the value of the variable. # and % are used to delete a prefix or suffix of the variable, and // is used to substitute one string for another.
${parameter#word}
${parameter##word}
The word is expanded to produce a pattern just as in filename expansion (see Filename Expansion). If the pattern matches the beginning of the expanded value of parameter, then the result of the expansion is the expanded value of parameter with the shortest matching pattern (the ‘#’ case) or the longest matching pattern (the ‘##’ case) deleted.
So ${scale##*[!0-9]*} means to remove the beginning of the string that matches anything followed by a non-digit followed by anything. So foobar becomes an empty string (because everything is removed), while 123 is left alone because [!0-9] never matches anything.
${parameter/pattern/string}
The pattern is expanded to produce a pattern just as in filename expansion. Parameter is expanded and the longest match of pattern against its value is replaced with string. If pattern begins with ‘/’, all matches of pattern are replaced with string. Normally only the first match is replaced. If pattern begins with ‘#’, it must match at the beginning of the expanded value of parameter. If pattern begins with ‘%’, it must match at the end of the expanded value of parameter. If string is null, matches of pattern are deleted and the / following pattern may be omitted.
So ${scale//[0-9]} simply removes all digits from the the value of the variable, then test -z is used to test if this is an empty string (meaning the original string only had digits).
From: http://tldp.org/LDP/abs/html/string-manipulation.html
${string##substring}
Deletes longest match of $substring from front of $string.
${string//substring/replacement}
Replace all matches of $substring with $replacement.
i'm learning to touch type and I pressed ' [ ' instead of ' - ' . When I went on to the next line I couldn't do anything until i typed in k].
I then got a command not found error.
is this a 'apt-get' thing or something else?
Thanks.
This is not an apt-get thing, this is a shell thing.
As #rici says, if you were doing an assignment, then the [ ] notation would be used for array element modification. But you are not, so it isn't.
This is part of globbing, also known as wildcards, or filename expansion. The [ ] bracket notation is known as a character class, and is a range or list of alternative characters, matching one character in the filename. For example, [123] means "1" or "2" or "3", and could also be written as [1-3].
The thing about globbing is that (in bash) a failure to find a file which matches the pattern means that the literal is passed as a command. Assuming you did not have a file called apt[\n] in your current directory, it would try to execute a program by that name! (\n is a common notation for a new-line character).
Here is an example test script:
# Let's create some files
touch apt1 apt2 apt3
# Let's see the expansion
set -x
apt[
]
apt[
123
]
Here are the results, the + is the prompt (PS4) given by the xtrace (set -x):
+ 'apt[
]'
gash.sh: line 10: $'apt[\n]': command not found
So with the first one it was looking for a file with that pattern.
+ apt1 apt2 apt3
gash.sh: line 14: apt1: command not found
But with the second one the \n is treated as whitespace and it matches the filenames. apt1 is not a script in $PATH so it does not find it as a command.
You will notice that it has expanded to all the filenames that match, not just one.
It's because of the bash syntax for setting array elements.
If arr is an array variable,
arr[7]=Seventh
sets element 7 of the array to the string Seventh. So if bash is expecting a possible assignment, then it interprets word[ as a subscript operator, even if word is not a known array variable. (It will be created as an array variable if it is not already an array variable.)
Array subscripts are numeric contexts (unless the array has been declared as an associative array). In a numeric context, you can type an arithmetic expression without using $ as a variable sigil. In numeric contexts, whitespace is ignored, so the newline is ok.
Had apt been declared as an associative array, you could still assign an element using, for example,
apt[
key
]=42
However, in this case, the newlines are not ignored, so they become part of the key.
If there is no = following the ], then the word was not an array assignment, so it is treated as a word. But by then it has already been read, newlines and all. (Pathname expansion occurs after word-splitting, so bash will try to match the word with a filename in the current directory, using the characters, including the newline, between the [ and ] as a character class.)
A contrived example... given
FOO="/foo/bar/baz"
this works (in bash)
BAR=$(basename $FOO) # result is BAR="baz"
BAZ=${BAR:0:1} # result is BAZ="b"
this doesn't
BAZ=${$(basename $FOO):0:1} # result is bad substitution
My question is which rule causes this [subshell substitution] to evaluate incorrectly? And what is the correct way, if any, to do this in 1 hop?
First off, note that when you say this:
BAR=$(basename $FOO) # result is BAR="baz"
BAZ=${BAR:0:1} # result is BAZ="b"
the first bit in the construct for BAZ is BAR and not the value that you want to take the first character of. So even if bash allowed variable names to contain arbitrary characters your result in the second expression wouldn't be what you want.
However, as to the rule that's preventing this, allow me to quote from the bash man page:
DEFINITIONS
The following definitions are used throughout the rest of this docu‐
ment.
blank A space or tab.
word A sequence of characters considered as a single unit by the
shell. Also known as a token.
name A word consisting only of alphanumeric characters and under‐
scores, and beginning with an alphabetic character or an under‐
score. Also referred to as an identifier.
Then a bit later:
PARAMETERS
A parameter is an entity that stores values. It can be a name, a num‐
ber, or one of the special characters listed below under Special Param‐
eters. A variable is a parameter denoted by a name. A variable has a
value and zero or more attributes. Attributes are assigned using the
declare builtin command (see declare below in SHELL BUILTIN COMMANDS).
And later when it defines the syntax you're asking about:
${parameter:offset:length}
Substring Expansion. Expands to up to length characters of
parameter starting at the character specified by offset.
So the rules as articulated in the manpage say that the ${foo:x:y} construct must have a parameter as the first part, and that a parameter can only be a name, a number, or one of the few special parameter characters. $(basename $FOO) is not one of the allowed possibilities for a parameter.
As for a way to do this in one assignment, use a pipe to other commands as mentioned in other responses.
Modified forms of parameter substitution such as ${parameter#word} can only modify a parameter, not an arbitrary word.
In this case, you might pipe the output of basename to a dd command, like
BAR=$(basename -- "$FOO" | dd bs=1 count=1 2>/dev/null)
(If you want a higher count, increase count and not bs, otherwise you may get fewer bytes than requested.)
In the general case, there is no way to do things like this in one assignment.
It fails because ${BAR:0:1} is a variable expansion. Bash expects to see a variable name after ${, not a value.
I'm not aware of a way to do it in a single expression.
As others have said, the first parameter of ${} needs to be a variable name. But you can use another subshell to approximate what you're trying to do.
Instead of:
BAZ=${$(basename $FOO):0:1} # result is bad substitution
Use:
BAZ=$(_TMP=$(basename $FOO); echo ${_TMP:0:1}) # this works
A contrived solution for your contrived example:
BAZ=$(expr $(basename $FOO) : '\(.\)')
as in
$ FOO=/abc/def/ghi/jkl
$ BAZ=$(expr $(basename $FOO) : '\(.\)')
$ echo $BAZ
j
${string:0:1},string must be a variable name
for example:
FOO="/foo/bar/baz"
baz="foo"
BAZ=eval echo '${'"$(basename $FOO)"':0:1}'
echo $BAZ
the result is 'f'