In my bash_profile I created a deploy function. There I read a path from a file:
deployPath=$(jq '.["prod-deploy-path"]' ./package.json)
When I print the path, I get the following:
"/var/www/user/websitefolder/dist"
Now I want to remove the last part of the URL:
pwdPath=$(dirname $deployPath)
but I receive
"/var/www/user/websitefolder/dist
(notice the last quotation mark is missing)
If I do the same thing with a fixed string instead of the variable
dirname "/var/www/user/websitefolder/dist"
I receive what I want to receive:
dirname "/var/www/user/websitefolder/"
What do I need to write in my bash_profile to get the same result with the variable as with a string?
You will need to use the -r option for jq:
deployPath=$(jq -r '.["prod-deploy-path"]' ./package.json)
which will output a raw string.
Your problem is not that you got a variable but that the content in the variable are wrapped in quotes, eg:
with_quotes='"something here"'
without_quotes='something here'
echo "$with_quotes" # "something here"
echo "$without_quotes" # something here
jq will by default print a JSON encoded string. This also means that if your string contains double quotes they will be escaped like this: "hello \"world"
From man jq:
--raw-output / -r
With this option, if the filter's result is a string then it will be written
directly to standard output rather than being formatted as a JSON string
with quotes. This can be useful for making jq filters talk to non-JSON-based
systems
Related
I wanna GET request with variable x in command like this
curl 'abc.com' --data '{"songs":{"module":"server","method":"get_songs","param":{"songid":$x}}}'
but the command can't get the value of x, because it's nested too deeply!
Hope you help!
single quotes is hiding the variable from bash, the nesting doesn't matter, it's just a string.
you can escape the variable form single quotes, let bash substitute the value and then join back to the rest of the string.
... '{"songs":{"module":"server","method":"get_songs","param":{"songid":'"$x"'}}}'
The variable $x is not replaced because it is embedded in a string enclosed in apostrophes. The variables are not expanded and the escape sequences do not work in the strings enclosed in apostrophes.
There are multiple solutions for your problem.
The simplest one is to close the string before the variable and use another string after it:
curl 'abc.com' \
--data '{"songs":{"module":"server","method":"get_songs","param":{"songid":'$x'}}}'
However, this produces unexpected results if the value of $x contains spaces or characters that are special to the shell (like * etc).
It has a very simple fix: let the variable $x be outside the strings enclosed in apostrophes but enclose it in quotes:
curl 'abc.com' \
--data '{"songs":{"module":"server","method":"get_songs","param":{"songid":'"$x"'}}}'
However, while this works, it is not easy to read and the quotes around $x can be confused with the quotes from the JSON.
I recommend to use the here-doc syntax to build the JSON:
DATA=$(cat << END
{"songs":{"module":"server","method":"get_songs","param":{"songid":$x}}}
END
)
curl 'abc.com' --data "$DATA"
This way the variable is expanded and there is no need for extra quotes or apostrophes.
This approach (and also your initial approach) has a major drawback. You need to write the JSON manually and a missing or an extra comma or quote will break it. A simple way to prevent this is to use a tool (for example jq) to generate the JSON:
DATA=$(echo -n "$x" | jq -s -R '
{
songs: {
module: "server",
method: "get_songs",
param: { songid: . }
}
}
')
curl 'abc.com' --data "$DATA"
The advantage of using jq (or other tool designed to work with JSON) becomes visible when $x contains characters that must be encoded when used in JSON (quotes, newlines). The variable substitution provided by the shell does not do any encoding for them, the resulting JSON is malformed and the server where you send it will reject your request.
I have a script like this -
#!/usr/bin/env bash
set -e
volume=$(docker volume inspect volume_name | jq '.[0].Mountpoint')
echo $volume
createdir=$volume/testdir
mkdir -p ${createdir}
But it does not create any directory, in the volume path. echo $volume does print the correct path - /var/lib/docker/volumes/volume_name/_data
When I give mkdir -p /var/lib/docker/volumes/volume_name/_data/testdir. It creates it. What am I doing wrong with substituting?
Your issue is because your jq call is missing a -r option switch to output a raw string rather than a JSON string that is not usable as a path.
See man jq:
--raw-output / -r:
With this option, if the filter´s result is a string then it will be written directly to standard output rather than being formatted as a JSON string with quotes. This can be useful for making jq filters talk to non-JSON-based systems.
Also, to prevent word splitting of paths, always add double quotes around variables's expansion.
I detail cases where double quotes are optional or mandatory in the code's comments; although, in doubt, adding double quotes is safe, except for special cases of:
Explicitly desirable word splitting or globbing match of pattern.
Variable expansion as a RegEx pattern.
Here is your code with fixes:
#!/usr/bin/env bash
# Expansion of sub-shell output does not need double quotes,
# because it is a simple variable assignment.
# It would be mandatory if the output was an argument to a command.
volume=$(docker volume inspect volume_name | jq -r '.[0].Mountpoint')
# Here double quotes are fancy optional but a good practice anyway.
# If not present and volume contained a globbing pattern,
# it would be expanded. It would also generate a path check access
# to the file-system. Better be safe with double quotes.
echo "$volume"
# Here double quotes are optional because it is an assignment and not
# an argument to a command.
createdir=$volume/testdir
# Here double quotes are mandatory,
# as the variable is an argument to the mkdir command.
mkdir -p -- "$createdir"
In a bash script (newbie to bash), I have a variable like this: "this is a message [REL]" and I want to convert it to this: "this is a message".
I tryed native bash string manipulation, like CLEAN_MESSAGE=${MESSAGE#REL} and didn't work so I'm trying with sed, I followed a few posts like this one: How to replace paired square brackets with other syntax with sed? but no luck
This is my current status:
MESSAGE=$(cat $1) #A message with a tag [REL]
RELTAG=\[[REL]\] #A variable with regexp for tag, tryed \[REL\], \[(REL)\], \[\(REL\)\] and other variants
CLEAN_MESSAGE=$(echo "$MESSAGE" | sed "s/$RELTAG//")
echo $CLEAN_MESSAGE
With some variants of the RELTAG I get EL] or [RE, but never the complete tag removed.
Help?
Use % to remove at end of string
$ s='this is a message [REL]'
$ echo "${s% \[REL]}"
this is a message
$ # if variable contains the removal glob
$ r=' \[REL]'
$ echo "${s%$r}"
this is a message
See wooledge Parameter Expansion for further reading
I'd like to use sed to do a replace, but not by searching for what to replace.
Allow me to explain. I have a variable set to a default value initially.
VARIABLE="DEFAULT"
I can do a sed to replace DEFAULT with what I want, but then I would have to put DEFAULT back when I was all done. This is becuase what gets stored to VARIABLE is unique to the user. I'd like to use sed to search for somthing else other than what to replace. For example, search for VARIABLE=" and " and replace whats between it. That way it just constantly updates and there is no need to reset VARIABLE.
This is how I do it currently:
I call the script and pass an argument
./script 123456789
Inside the script, this is what happens:
sed -i "s%DEFAULT%$1%" file_to_modify
This replaces
VARIABLE="DEFAULT"
with
VARIABLE="123456789"
It would be nice if I didn't have to search for "DEFAULT", because then I would not have to reset VARIABLE at end of script.
sed -r 's/VARIABLE="[^"]*"/VARIABLE="123456789"/' file_to_modify
Or, more generally:
sed -r 's/VARIABLE="[^"]*"/VARIABLE="'"$1"'"/' file_to_modify
Both of the above use a regular expression that looks for 'VARIABLE="anything-at-all"' and replaces it with, in the first example above 'VARIABLE="123456789"' or, in the second, 'VARIABLE="$1"' where "$1" is the first argument to your script. The key element is [^"]. It means any character other than double-quote. [^"]* means any number of characters other than double-quote. Thus, we replace whatever was in the double-quotes before, "[^"]*", with our new value "123456789" or, in the second case, "$1".
The second case is a bit tricky. We want to substitute $1 into the expression but the expression is itself in single quotes. Inside single-quotes, bash will not substitute for $1. So, the sed command is broken up into three parts:
# spaces added for exposition but don't try to use it this way
's/VARIABLE="[^"]*"/VARIABLE="' "$1" '"/'
The first part is in single quotes and bash passes it literally to sed. The second part is in double-quotes, so bash will subsitute in for the value of `$``. The third part is in single-quotes and gets passed to sed literally.
MORE: Here is a simple way to test this approach on the command line without depending on any files:
$ new=1234 ; echo 'VARIABLE="DEFAULT"' | sed -r 's/VARIABLE="[^"]*"/VARIABLE="'"$new"'"/'
VARIABLE="1234"
The first line above is the command run at the prompt ($). The second is the output from running the command..
If I have a variable containing an unescaped dollar sign, is there any way I can echo the entire contents of the variable?
For example something calls a script:
./script.sh "test1$test2"
and then if I want to use the parameter it gets "truncated" like so:
echo ${1}
test1
Of course single-quoting the varaible name doesn't help. I can't figure out how to quote it so that I can at least escape the dollar sign myself once the script recieves the parameter.
The problem is that script receives "test1" in the first place and it cannot possibly know that there was a reference to an empty (undeclared) variable. You have to escape the $ before passing it to the script, like this:
./script.sh "test1\$test2"
Or use single quotes ' like this:
./script.sh 'test1$test2'
In which case bash will not expand variables from that parameter string.
The variable is replaced before the script is run.
./script.sh 'test1$test2'
by using single quotes , meta characters like $ will retain its literal value. If double quotes are used, variable names will get interpolated.
As Ignacio told you, the variable is replaced, so your scripts gets ./script.sh test1 as values for $0 and $1.
But even in the case you had used literal quotes to pass the argument, you shoudl always quote "$1" in your echo "${1}". This is a good practice.