This question already has answers here:
Why does shell ignore quoting characters in arguments passed to it through variables? [duplicate]
(3 answers)
Closed 6 years ago.
I'm using FPM to create Debian packages, and I've run into a little problem.
My bash script takes 5 arguments.
TARGET=$1
VERSION=$2
DESCRIPTION=$3
DEPENDENCIES=$4
REVISION=$5
The troublesome one is $4, where I pass it the following string
-d "apt-transport-https > 0.8.16~exp12ubuntu10.15" -d "mongodb > 1:2.0.4-1ubuntu2" -d "ntp > 1:4.2.6.p3+dfsg-1ubuntu3.1"
The double quotes are escaped in my string, so when I echo $DEPENDENCIES, the quotes show up correctly.
FPM uses the -d flag can be used multiple times, and I need to be able to pass a list of parameters from my script to fpm.
I would like to do something like:
fpm ...blah blah details... $DEPENDENCIES path
$DEPENDENCIES should pass the multiple flags to fpm, but it only seems to recognize the first one. If I write the arguments out manually, it works fine, but trying to use a string to pass the parameters doesn't work.
I'm not sure what's up. Help?
This will be helpful reading: http://mywiki.wooledge.org/BashFAQ/050
If I were you, I would rearrange your parameters
#!/bin/bash
TARGET=$1
VERSION=$2
DESCRIPTION=$3
REVISION=$4
shift 4
DEPENDENCIES=( "$#" )
Dependencies is "all the rest" of the arguments, properly separated. You will invoke your script like this
./script tgt ver "this is the description" rev-1.1.1 -d "apt-transport-https > 0.8.16~exp12ubuntu10.15" -d "mongodb > 1:2.0.4-1ubuntu2" -d "ntp > 1:4.2.6.p3+dfsg-1ubuntu3.1"
In your script, the DEPENDENCIES array will contain these elements
DEPENDENCIES[0]=-d
DEPENDENCIES[1]="apt-transport-https > 0.8.16~exp12ubuntu10.15"
DEPENDENCIES[2]=-d
DEPENDENCIES[3]="mongodb > 1:2.0.4-1ubuntu2"
DEPENDENCIES[4]=-d
DEPENDENCIES[5]="ntp > 1:4.2.6.p3+dfsg-1ubuntu3.1"
In your script, call fpm like:
fpm ...blah blah details... "${DEPENDENCIES[#]}" path
Related
I'm writing a bash script to call functions of the Veeam Backup CLI.
In this script I have a function to configure a new backup job.
function configureBackupJob() {
jobname="$1"
reponame="$2"
objects="$3"
advancedOptions="$4"
scheduleOptions="$5"
activeFullBackupOptions="$6"
indexingOptions="$7"
command="veeamconfig job create filelevel --name ${jobname} --reponame ${reponame} --includedirs ${objects} ${advancedOptions} ${scheduleOptions} ${activeFullBackupOptions} ${indexingOptions} --nosnap"
echo "${command}"
veeamconfig job create filelevel --name "$jobname" --reponame "$reponame" --includedirs "$objects" "$advancedOptions" "$scheduleOptions" "$activeFullBackupOptions" "$indexingOptions" --nosnap
}
When calling the script I use a case to determine which function shall be called:
case $command in
# More cases before and after this one
configureBackupJob)
configureBackupJob "$2" "$3" "$4" "$5" "$6" "$7" "$8"
;;
*)
showHelp
;;
esac
I call the script like this:
sudo ./script.sh configureBackupJob "TheJobsName" "RepositoryName" "/path/FoldertoBeBackedUpByVeeam" "--daily --at 12:15" "--weekdays-full Monday,Wednesday" "--indexall"
I used this site from the Veeam help center to know the arguments: Veeam Help Center: Creating File-Level Backup Job
Calling the script results in an error message
Unknown argument: [--daily --at 12:15].
If I call veeamconfig manually the command that my echo shows works fine.
Why can I call the command directly but not from within the script? I tried calling the function without the double quotation marks but that doesn't work.
I can't hardcode all arguments like the
--includedirs
so I need to find a way to pass arguments like the
--daily --at 12:15
The basic problem is that you're passing "--daily --at 12:15" to the veeamconfig command as a single argument, rather than three separate arguments ("--daily", "--at", and "12:15"). This confuses veeamconfig. It looks ok when you echo it because that looses the distinction between spaces between arguments and spaces within arguments.
The best way to handle this depends on a couple of things: First, does your script care which options have to do with scheduling vs indexing vs full backups vs whatever, or is it ok if there's just a list of options to be given to the veeamconfig command? Second, is it possible that the paths/options/whatever might contain spaces (or filename wildcards) in them, as well as between them? (Note: I mostly use macOS, where paths with spaces are very common.)
If the script doesn't have to differentiate between the different types of options, it's pretty simple: just pass all of the options as separate arguments all the way through, and where necessary store them as an array rather than as separate strings (and use bash printf's %q option to properly quote/escape the command options for printing):
function configureBackupJob() {
jobname="$1"
reponame="$2"
object="$3"
allOptions=("${#:4}") # This stores all arguments starting with $4 in an array
printf -v command '%q ' veeamconfig job create filelevel --name "$jobname" --reponame "$reponame" --includedirs "$object" "${allOptions[#]}" --nosnap
echo "${command}"
veeamconfig job create filelevel --name "$jobname" --reponame "$reponame" --includedirs "$object" "${allOptions[#]}" --nosnap
}
And call it like this:
case $command in
# More cases before and after this one
configureBackupJob)
configureBackupJob "${#:2}"
...
And run the overall script like this:
sudo ./script.sh configureBackupJob "TheJobsName" "RepositoryName" "/path/FoldertoBeBackedUpByVeeam" --daily --at 12:15 --weekdays-full Monday,Wednesday --indexall
If the script needs to tell the different types of option apart, things get messier. If there's no possibility of spaces or wildcard-like characters in the options, you could leave those variables unquoted when you pass them to the veeamconfig command, and let the shell's word splitting break them up into individual arguments:
veeamconfig job create filelevel --name "$jobname" --reponame "$reponame" --includedirs "$objects" $advancedOptions $scheduleOptions $activeFullBackupOptions $indexingOptions --nosnap
Note that if you go this route, you need to keep them safely double-quoted at all other points in the process, especially when passing them to the configureBackupJob function. If word-splitting happens too early, it'll just make a mess.
If you need to keep types of options separate and also allow spaces and/or funny characters in the options, it's even more difficult. You might be tempted to put quotes and/or escapes within the options to control this, but word splitting doesn't respect those, so it doesn't work. I think I'll just refer you to this question and hope this doesn't apply.
I have a Bash script in which I call rsync in order to perform a backup to a remote server. To specify that my Downloads folder be backed up, I'm passing "'${HOME}/Downloads'" as an argument to rsync which produces the output:
rsync -avu '/Volumes/Norman Data/Downloads' me#example.com:backup/
Running the command with the variable expanded as above (through the terminal or in the script) works fine, but because of the space in the expanded variable and the fact that the quotes (single ticks) are ignored when included in the variable being passed as part of an argument (see here), the only way I can get it not to choke on the space is to do:
stmt="rsync -avu '${HOME}/Downloads' me#examle.com:backup/"
eval ${stmt}
It seems like there would be some vulnerabilities presented by running eval on anything not 100% private to that script. Am I correct in thinking I should be doing it a different way? If so, any hints for a bash-script-beginner would be greatly appreciated.
** EDIT ** - I actually have a bit more involved use case than. the example above. For the paths passed, I have an array of them, each containing spaces, that I'm then combining into 1 string kind of like
include_paths=(
"'${HOME}/dir_a'"
"'${HOME}/dir_b' --exclude=video"
)
for item in "${include_paths[#]}"
do
inc_args="${inc_args}" ${item}
done
inc_args evaluates to '/Volumes/Norman Data/me/dir_a' '/Volumes/Norman Data/me/dir_b' --exclude=video
which I then try to pass as an argument to rsync but the single ticks are read as literals and it breaks after the 1st /Volumes/Norman because of the space.
rsync -avu "${inc_args}" me#example.com:backup/
Using eval seems to read the single ticks as quotes and executes:
rsync -avu '/Volumes/Norman Data/me/dir_a' '/Volumes/Norman Data/me/dir_b' --exclude=video me#example.com:backup/
like I need it to. I can't seem to get any other way to work.
** EDIT 2 - SOLUTION **
So the 1st thing I needed to do was modify the include_paths array to:
remove single ticks from within double quoted items
move any path-specific flags (ex. --exclude) to their own items directly after the path it should apply to
I then built up an array containing the rsync command and its options, added the expanded include_paths and exclude_paths arrays and the connection string to the remote host.
And finally expanded that array, which ran my entire, properly quoted rsync command. In the end the modified array include_paths is:
include_paths=(
"${HOME}/dir_a"
"${HOME}/dir_b"
"--exclude=video"
"${HOME}/dir_c"
)
and I put everything together with:
cmd=(rsync -auvzP)
for item in "${exclude_paths[#]}"
do
cmd+=("--exclude=${item}")
done
for item in "${include_paths[#]}"
do
cmd+=("${item}")
done
cmd+=("me#example.com:backup/")
set -x
"${cmd[#]}"
Use an array for the commands/option instead of a plain variable.
stmt=(rsync -avu "${HOME}/Dowloads" me#example.com:backup/)
Execute it using the builtin command
command "${stmt[#]}"
...Or I personally just put the options/arguments in an array.
options=(-avu "${HOME}/Download" me#example.com:backup/)
The execute it using rsync
rsync "${options[#]}"
If you have newer version of bash which that supports the additional P.E. parameter expansion, then you could probably quote the array.
options=(-avu "${HOME}/Download" me#example.com:backup/)
Check the output by applying the P.E.
echo "${options[#]#Q}"
Should print
'-avu' '/Volumes/Norman Data/Downloads' 'me#examle.com:backup/'
Then you can just
rsync "${options[#]#Q}"
This question already has an answer here:
Accessing environment variables that don't map to valid shell variable names
(1 answer)
Closed 5 years ago.
We are using elastic beanstalk. Some values are environment properties of the environment. When I perform a container_command I'm able to read this properties as environment variables. The problem is the following: a lot of properties are named like this db:user or collector:server and after that the value.
How can I read this values? I can interpret them as environment variables. So the environment properties with 'normal' names I can read. But not those ones who contain a ':' in their name:
To test (+ make it clearer for people who don't know elastic beanstalk) I've created this. The global goal is to read the value of a variable which contains a ':' in its name.
#!/bin/bash
${myvar:test}="hey"
echo ${myvar:test}
$./test.sh
$./test.sh: line 3: =hey: command not found
: isn't an allowed character in shell variables at all, and ${test:foo} has a completely separate meaning (it expands $test with a default value of foo if no variable named test is defined).
If your operating system is Linux, however, you can directly parse your original environment variables from procfs:
#!/usr/bin/env bash
declare -A env=( ) ## <- note that this requires bash 4.0 or newer
while IFS= read -r -d '' envvar; do
[[ $envvar = *:* ]] || continue
varname=${envvar%%=*}
value=${envvar#*=}
env[$varname]=$value
done < /proc/self/environ
echo "The value of the environment variable db:user is: <<${env[db:user]}>>"
Note that to test this, though, you'll need to be able to actually create an environment variable with a literal colon, and your code currently fails at doing so. Consider instead:
env db:user="test value for db:user" ./yourscript
This question already has answers here:
How to use a variable's value as another variable's name in bash [duplicate]
(6 answers)
Closed 6 years ago.
I'm trying to get the values of some php variables into a bash script in order to set other bash variables. Basically I need to be able to access the value of a variable - variable name.
the script can find & read the file, find the php variables, but when I try to set them it just hangs. Here is what I have:
variables=(database_user database_password dbase);
paths=(modx_core_path modx_connectors_path modx_manager_path modx_base_path);
for index in "${variables[#]}"; do
index=\$$index
echo $index;
$index="$(grep -oE '\$${!index} = .*;' $config | tail -1 | sed 's/$${!index} = //g;s/;//g')";
done
not sure what I am doing wrong here...
You are trying to perform an indirect assignment.
You should get rid of these two lines :
index=\$$index
echo $index;
By simply writing :
echo "${!index}"
Which does an indirect expansion cleanly (gives you the value of the variable whose name is contained in variable index).
Next, the problematic line is this one:
$index="$(grep -oE '\$${!index} = .*;' ... (rest omitted)
In Bash, an assignment cannot begin with a $.
One way you can perform an indirect assignment is this (after removing the index re-assignment as suggested above) :
printf -v "$index" "$(grep -oE '\$${!index} = .*;' ... (rest omitted)
This uses the -v option of printf, which causes the value passed as the final argument to be assigned to a variable, the name of which is passed to the -v option.
There are other ways of handling indirect assignment/expansions, some with code injection risks, as they use (potentially uncontrolled) data as code. This is something you may want to research further.
Please note I am assuming the actual grep command substitution works (I have not tested).
I need to pass further original parameters and also I want to add some others. Something like this:
#!/bin/bash
params="-D FOREGROUND"
params+=" -c Include conf/dev.conf"
/usr/local/apache/bin/apachectl $params "$#"
This code above don't work as expected if params contains of two or more parameters, it treated as one parameter.
The code in your example should work if the following command is valid when executed at the command line written exactly like this :
/usr/local/apache/bin/apachectl -D FOREGROUND -c Include conf/dev.conf "$#"
A quick web search leads me to think that what you want is this (notice the additional double quotes) :
/usr/local/apache/bin/apachectl -D FOREGROUND -c "Include conf/dev.conf" "$#"
Here is how to achieve that simply and reliably with arrays, in a way that sidesteps "quoting inside quotes" issues:
#!/bin/bash
declare -a params=()
params+=(-D FOREGROUND)
params+=(-c "Include conf/dev.conf")
/usr/local/apache/bin/apachectl "${params[#]}" "$#"
The params array contains 4 strings ("-D", "FOREGROUND", "-c" and "Include conf/dev/conf"). The array expansion ("${params[#]}", note that the double quotes are important here) expands them to these 4 strings, as if you had written them with double quotes around them (i.e. without further word splitting).
Using arrays with this kind of expansion is a flexible and reliable to way to build commands and then execute them with a simple expansion.
If the issue is the space in the parameter "-c Include conf/dev.conf" then you could just use a backspace to preserve the space character:
params+="-c Include\ conf/dev.conf"