Veeam CLI Unknown Argument when Passing Arguments in Bash Script - bash

I'm writing a bash script to call functions of the Veeam Backup CLI.
In this script I have a function to configure a new backup job.
function configureBackupJob() {
jobname="$1"
reponame="$2"
objects="$3"
advancedOptions="$4"
scheduleOptions="$5"
activeFullBackupOptions="$6"
indexingOptions="$7"
command="veeamconfig job create filelevel --name ${jobname} --reponame ${reponame} --includedirs ${objects} ${advancedOptions} ${scheduleOptions} ${activeFullBackupOptions} ${indexingOptions} --nosnap"
echo "${command}"
veeamconfig job create filelevel --name "$jobname" --reponame "$reponame" --includedirs "$objects" "$advancedOptions" "$scheduleOptions" "$activeFullBackupOptions" "$indexingOptions" --nosnap
}
When calling the script I use a case to determine which function shall be called:
case $command in
# More cases before and after this one
configureBackupJob)
configureBackupJob "$2" "$3" "$4" "$5" "$6" "$7" "$8"
;;
*)
showHelp
;;
esac
I call the script like this:
sudo ./script.sh configureBackupJob "TheJobsName" "RepositoryName" "/path/FoldertoBeBackedUpByVeeam" "--daily --at 12:15" "--weekdays-full Monday,Wednesday" "--indexall"
I used this site from the Veeam help center to know the arguments: Veeam Help Center: Creating File-Level Backup Job
Calling the script results in an error message
Unknown argument: [--daily --at 12:15].
If I call veeamconfig manually the command that my echo shows works fine.
Why can I call the command directly but not from within the script? I tried calling the function without the double quotation marks but that doesn't work.
I can't hardcode all arguments like the
--includedirs
so I need to find a way to pass arguments like the
--daily --at 12:15

The basic problem is that you're passing "--daily --at 12:15" to the veeamconfig command as a single argument, rather than three separate arguments ("--daily", "--at", and "12:15"). This confuses veeamconfig. It looks ok when you echo it because that looses the distinction between spaces between arguments and spaces within arguments.
The best way to handle this depends on a couple of things: First, does your script care which options have to do with scheduling vs indexing vs full backups vs whatever, or is it ok if there's just a list of options to be given to the veeamconfig command? Second, is it possible that the paths/options/whatever might contain spaces (or filename wildcards) in them, as well as between them? (Note: I mostly use macOS, where paths with spaces are very common.)
If the script doesn't have to differentiate between the different types of options, it's pretty simple: just pass all of the options as separate arguments all the way through, and where necessary store them as an array rather than as separate strings (and use bash printf's %q option to properly quote/escape the command options for printing):
function configureBackupJob() {
jobname="$1"
reponame="$2"
object="$3"
allOptions=("${#:4}") # This stores all arguments starting with $4 in an array
printf -v command '%q ' veeamconfig job create filelevel --name "$jobname" --reponame "$reponame" --includedirs "$object" "${allOptions[#]}" --nosnap
echo "${command}"
veeamconfig job create filelevel --name "$jobname" --reponame "$reponame" --includedirs "$object" "${allOptions[#]}" --nosnap
}
And call it like this:
case $command in
# More cases before and after this one
configureBackupJob)
configureBackupJob "${#:2}"
...
And run the overall script like this:
sudo ./script.sh configureBackupJob "TheJobsName" "RepositoryName" "/path/FoldertoBeBackedUpByVeeam" --daily --at 12:15 --weekdays-full Monday,Wednesday --indexall
If the script needs to tell the different types of option apart, things get messier. If there's no possibility of spaces or wildcard-like characters in the options, you could leave those variables unquoted when you pass them to the veeamconfig command, and let the shell's word splitting break them up into individual arguments:
veeamconfig job create filelevel --name "$jobname" --reponame "$reponame" --includedirs "$objects" $advancedOptions $scheduleOptions $activeFullBackupOptions $indexingOptions --nosnap
Note that if you go this route, you need to keep them safely double-quoted at all other points in the process, especially when passing them to the configureBackupJob function. If word-splitting happens too early, it'll just make a mess.
If you need to keep types of options separate and also allow spaces and/or funny characters in the options, it's even more difficult. You might be tempted to put quotes and/or escapes within the options to control this, but word splitting doesn't respect those, so it doesn't work. I think I'll just refer you to this question and hope this doesn't apply.

Related

Bash completion scripting - getting a "transparent proxy"-like behaviour

I am trying to write a simple Bash completion script for a program that runs its arguments as a command. A good example of this is kind of program is the prime-run script provided by the nvidia-prime package:
#!/bin/bash
__NV_PRIME_RENDER_OFFLOAD=1 __VK_LAYER_NV_optimus=NVIDIA_only __GLX_VENDOR_LIBRARY_NAME=nvidia "$#"
This script sets a few environment variables, which instructs the prime driver to use the Nvidia dGPU on a hybrid system. The first argument is treated as the command, and all trailing arguments are passed through. So for example you can run prime-run code . and VSCode will start in the current directory using the dGPU.
Therefore from a completion-script POV, what we want is to basically try to complete as if the prime-run token isn't there (hence "transparent proxy"-like behaviour). To give a rather contrived example:
> prime-run journalc<TAB>
(completes journalctl)
> prime-run journalctl --us<TAB>
(completes --user)
However I am finding this surprisingly difficult in Bash (not that I know how in other shells). So the question is simple: is it possible and if so how?
Ideas I've (hopelessly) had
The simple complete -A command prime-run: the first argument gets completed as a command as expected (let's call it foo), but the following arguments are also completed as commands rather than as arguments to foo
Use some combination of compgen and complete -p to invoke the completion function of foo, but AFAIK the completion function for all foo is locally defined and thus uncallable
TL;DR
bash-completion provides a function named _command_offset (permalink), which is exactly what I need.
# A meta-command completion function for commands like sudo(8), which need to
# first complete on a command, then complete according to that command's own
# completion definition.
Keep reading if you are interested in how I got here.
So I was daydreaming the other day, when it hit me - doesn't sudo basically have the exact same behaviour I want? So the task became simple - reverse engineer the completion script for sudo. Source available here: permalink.
Turns out, most of the code has to do with completing the various options, so it's safe to simply throw most of it out:
L 8-11, 50-52: Related to sudo's edit mode. Safe to ditch.
L 19-24, 27-39, 43-49: These complete sudo's options. Safe to ditch.
So we're left with this:
_sudo()
{
local cur prev words cword split
_init_completion -s || return
for ((i = 1; i <= cword; i++)); do
if [[ ${words[i]} != -* ]]; then
local PATH=$PATH:/sbin:/usr/sbin:/usr/local/sbin
local root_command=${words[i]}
_command_offset $i
return
fi
done
$split && return
} &&
complete -F _sudo sudo sudoedit
The for and if block are there to deal with sudo's options that precede the "guest command". Safe to ditch (after replacing all $i with 1).
The variable $split is only referenced in _init_completion (permalink), and it seems to be used for handling different argument styles (--foo=bar v.s. --foo bar). Same with the -s flag. Irrelevant.
Appending to $PATH and setting $root_command have to do with privilege escalation. Only relevant to sudo.
So after the dust has cleared, by process of elimination, I ended up with this simple chunk of code:
_my-script()
{
local cur prev words cword
_init_completion || return
_command_offset 1
} && complete -F _my-script my-script
Declaring these four local variables and calling _init_completion is standard for all completion scipts, so really it's as simple as one command. Of course someone had to write the massively-complex _command_offset function so lucky me I guess?
Anyways, thank you for reading the story of me messing around and hopefully this will be helpful to some other person in the future.

Programmatically create bash command with flags for items in array

I have a list/array like so:
['path/to/folder/a', 'path/to/folder/b']
This is an example, the array can be of any length. But for each item in the array I'd like to set up the following as a single command:
$ someTool <command> --flag <item-1> --flag <item-2> ... --flag <item-N>
At the moment I am currently doing a loop over the array but I am just wondering if doing them individually has a different behaviour to doing them all at once (which the tool specifies I should do).
for i in "${array[#]}"; do
someTool command --flag $i
done
Whether passing all flag arguments to a single invocation of the tool does the same thing as passing them one-at-a-time to separate invocations depends entirely on the tool and what it does. Without more information, it's impossible to say for sure, but if the instructions recommend passing them all at once, I'd go with that.
The simplest way to do this in bash is generally to create a second array with the flags and arguments as they need to be passed to the tool:
flagsArray=()
for i in "${array[#]}"; do
flagsArray+=(--flag "$i")
done
someTool command "${flagsArray[#]}"
Note: all of the above syntax -- all the quotes, braces, brackets, parentheses, etc -- matter to making this run properly and robustly. Don't leave anything out unless you know why it's there, and that leaving it out won't cause trouble.
BTW, if the option (--flag) doesn't have to be passed as a separate argument (i.e. if the tool allows --flag=path/to/folder/a instead of --flag path/to/folder/a), then you can use a substitution to add the --flag= bit to each element of the array in a single step:
someTool command "${array[#]/#/--flag=}"
Explanation: the /# means "replace at the beginning (of each element)", then the empty string for the thing to replace, / to delimit that from the replacement string, and --flag= as the replacement (/addition) string.

are there security issues with using eval on an environment variable in a bash script?

I have a Bash script in which I call rsync in order to perform a backup to a remote server. To specify that my Downloads folder be backed up, I'm passing "'${HOME}/Downloads'" as an argument to rsync which produces the output:
rsync -avu '/Volumes/Norman Data/Downloads' me#example.com:backup/
Running the command with the variable expanded as above (through the terminal or in the script) works fine, but because of the space in the expanded variable and the fact that the quotes (single ticks) are ignored when included in the variable being passed as part of an argument (see here), the only way I can get it not to choke on the space is to do:
stmt="rsync -avu '${HOME}/Downloads' me#examle.com:backup/"
eval ${stmt}
It seems like there would be some vulnerabilities presented by running eval on anything not 100% private to that script. Am I correct in thinking I should be doing it a different way? If so, any hints for a bash-script-beginner would be greatly appreciated.
** EDIT ** - I actually have a bit more involved use case than. the example above. For the paths passed, I have an array of them, each containing spaces, that I'm then combining into 1 string kind of like
include_paths=(
"'${HOME}/dir_a'"
"'${HOME}/dir_b' --exclude=video"
)
for item in "${include_paths[#]}"
do
inc_args="${inc_args}" ${item}
done
inc_args evaluates to '/Volumes/Norman Data/me/dir_a' '/Volumes/Norman Data/me/dir_b' --exclude=video
which I then try to pass as an argument to rsync but the single ticks are read as literals and it breaks after the 1st /Volumes/Norman because of the space.
rsync -avu "${inc_args}" me#example.com:backup/
Using eval seems to read the single ticks as quotes and executes:
rsync -avu '/Volumes/Norman Data/me/dir_a' '/Volumes/Norman Data/me/dir_b' --exclude=video me#example.com:backup/
like I need it to. I can't seem to get any other way to work.
** EDIT 2 - SOLUTION **
So the 1st thing I needed to do was modify the include_paths array to:
remove single ticks from within double quoted items
move any path-specific flags (ex. --exclude) to their own items directly after the path it should apply to
I then built up an array containing the rsync command and its options, added the expanded include_paths and exclude_paths arrays and the connection string to the remote host.
And finally expanded that array, which ran my entire, properly quoted rsync command. In the end the modified array include_paths is:
include_paths=(
"${HOME}/dir_a"
"${HOME}/dir_b"
"--exclude=video"
"${HOME}/dir_c"
)
and I put everything together with:
cmd=(rsync -auvzP)
for item in "${exclude_paths[#]}"
do
cmd+=("--exclude=${item}")
done
for item in "${include_paths[#]}"
do
cmd+=("${item}")
done
cmd+=("me#example.com:backup/")
set -x
"${cmd[#]}"
Use an array for the commands/option instead of a plain variable.
stmt=(rsync -avu "${HOME}/Dowloads" me#example.com:backup/)
Execute it using the builtin command
command "${stmt[#]}"
...Or I personally just put the options/arguments in an array.
options=(-avu "${HOME}/Download" me#example.com:backup/)
The execute it using rsync
rsync "${options[#]}"
If you have newer version of bash which that supports the additional P.E. parameter expansion, then you could probably quote the array.
options=(-avu "${HOME}/Download" me#example.com:backup/)
Check the output by applying the P.E.
echo "${options[#]#Q}"
Should print
'-avu' '/Volumes/Norman Data/Downloads' 'me#examle.com:backup/'
Then you can just
rsync "${options[#]#Q}"

How do I pass a command parameter in a variable holding the command?

I want to produce the same output as this:
bash utilities.bash "is_net_connected"
But I don't know how to pass "is_net_connected" if command and file is stored in a variable like this:
T=$(bash utilities.bash)
I've tried these but it doesn't seem to work. It's not picking up ${1} in utilities.bash.
$(T) "is_net_connected"
$(T "is_net_connected")
Not the best way to inport but I'm trying to avoid cluttering my main script with function blocks.
T=$(bash utilities.bash) doesn't save the command; it runs the command and saves its output. You want to define a function instead.
T () {
bash utilities.bash "$#"
}
# Or on one line,
# T () { bash utilities.bash "$#"; }
Now
T "is_net_connected"
will run bash utilities.bash with whatever arguments were passed to T. In a case like this, an alias would work the same: alias T='bash utilities.bash'. However, any changes to what T should do will probably require switching from an alias to a function anyway, so you may as well use the function to start. (Plus, you would have to explicitly enable alias expansion in your script.)
You might be tempted to use
T="bash utilities.bash"
$T is_net_connected
Don't be. Unquoted parameter expansions are bad practice that only work in select situations, and you will get bitten eventually if you try to use them with more complicated commands. Use a function; that's why the language supports them.

Append bash parameters and pass forward to other script

I need to pass further original parameters and also I want to add some others. Something like this:
#!/bin/bash
params="-D FOREGROUND"
params+=" -c Include conf/dev.conf"
/usr/local/apache/bin/apachectl $params "$#"
This code above don't work as expected if params contains of two or more parameters, it treated as one parameter.
The code in your example should work if the following command is valid when executed at the command line written exactly like this :
/usr/local/apache/bin/apachectl -D FOREGROUND -c Include conf/dev.conf "$#"
A quick web search leads me to think that what you want is this (notice the additional double quotes) :
/usr/local/apache/bin/apachectl -D FOREGROUND -c "Include conf/dev.conf" "$#"
Here is how to achieve that simply and reliably with arrays, in a way that sidesteps "quoting inside quotes" issues:
#!/bin/bash
declare -a params=()
params+=(-D FOREGROUND)
params+=(-c "Include conf/dev.conf")
/usr/local/apache/bin/apachectl "${params[#]}" "$#"
The params array contains 4 strings ("-D", "FOREGROUND", "-c" and "Include conf/dev/conf"). The array expansion ("${params[#]}", note that the double quotes are important here) expands them to these 4 strings, as if you had written them with double quotes around them (i.e. without further word splitting).
Using arrays with this kind of expansion is a flexible and reliable to way to build commands and then execute them with a simple expansion.
If the issue is the space in the parameter "-c Include conf/dev.conf" then you could just use a backspace to preserve the space character:
params+="-c Include\ conf/dev.conf"

Resources