I have following scripts
#!/bin/bash
set -o xtrace
gluster_volume="a.example.com:/data/brick1/gv0 b.example.com:/data/brick1/gv0"
gluster volume create gv0 replica 2 ${gluster_volume} force
While executing second line this script throwing error related to gluster_volume that its unable read complete line and just considering a.example.com:/data/brick1/gv0 b.example.com(so look like gluster command not able parse it).
Then I run this script with trace and found that command is like
gluster volume create gv0 replica 2 'a.example.com:/data/brick1/gv0 b.example.com:/data/brick1/gv0' force
so script is adding single quotes which are creating problem. I was under the impression that it's due to set -o xtrace the single quotes are not part of the value, just part of the displayed command line. But its not true. if i run above command on command prompt it gives same error and on removing quotes it works.
How i can change my script so it don't add quotes? (I tried to remove quote with sed but not working, something happening at run time).
I need gluster_volume with dynamic values.
Update1:
workers=`echo "${WORKER_HOST_IP}"|sed "s/,$//"`
IFS=','
for worker in ${workers}; do
IFS='-' read -r -a array <<< "$worker"
gluster_volume+=${array[0]}':/data/brick1/gv0 '
done
You can try eval(Be careful if the list of bricks or the string comes from an external untrusted source)
#!/bin/bash
set -o xtrace
gluster_volume="a.example.com:/data/brick1/gv0 b.example.com:/data/brick1/gv0"
eval "gluster volume create gv0 replica 2 ${gluster_volume} force"
Related
I have a bash script that runs another script in a screen on a remote computer. The environment variable GITLAB_CI_TOKEN is set on the host machine and is defined properly. However, the script configure.sh on the remote machine tells that this environment variable is empty when it is executed, even if it is defined on the same line as the script...
Here is the command I am using:
ssh -o "StrictHostKeyChecking=accept-new" "${COMPUTERS_IPS[i]}" \
screen -S "deploy_${COMPUTERS_IPS[i]}" -dm " \
GITLAB_CI_TOKEN=${GITLAB_CI_TOKEN} \
bash \"${REMOTE_FOLDER}/configure.sh\" \"${REMOTE_FOLDER}\" > ${LOG_FILE} 2>&1;
"
Additionally, the logs are not being written to LOG_FILE, but are being displayed on the console of the screen. I have been pulling my hair out over this for the past two days... Any help or guidance would be greatly appreciated :)
Why GITLAB_CI_TOKEN is "empty":
Passing a command to a remote host over ssh is very similar to running it through eval. For example in your case, escaped newlines on the first evaluation become unescaped newlines on a subsequent evaluation. Consider this very simple program named args (place it in bin or somewhere else on your path to demo):
#!/bin/bash
for arg ; do
echo "|$arg|"
done
And these two use cases:
args "\
Hello \
World"
# prints:
# |Hello World|
ssh host args "\
Hello \
World"
# prints:
# |Hello|
# |World|
As you can see, when we run this via ssh the newline we attempted to escape splits our data into two separate lines even though we tried to keep it all on one line. This means your assignment of GITLAB_CI_TOKEN is just a regular shell variable instead of a scoped environment variable for your bash command. A scoped environment variable requires the declaration and the command to happen on the same line.
The easiest thing to do is likely to just export the variable explicitly with export GITLAB_CI_TOKEN=${GITLAB_CI_TOKEN}.
For similar reasons, the output of your command is going to the screen and not the logfile because the outer quotes of screen -dm "commands >output" are getting stripped on the first evaluation, and then the remote host is parsing screen -dm commands >output and assigning the output redirection to screen instead of commands. That means your configure.sh is writing to the screen, and it's the screen program that's writing its own output to a logfile.
To send complex commands to a remote host, you may want to look into tools like printf %q which can produce escaped output suitable for being safely evaluated in an eval-like context. Take a look at BashFAQ/096 for an example.
I am quite new to bash (barely any experience at all) and I need some help with a bash script.
I am using docker-compose to create multiple containers - for this example let's say 2 containers. The 2nd container will execute a bash command, but before that, I need to check that the 1st container is operational and fully configured. Instead of using a sleep command I want to create a bash script that will be located in the 2nd container and once executed do the following:
Execute a command and log the console output in a file
Read that file and check if a String is present. The command that I will execute in the previous step will take a few seconds (5 - 10) seconds to complete and I need to read the file after it has finished executing. I suppose i can add sleep to make sure the command is finished executing or is there a better way to do this?
If the string is not present I want to execute the same command again until I find the String I am looking for
Once I find the string I am looking for I want to exit the loop and execute a different command
I found out how to do this in Java, but if I need to do this in a bash script.
The docker-containers have alpine as an operating system, but I updated the Dockerfile to install bash.
I tried this solution, but it does not work.
#!/bin/bash
[command to be executed] > allout.txt 2>&1
until
tail -n 0 -F /path/to/file | \
while read LINE
do
if echo "$LINE" | grep -q $string
then
echo -e "$string found in the console output"
fi
done
do
echo "String is not present. Executing command again"
sleep 5
[command to be executed] > allout.txt 2>&1
done
echo -e "String is found"
In your docker-compose file make use of depends_on option.
depends_on will take care of startup and shutdown sequence of your multiple containers.
But it does not check whether a container is ready before moving to another container startup. To handle this scenario check this out.
As described in this link,
You can use tools such as wait-for-it, dockerize, or sh-compatible wait-for. These are small wrapper scripts which you can include in your application’s image to poll a given host and port until it’s accepting TCP connections.
OR
Alternatively, write your own wrapper script to perform a more application-specific health check.
In case you don't want to make use of above tools then check this out. Here they use a combination of HEALTHCHECK and service_healthy condition as shown here. For complete example check this.
Just:
while :; do
# 1. Execute a command and log the console output in a file
command > output.log
# TODO: handle errors, etc.
# 2. Read that file and check if a String is present.
if grep -q "searched_string" output.log; then
# Once I find the string I am looking for I want to exit the loop
break;
fi
# 3. If the string is not present I want to execute the same command again until I find the String I am looking for
# add ex. sleep 0.1 for the loop to delay a little bit, not to use 100% cpu
done
# ...and execute a different command
different_command
You can timeout a command with timeout.
Notes:
colon is a utility that returns a zero exit status, much like true, I prefer while : instead of while true, they mean the same.
The code presented should work in any posix shell.
Inb4 anyone saying this is a bad idea, it's actually a reasonable approach for things like this.
I'm writing this Docker container that can execute user provided commands through contained OpenVPN connection in Docker, e.g. docker run vpntunnel curl example.com.
So the ENTRYPOINT of the image will fire up OpenVPN, after the VPN tunnel is up, execute the user provided CMD line.
Problem is, the standard way to run commands after OpenVPN is up is through the --up option of OpenVPN. Here is the man page description of this option:
--up cmd
Run command cmd after successful TUN/TAP device open (pre --user UID change).
cmd consists of a path to script (or executable program), optionally followed
by arguments. The path and arguments may be single- or double-quoted and/or
escaped using a backslash, and should be separated by one or more spaces.
So the reasonable approach here is for ENTRYPOINT script to correctly escape the user provided CMD line and pass the whole thing as one parameter to --up option of OpenVPN.
In case my Docker image needs to perform some initializations after the tunnel is up and before the user command line is executed, I can prepend a script before the user provided CMD line like this: --up 'tunnel-up.sh CMD...' and in the last line of tunnel-up.sh use "$#" to execute user provided arguments.
Now as you may guess, the only problem left is how to correctly escape an entire command line to be able to passed as a single argument.
The naive approach is just --up "tunnel-up.sh $#" but it surely can't distinguish command lines between a b c and "a b" c.
In bash 4.4+ you can use parameter transformation with # to quote values:
--up "tunnel-up.sh ${*#Q}"
In prior versions you could use printf '%q' to achieve the same effect:
--up "tunnel-up.sh $((($#)) && printf '%q ' "$#")"
(The (($#)) check makes sure there are parameters to print before calling printf.)
ssh="ssh user#host"
dumpstructure="mysqldump --compress --default-character-set=utf8 --no-data --quick -u user -p database"
mysql=$ssh "$dumpstructure"
$mysql | gzip -c9 | cat > db_structure.sql.gz
This is failing on the third line with:
mysqldump --compress --default-character-set=utf8 --no-data --quick -u user -p database: command not found
I've simplified my actualy script for the purpose of debugging this specific error. $ssh and $dumpstructure aren't always being joined together in the real script.
Variables are meant to hold data, not commands. Use a function.
mysql () {
ssh user#host mysqldump --compress --default-character-set=utf8 --nodata --quick -u user -p database
}
mysql | gzip -c9 > db_structure.sql.gz
Arguments to a command can be stored in an array.
# Although mysqldump is the name of a command, it is used here as an
# argument to ssh, indicating the command to run on a remote host
args=(mysqldump --compress --default-character-set=utf8 --nodata --quick -u user -p database)
ssh user#host "${args[#]}" | gzip -c9 > db_structure.sql.gz
Chepner's answer is correct about the best way to do things like this, but the reason you're getting that error is actually even more basic. The line:
mysql=$ssh "$dumpstructure"
doesn't do anything like what you want. Because of the space between $ssh and "$dumpstructure", it'll parse this as environmentvar=value command, which means it should execute the "mysqldump..." part with the environment variable mysql set to ssh user#host. But it's worse than that, since the double-quotes around "$dumpstructure" mean that it won't be split into words, and so the entire string gets treated as the command name (rather than mysqldump being the command name, and the rest being arguments to it).
If this had been the right way to go about building the command, the right way to stick the parts together would be:
mysql="$ssh $dumpstructure"
...so that the whole combined string gets treated as part of the value to assign to mysql. But as I said, you really should use Chepner's approach instead.
Actually, commands in variables should also work and can be in form of `$var` or just $($var). If it says command not found, it could because the command maybe not in you PATH. Or you should give full path of you command.
So let's put this vote down away and talk about this question.
The real problem is mysql=$ssh "$dumpstructure". This means you'll execute $dumpstructure with additional environment mysql=$ssh. So we got command not found exception. It's actually because mysqldump is located on remote server not this host, so it's reasonable this command is not found.
From this point, let's see how to fix this question.
OP want to dumpplicate mysql data from remote server, which means $dumpstructure shoud be executed remotely. Let's see third line mysql=$ssh "$dumpstructure". Now we figure out this would result in problem. So what should be the correct command? The simplest command should be like mysql="$ssh $dumpstructure", which means both $ssh and $dumpstructure will be join into single command line in variable $mysql.
At the end, let's talk about the last command line. I do not agree with variable are meant to hold data, not command. Cause command is also a kind of data. The real problem is how to use it correctly.
OP's command is also supported, at least it is supported on bash 4.2.46.
So the real problem is how to use a variable to hold commands not import a new method to do that, wraping them into a bash function, for example.
So who can tell me why this answer does not come into readers' notice but be voted down?
I'm trying to send this command via ssh:
ssh <user1>#<ip1> ssh <user2>#<ip2> /opt/user/bin -f /opt/user/slap.conf -l /home/admin/`date +%Y%m%d`_Export_file$nr.gz -s "ou=multi" -a "(& (entry=$nr)(serv=PS))" -o wrap=no
this command is customized so do not confuse with this...
But it's not executed, smth like: unexpected '(
If i log in to the server and i give this command it gets executed correctly. So i think it should be something with bracket and parentheses rules.
Please can someone help me?
thank you in advance.
You will need to escape the quotes, possibly twice, since each invocation of ssh will involve stripping a layer off. Put escaped single quotes round the entire command, and then nested unescaped single quotes round the inner command:
ssh <user1>#<ip1> \'ssh <user2>#<ip2> '/opt/user/bin -f /opt/user/slap.conf -l /home/admin/`date +%Y%m%d`_Export_file$nr.gz -s "ou=multi" -a "(& (entry=$nr)(serv=PS))" -o wrap=no'\'
This assumes, by the way, that you want the backticks to be unpacked and the command executed on ip2, rather than beforehand on your source machine, and similarly with the decoding of the $nr variable. It's not clear how you want them interpreted.