I'm working on something at the moment and just now I even wonder if what I am working on is even possible.
I want to SSH from jenkins to a shell script and use variables form a rc file that are in a git Repository. (The Shell script and rc file are in the same repo)
Nothing that I tried works and now I'm starting to wondering if it's even possible.
Here's is my local script but i get the same output on jenkins.
docker exec -it test-container bash 'sed -f <(printf "s/${DOMAIN}\.%s/www.&.${DOMAIN_SUFFIX_STAGE}/g\n" ${LANG_KEYS}) /var/www/foo/sed/test.txt > /var/www/foo/sed/new-2.txt'
No matter what I do I get this error
bash: sed -f <(printf "s/${DOMAIN}\.%s/www.&.${DOMAIN_SUFFIX_STAGE}/g\n" ${LANG_KEYS}) /var/www/foo/sed/test.txt > /var/www/foo/sed/new-2.txt: No such file or directory
And yes I can confirm that the directory is there
Here's an easier way to reproduce your problem:
$ bash "echo Hello"
bash: echo Hello: No such file or directory
This happens because the expected syntax is bash yourfile. The string you are passing is not a useful filename, so it fails.
To run a string argument as a command, you can use bash -c commandstring:
$ bash -c "echo Hello"
Hello
This makes bash interpret the parameter as a shell command to execute, instead of a filename to open.
I have a batch file that contains this:
bash -c "shell/rsync_A.sh"
bash -c "shell/rsync_B.sh"
Each of the shell scripts look like this:
rsync_A.sh:
rsync --info=progress2 -rptz --delete -e "ssh -i /root/.ssh/[MY_CERT].pem" [MY_REMOTE_UBUNTU_ON_AWS]:[MY_REMOTE_FOLDER1] [MY_LOCAL_DESTINATION_FOLDER1]
rsync --info=progress2 -rptz --delete -e "ssh -i /root/.ssh/[MY_CERT].pem" [MY_REMOTE_UBUNTU_ON_AWS]:[MY_REMOTE_FOLDER2] [MY_LOCAL_DESTINATION_FOLDER2]
rsync_B.sh:
rsync --info=progress2 -rptz --delete -e "ssh -i /root/.ssh/[MY_CERT].pem" [MY_REMOTE_UBUNTU_ON_AWS]:[MY_REMOTE_FOLDER3] [MY_LOCAL_DESTINATION_FOLDER3]
The problem is that bash always, without fail, hangs when I run the batch file. The first rsync command always seems to run fine, the second always fails (whether inside the same sh file or a different one).
By "hangs" I mean that I see a blinking cursor but no bash prompt and there is no way to get out of it without restarting the entire system (lxssmanager hangs when attempting to restart).
Everything always runs 100% fine when I enter bash and run the shell scripts, but as soon as I get batch involved it breaks.
I have no idea why or how... but the solution was to uninstall BitDefender.
I want to write a shell script that enters into a running docker container, edits a specific file and then exits it.
My initial attempt was this -
Create run.sh file.
Paste the following commands into it
docker exec -it container1 bash
sed -i -e 's/false/true/g' /opt/data_dir/gs.xml
exit
Run the script -
bash ./run.sh
However, once the script enters into the container1 it lands to the bash terminal of it. Seems like the whole script breaks as soon as I enter into the container, leaving parent container behind which contains the script.
The issue is solved By using the below piece of code
myHostName="$(hostname)"
docker exec -i -e VAR=${myHostName} root_reverse-proxy_1 bash <<'EOF'
sed -i -e "s/ServerName .*/ServerName $VAR/" /etc/httpd/conf.d/vhosts.conf
echo -e "\n Updated /etc/httpd/conf.d/vhosts.conf $VAR \n"
exit
I think you are close. You can try something like:
docker exec container1 sed -i -e 's/false/true/g' /opt/data_dir/gs.xml
Explanations:
-it is for interactive session, so you don't need it here.
docker can execute any command (like sed). You don't have to run sed via bash
My jenkins server has a problem of running shell commands in reverse order.
I specify the commands to run
copy over a file to another server
run the update script
For example,
$nohup scp -i .ssh/blah -o StrictHostKeyChecking=no foo.txt tomcat#foo.coo.com:/tmp/FOO.txt &> /dev/null
$nohup ssh -t -t -n -i .ssh/blah -o StrictHostKeyChecking=no tomcat#foo.coo.com '/home/tomcat/bin/update.sh /tmp/FOO.txt.war'
instead the jenkins output console would show:
running update.sh
copying over the file
the same problem also occurs when i pair the two commands into one with &&
and it happens with all my jobs on jenkins
i'm currently running jenkins 1.469 on a tomcat6 server
any help would be appreciated thanks!
EDIT:
i'm running these commands as batch tasks for each job. the problem doesn't seem to be jenkins as this ran correctly
[workspace] $ /bin/sh -xe /tmp/tomcat6-tomcat6-tmp/hudson8724999678434432030.sh
+ echo 1
1
+ echo 2
2
+ echo 3
3
+ echo 4
4
The use of &> to redirect both stdout and stderr is a feature of the bash shell. If you want to use bash-specific features, you need to let Jenkins know the build step should be executed using bash.
This can be done in two ways:
1) Change the default shell in Jenkins global configuration or
2) The first line of your build step must start with #!/bin/bash ...
Note that /bin/sh is not always a symlink to /bin/bash.
I do this in a script:
read direc <<< $(basename `pwd`)
and I get:
Syntax error: redirection unexpected
in an ubuntu machine
/bin/bash --version
GNU bash, version 4.0.33(1)-release (x86_64-pc-linux-gnu)
while I do not get this error in another suse machine:
/bin/bash --version
GNU bash, version 3.2.39(1)-release (x86_64-suse-linux-gnu)
Copyright (C) 2007 Free Software Foundation, Inc.
Why the error?
Does your script reference /bin/bash or /bin/sh in its hash bang line? The default system shell in Ubuntu is dash, not bash, so if you have #!/bin/sh then your script will be using a different shell than you expect. Dash does not have the <<< redirection operator.
Make sure the shebang line is:
#!/bin/bash
or
#!/usr/bin/env bash
And run the script with:
$ ./script.sh
Do not run it with an explicit sh as that will ignore the shebang:
$ sh ./script.sh # Don't do this!
If you're using the following to run your script:
sudo sh ./script.sh
Then you'll want to use the following instead:
sudo bash ./script.sh
The reason for this is that Bash is not the default shell for Ubuntu. So, if you use "sh" then it will just use the default shell; which is actually Dash. This will happen regardless if you have #!/bin/bash at the top of your script. As a result, you will need to explicitly specify to use bash as shown above, and your script should run at expected.
Dash doesn't support redirects the same as Bash.
Docker:
I was getting this problem from my Dockerfile as I had:
RUN bash < <(curl -s -S -L https://raw.githubusercontent.com/moovweb/gvm/master/binscripts/gvm-installer)
However, according to this issue, it was solved:
The exec form makes it possible to avoid shell string munging, and
to RUN commands using a base image that does not contain /bin/sh.
Note
To use a different shell, other than /bin/sh, use the exec form
passing in the desired shell. For example,
RUN ["/bin/bash", "-c", "echo hello"]
Solution:
RUN ["/bin/bash", "-c", "bash < <(curl -s -S -L https://raw.githubusercontent.com/moovweb/gvm/master/binscripts/gvm-installer)"]
Notice the quotes around each parameter.
You can get the output of that command and put it in a variable. then use heredoc. for example:
nc -l -p 80 <<< "tested like a charm";
can be written like:
nc -l -p 80 <<EOF
tested like a charm
EOF
and like this (this is what you want):
text="tested like a charm"
nc -l -p 80 <<EOF
$text
EOF
Practical example in busybox under docker container:
kasra#ubuntu:~$ docker run --rm -it busybox
/ # nc -l -p 80 <<< "tested like a charm";
sh: syntax error: unexpected redirection
/ # nc -l -p 80 <<EOL
> tested like a charm
> EOL
^Cpunt! => socket listening, no errors. ^Cpunt! is result of CTRL+C signal.
/ # text="tested like a charm"
/ # nc -l -p 80 <<EOF
> $text
> EOF
^Cpunt!
do it the simpler way,
direc=$(basename `pwd`)
Or use the shell
$ direc=${PWD##*/}
Another reason to the error may be if you are running a cron job that updates a subversion working copy and then has attempted to run a versioned script that was in a conflicted state after the update...
On my machine, if I run a script directly, the default is bash.
If I run it with sudo, the default is sh.
That’s why I was hitting this problem when I used sudo.
In my case error is because i have put ">>" twice
mongodump --db=$DB_NAME --collection=$col --out=$BACKUP_LOCATION/$DB_NAME-$BACKUP_DATE >> >> $LOG_PATH
i just correct it as
mongodump --db=$DB_NAME --collection=$col --out=$BACKUP_LOCATION/$DB_NAME-$BACKUP_DATE >> $LOG_PATH
Before running the script, you should check first line of the shell script for the interpreter.
Eg:
if scripts starts with /bin/bash , run the script using the below command
"bash script_name.sh"
if script starts with /bin/sh, run the script using the below command
"sh script_name.sh"
./sample.sh - This will detect the interpreter from the first line of the script and run.
Different Linux distributions having different shells as default.