My jenkinsfile contains a really long one-line shell command to create gradle.properties. For better readability I replaced it with Heredoc. It does fail now though and I am unsure why. Can you spot the error?
One-liner that does not fail is in this format:
sh("echo -e \"\nSOME_KEY=${VARIABLE_1}\nANOTHER_KEY=${VARIABLE_2}\nYET_ANOTHER_KEY=${VARIABLE_3}\" >> gradle.properties")
Failing Heredoc format:
sh("""cat << EOF >> gradle.properties
SOME_KEY=${VARIABLE_1}
ANOTHER_KEY=${VARIABLE_2}
YET_ANOTHER_KEY=${VARIABLE_3}
EOF""")
Related
I'm trying to run a commands remotely by doing SSH but variable which is defined inside EOF is not getting picked up.
Variable $BASE_PATH is not getting called inside another variable which i'm defining by name FOLDER_NAME . Mentioned inside script too.
I couldn't use EOF in single quotes ('EOF') because i have to use variable from exiting shell too.
#!/usr/bin/ksh
FILE_NAME=$2
jvm_list=$1
for jvm in `echo $jvm_list`
do
ssh $jvm << EOF
echo FILE_NAME=${FILE_NAME}
export BASE_PATH="\${WEB_DOMAIN_HOME}/Server";
echo BASE_PATH=\${BASE_PATH}; ##Value of BASE_PATH is getting picked up##
export FOLDER_NAME=\`ls -1d $"{BASE_PATH}"/properties* | grep -i -v old\` ##Value of BASE_PATH is coming blank here##
echo $FOLDER_NAME
EOF
done
The construct started with << EOF acts like double-quoted string, so dollar signs are special within it, unless escaped.
Here, you escaped the $, so the shell eventually started by ssh evaluates that variable
echo BASE_PATH=\${BASE_PATH}; ##Value of BASE_PATH is getting picked up##
Here, you didn't escape it, so the evaluation happens in the outer shell:
export FOLDER_NAME=\`ls -1d $"{BASE_PATH}"/properties* ...\`
I do also suspect that the quotes there are misplaced. In Bash $".." is a locale-specific translation, and it seems to be the same in ksh. You probably don't have a translation for that, so the string should come back as is: {BASE_PATH}.
Somewhat related to this may be the backticks, since they need to be quoted too. You could use the $( ... ) form of command substitution, so you'd again, only need to think about the $.
I think this may be what you want:
ssh "$jvm" << EOF
export FOLDER_NAME=\$(ls -1d "\${BASE_PATH}"/properties* | grep -i -v old)
EOF
Sanity check:
$ foo=out; bash <<EOF
> foo=in; echo $foo
> EOF
out
$ foo=out; bash <<EOF
> foo=in; echo \$foo
> EOF
in
there is a problem with the invoked via ssh bash, although i have read mans about it i still can't explain the following:
Here is a script, very simple
#!/bin/bash
theUser=$1
theHost=$2
ssh -tt $theUser#$theHost 'bash' << EOF
a=1
echo 'dat '$a
exit
EOF
and here is the result:
victor#moria:~$ bash thelast.sh victor 10.0.0.8
victor#10.0.0.8's password:
a=1
echo 'dat '
exit
victor#mordor:~$ a=1
victor#mordor:~$ echo 'dat '
dat
victor#mordor:~$ exit
exit
Connection to 10.0.0.8 closed.
As you may see, the environment doesn't store the value of the variable "a" so it can't echo it, but any other commands like ls or date return the result.
So the question is what i am doing wrong and how to avoid such behavior?
p.s. i can't replace ssh -tt, but any other command may be freely replaced.
Thanks in advance
Inside the here document, the $a is expanded locally before feeding the input to the ssh command. You can prevent that by quoting the terminator after the << operator as in
ssh -tt $theUser#$theHost 'bash' << 'EOF'
$a is being expanded in the local shell, where it is undefined. In order to prevent this from happening, you should escape it:
echo "dat \$a"
Escaping the $ causes it to be passed literally to the remote shell, rather than being interpreted as an expansion locally. I have also added some double quotes, as it is good practice to enclose parameter expansions inside them.
As per the title, is it possible to get the raw command line used to invoke a ruby script?
The exact behaviour I'm after is similar to SSH when invoking a command directly:
ssh somehost -- ls -l
SSH will run "ls -l" on the server. It needs to be unparsed because if the shell has already interpreted quotes and performed expansions etc the command may not work correctly (if it contains quotes and such). This is why ARGV is no good; quotes are stripped.
Consider the following example:
my-command -- sed -e"s/something/something else/g"
The ARGV for this contains the following:
--
sed
-es/something/something else/g
The sed command will fail as the quotes will have been stripped and the space in the substitution command means that sed will not see "else/g".
So, to re-iterate, is it possible to get the raw command line used to invoke a ruby script?
No, this is at the OS level.
You could try simply quoting the entire input:
my-command -- "sed -e\"s/something/something else/g\""
In Ruby, this could be used like this:
ruby -e "puts ARGV[0]" -- "sed -e\"s/something/something else/g\""
(output) sed -e"s/something/something else/g"
Or, in a file putsargv1.rb (with the contents puts ARGV[1]):
ruby -- "putsargv1.rb" "sed -e\"s/something/something else/g\""
(output) sed -e"s/something/something else/g"
Your example is misguided. ssh somehost -- ls * will expand * on localhost (into e.g. ls localfile1 localfile2 localfile3), then execute that on the remote host, with the result of lots and lots of ls: cannot access xxx: No such file or directory errors. ssh does not see the uninterpreted command line.
As you said, you would get -es/something/something else/g as a single parameter. That is exactly what sed would get, too. This is, in fact, identical to what you get if you write -e"s/something/something else/g" and to "-es/something/something else/g", and to -es/something/something\ else.
Using this fact, you can use Shellwords.shellescape to "protect" the spaces and other unmentionables before handing them off to an external process. You can't get the original line, but you can make sure that you preserve the semantics.
Shellescape on the args worked but didn't quite mimic SSH. Take the following example (see below for test.rb contents):
ruby test.rb -- ls -l / \| sed -e's/root/a b c/g'
This will fail using the shellescape approach but succeed with SSH. I opted for manually escaping quotes and spaces. There may be some edge cases this doesn't capture but it seems to work for the majority of cases.
require 'shellwords'
unparsed = if ARGV.index('--')
ARGV.slice(ARGV.index('--') + 1, ARGV.length)
end || []
puts "Unparsed args: #{unparsed}"
exit if unparsed.empty?
shellescaped = unparsed.map(&Shellwords.method(:shellescape)).join(" ")
quoted = unparsed.map do |arg|
arg.gsub(/(["' ])/) { '\\' + $1 }
end.join(" ")
puts "Shellescaped: #{shellescaped}"
puts `bash -c #{shellescaped.shellescape}`
puts "Quoted: #{quoted}"
puts `bash -c #{quoted.shellescape}`
Thanks for your answers :)
If I have a script such as the below where I define a command to run in CMD_VAR, a variable, how do I get it executed in the same Bash script?
I do it this way because I want to log CMD_VAR to a file as well.
#!/bin/sh
CMD_VAR="echo hello world >> somelogfile"
In general you should not store redirections in a variable. And you should store commands in an array.
cmd=(echo "hello world")
log="somelogfile"
"${cmd[#]}" >> "$log"
As part of a system build script I have a script that creates various files and configurations.
However one part of the build script creates a new script that contains variables that I don't want resolved when the build script runs. Code snippet example
cat - > /etc/profile.d/mymotd.sh <<EOF
hostname=`uname -n`
echo -e "Hostname is $hostname"
EOF
I have tried all sorts of combinations of ' and " and ( and [ but I cannot get the script to send the content without substituting the values and placing the substitutes in the new script rather than the original text.
Ideas?
The easiest method, assuming you don't want anything to be substituted in the here doc, is to put the EOF marker in quotes, like this:
cat - > /etc/profile.d/mymotd.sh <<'EOF'
hostname=`uname -n`
echo -e "Hostname is $hostname"
EOF
Easiest is to escape the $
echo -e "Hostname is \$hostname"