Below is a snippet from my Jenkins file -
stage('Configure replication agents') {
environment {
AUTHOR_NAME="XX.XX.XX.XX"
PUBLISHER_NAME="XX.XX.XX.XX"
REPL_USER="USER"
REPL_PASSWORD="PASSWORD"
AUTHOR_PORT="4502"
PUBLISHER_PORT="4503"
AUTHOR="http://${AUTHOR_NAME}:${AUTHOR_PORT}"
PUBLISHER="http://${PUBLISHER_NAME}:${PUBLISHER_PORT}"
S_URI= "${PUBLISHER}/bin/receive?sling:authRequestLogin=1"
}
steps {
sh 'curl -u XX:XX --data "status=browser&cmd=createPage&label=${PUBLISHER_NAME}&title=${PUBLISHER_NAME}&parentPath =/etc/replication/agents.author&template=/libs/cq/replication/templates/agent" ${AUTHOR}/bin/wcmcommand'
}
The above command, in Jenkins console, is printed as
curl -u XX:XX --data status=browser&cmd=createPage&label=XXXX&title=XXX&parentPath =/etc/replication/agents.author&template=/libs/cq/replication/templates/agent http://5XXXX:4502/bin/wcmcommand
Note how the double quotes "" are missing.
I need to preserve the double quotes after --data in this command. How do I do it?
I tried using forward slashes but that didnt work.
Cheers
To expand on my comment, a quick test revealed its the case.
You need to escape twice, once the quote for the shell with a slash, and once that slash with a slash for groovy itself.
node() {
sh 'echo "asdf"'
sh 'echo \"asdf\"'
sh 'echo \\"asdf\\"'
}
Result
[Pipeline] {
[Pipeline] sh
+ echo asdf
asdf
[Pipeline] sh
+ echo asdf
asdf
[Pipeline] sh
+ echo "asdf"
"asdf"
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
After long time of struggling and googling, this is what has worked for me on similar use case:
sh("ssh root#my.server.com \"su user -c \\\"mkdir ${newDirName}\\\"\"")
Update: How I think it gets interpreted
1] sh extension strips first escaping (\" becomes " and \\ becomes \, first and last " are not part of input)
ssh root#my.server.com "su user -c \"mkdir ${newDirName}\""
2] ssh command strips second level of escaping (\" becomes ", while outer " also not part of input)
su user -c "mkdir ${newDirName}"
I had double quotes inside the variable, so escaped single quotes worked for me:
sh "git commit -m \'${ThatMayContainDoubleQuotes}\'"
I needed the output to be with trailing \\ so I had to do something like this
echo 'key1 = \\\\"__value1__\\\\"' > auto.file
File looks like
cat auto.file
key1 = \\"__value1__\\"
Dependent Script
export value1="some-value"
var=${value1}
# Read in template one line at the time, and replace variables
tmpfile=$(mktemp)
sed -E 's/__(([^_]|_[^_])*)__/${\\1}/g' auto.file > ${tmpfile}
while read auto
do
eval echo "$auto"
done < "${tmpfile}" > autoRendered.file
rm -f ${tmpfile}
Rendered File looks like
cat autoRendered.file
key1 = "some-value"
For anyone who comes looking for a fix to a similar issue with quoting numbers during helm install/upgrade, you can use --set-string instead of --set
Ref: https://helm.sh/docs/chart_best_practices/values/#consider-how-users-will-use-your-values
Related
I need to extract some variables and functions from a zsh script into a bash script. Is there any way to do this? What I've tried (some are embarrassingly wrong, but covering everything):
. /script/path.zsh (zsh-isms exist, so it fails)
exec zsh
. /script/path.zsh
exec bash
zsh << 'EOF'
. /script/path.zsh
EOF
chsh -s zsh
. /script/path.zsh
chsh -s bash
This thread is the closest I've found. Unfortunately, I have too many items to import for that to be feasible, and neither script is anywhere near a polyglot. However, the functions and variables that I need to import are polyglots.
You can "scrape" the zsh source file for what you need, then execute the code in bash using eval. Here's an example for doing this for a few functions:
File script.zsh:
test1() {
echo "Hello from test1"
}
test2() {
echo $((1 + $1))
}
File script.sh (bash):
# Specify source script and functions
source_filename="script.zsh"
source_functions=" \
test1 \
test2 \
"
# Perform "sourcing"
function_definitions="$(python -B -c "
import re
with open('${source_filename}', mode='r') as file:
content = file.read()
for func in '${source_functions}'.split():
print(re.search(func + r'\(\).*?\n}', content, flags=re.DOTALL).group())
" )"
eval "${function_definitions}"
# Try out test functions
test1 # Hello from test1
n=5
echo "$n + 1 is $(test2 $n)" # 5 + 1 is 6
Run the bash script and it will make use of the functions test1 and test2 defined in the zsh script:
bash script.sh
The above makes use of Python, specifically its re module. It simply looks for character sequences of the form funcname(), and assumes that the function ends at the first }. So it's not very general, but works if you write your functions in this manner.
While running this Bash command in an sh """ block I am getting an error in the below Groovy Jenkins code.
I am getting this error:
/home/jenkins/workspace/_api-build_features_SSSVCS-12870#tmp/durable-be642e71/script.sh: line 1: syntax error: unterminated quoted string
for the below 2 lines of Groovy code:
aa = sh(script: "aa=\"\$(helm2 version --short --client|awk '{print substr(\$2,1,2)}'\"; echo \$aa", returnStdout: true).trim()
I am using the variable aa in below if then else loop, Please suggest me.
sh """
if [ aa == v2 ]; then
helm package --save=false ${extraArgs} ${PROJ}
else
helm package ${PROJ}
fi
sh """
In the first line, you miss a closing the bracket ')'. In the second block, consider removing the second sh, otherwise will be part of the whole string between the two """.
I am using the below Jenkinsfile. It is printing the value of upload_id when it is executing with the jq command, but when printing the value it is showing null.
Please help me to fix this variable issue.
sh label: '', script: 'upload_id=$(cat output_file.json | jq -r \'.upload_id\')'
sh "echo \"$upload_id\""**
Output :
[Pipeline] sh
cat output_file.json
jq -r .upload_id
upload_id=8f304c6d-804b-440a-xxxx**
[Pipeline] sh
echo upload_id : null
upload_id : null
[Pipeline] }
I would strongly recommend to go with Matt's answer, because it is the cleanest way.
Anyway, there are situations where there is no other choice than to use the shell, so here is the shell way:
script {
def upload_id = sh label: '',
script: 'echo $(cat output_file.json | jq -r \'.upload_id\')',
returnStdout: true
upload_id = upload_id.trim() // remove extraneous whitespace
sh "echo \"$upload_id\""
}
I've already linked to a more detailed answer of mine but you were probably not getting it to work, because you are using a declarative pipeline. In declarative pipeline, you have to use a script block to be able to store return values of steps.
You can easily do this intrinsically in Jenkins Pipeline to avoid all of the issues with subprocess execution. First, read in and parse the JSON file:
upload_info = readJSON(file: 'output_file.json')
Then, you can access the returned values in the assigned upload_info Map normally:
upload_id = upload_info['upload_id']
I am trying to execute set of commands from jenkinsfile.
The problem is, when I try to assign the value of stdout to a variable it is not working.
I tried different combinations of double quotes and single quotes, but so far no luck.
Here I executed the script with latest version of jenkinsfile as well as old version. Putting shell commands inside """ """ is not allowing to create new variable and giving error like client_name command does not exist.
String nodeLabel = env.PrimaryNode ? env.PrimaryNode : "slave1"
echo "Running on node [${nodeLabel}]"
node("${nodeLabel}"){
sh "p4 print -q -o config.yml //c/test/gradle/hk/config.yml"
def config = readYaml file: 'devops-config.yml'
def out = sh (script:"client_name=${config.BasicVars.p4_client}; " +
'echo "client name: $client_name"' +
" cmd_output = p4 clients -e $client_name" +
' echo "out variable: $cmd_output"',returnStdout: true)
}
I want to assign the stdout from the command p4 clients -e $client_name to variable cmd_output.
But when I execute the code the error that is thrown is:
NoSuchPropertyException: client_name is not defined at line cmd_output = p4 clients -e $client_name
What am I missing here?
Your problem here is that all the $ are interpreted by jenkins when the string is in double quotes. So the first 2 times there's no problem since the first variable comes from jenkins and the second time it's a single quote string.
The the third variable is in a double quote string, therefore jenkins tries to replace the variable with its value but it can't find it since it's generated only when the shell script is executed.
The solution is to escape the $ in $client_name (or define client_name in an environment block).
I rewrote the block:
String nodeLabel = env.PrimaryNode ? env.PrimaryNode : "slave1"
echo "Running on node [${nodeLabel}]"
node("${nodeLabel}"){
sh "p4 print -q -o config.yml //c/test/gradle/hk/config.yml"
def config = readYaml file: 'devops-config.yml'
def out = sh (script: """
client_name=${config.BasicVars.p4_client}
echo "client name: \$client_name"
cmd_output = p4 clients -e \$client_name
echo "out variable: \$cmd_output"
""", returnStdout: true)
}
I am trying to access an env variable in Jenkins pipeline and want to use it in a Shell Script executing in the same pipeline but a differnt step,
pipeline {
agent any
tools {
maven 'M2'
}
environment {
stable_revision = sh(script: 'curl -H "Authorization: Basic $base64encoded" "https://xyz.sad" | jq -r "name"', returnStdout: true)
}
stages {
stage('Initial-Checks') {
steps {
echo "Stable Revision: ${env.stable_revision}" //displays the value
bat "sh && sh undeploy.sh"
}}
...
}}
This is a sample Shell script, it has many lines, but I have an issue in only accessing the above stable_revision variable,
#!/bin/bash
echo xyz = ${stable_revision} #### this gives the right value xyz = 22
echo xyz2 = ${stable_revision}/d ### here after the value the /d is written in new line
For example, let's say the stable_revision value is 22, then in the SH script echo I am getting the value as,
xyz2 = 22
/d
I want the value to be xyz2 = 22/d
You can use .trim() to strip off a trailing newline.
environment {
stable_revision = sh(script: 'curl -H "Authorization: Basic $base64encoded" "https://xyz.sad" | jq -r "name"', returnStdout: true).trim()
}
returnStdout (optional):
If checked, standard output from the task is returned as the step value as a String, rather than being printed
to the build log. (Standard error, if any, will still be printed to
the log.) You will often want to call .trim() on the result to strip
off a trailing newline.
https://jenkins.io/doc/pipeline/steps/workflow-durable-task-step/#sh-shell-script
If you use bash instead of sh for your commands, you can benefit from Bash's built-in string transformations
Here it trims all trailing characters from the [:space:] class witch includes actual spaces and newlines.
echo "xyz2 = ${stable_revision%%[[:space:]]}/d"
If $stable_revision is always an integer, you can force the shell to use it like an integer with:
echo "xyz2 = $((stable_revision))/d"
If you are sure that $stable_revision contains no space, you can force the shell to trim all spaces by using it like a table element:
sr=($stable_revision); echo "xyz2 = ${sr[0]}/d"
You can also use the automatic trimming of a sub-shell returned value, that would trim any leading, trailing and duplicate spaces in-between:
echo "xyz2 = $(echo ${stable_revision})/d"`