I have Jenkins Pipeline that will deploy a pod with a container using my ruby's application image, and there is a stage that will run a rake command from a $COMMAND parameter like this:
rake db_command:count_data[histories,status IN ("active","pending")] inside the container.
this is the step I wrote on my stage to run the rake command from a $COMMAND parameter:
stage("Rake Task") {
try {
container("ruby-rake-task") {
sh '''
cd /var/www/mos
${COMMAND}
'''
}
} catch(e) {
echo "rake failed"
sh "exit 1"
}
}
but I get this error from console output jenkins:
+ cd /var/www/mos
+ rake 'db_command:count_data[histories,status' IN '("active","pending")]'
rake aborted!
Don't know how to build task 'db_command:count_data[histories,status' (See the list of available tasks with `rake --tasks`)
/usr/local/bundle/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
(See full trace by running task with --trace)
it seems the jenkins or groovy won't execute the rake command as a whole command, and instead it split the command to three parts. Jenkins is somehow always give a quote for all parameters after 'rake' command this really hard when the parameter is having spaces like above.
I already try adjust my stage step like this:
stage("Rake Task") {
try {
container("ruby-rake-task") {
sh "cd /var/www/mos"
def rakeTask = $/eval ${COMMAND}/$
resultRake = sh(script: "${rakeTask}", returnStdout: true).toString()
echo "${resultRake}"
}
} catch(e) {
echo "rake failed"
sh "exit 1"
}
}
yet the result is produce another error like this:
[Pipeline] stage
[Pipeline] { (Rake Task)
[Pipeline] container
[Pipeline] {
[Pipeline] sh
+ cd /var/www/mos
[Pipeline] sh
/home/jenkins/agent/workspace/RUBY-APP-RAKE-TASK#tmp/durable-c0e8d0d4/script.sh: line 1: syntax error: unexpected "("
[Pipeline] }
[Pipeline] // container
what do I need to do to be able run this rake task as a whole command included the spaces parameter? can someone tell me what am I missing here?
Thank you!
I'm using a node container with entrypoint "bash" and resolved errors like your.
I adapt it for you:
"-c \"cd /var/www/mos && ${command}\""
I hope it helps.
Related
I'm running a very simple pipeline to create maven projects, the pipeline is to run a single maven install command.
The trouble is substituting variables into the one-liner. I've tried various different things but stuck in a strange place. Take example pipeline below:
node {
stage('Set Vars') {
GIT_HASH = sh (script: "git log -n 1 --pretty=format:'%H'" , returnStdout: true)
echo "git hash is >>>>>>>> $GIT_HASH"
BUILD_NUM="${env.BUILD_NUMBER}"
echo "build no is >>>>>>>> ${BUILD_NUM}"
}
stage('Build Project') {
sh "mvn clean install -PrunInstallPackage -Duser=admin -Dpass=guessing -Dip=200.0.0.1 -Dport=4444 -Dhash=${GIT_HASH} -Dbuildnumber=${BUILD_NUM}"
}
}
I would expect to see the variables substitued in the mvn install command, this does not seem to work this way though.
This build results in:
sh "mvn clean install -PrunInstallPackage -Duser=admin -Dpass=guessing -Dip=200.0.0.1 -Dport=4444 -Dhash=${GIT_HASH}
Both echo commands print out correct output.
Why does the last command get cut off after the first variable substitution?
Also, for some unknown reason, I cannot substitute -Dbuildnumber=${env.BUILD_NUMBER}" directly into the maven command. Seems like something that a user should be able to do. What am I missing here?
Many thanks in advance
I am trying to remember how I solved same issues in the past..
Problem 1
You are using the GIT_HASH variable across two stages, so you have to declare it global to share it across them:
// Global variables declaration
GIT_HASH = null
// Pipeline code
node {
stage('Set Vars') {
GIT_HASH = sh (script: "git log -n 1 --pretty=format:'%H'" , returnStdout: true)
echo "git hash is >>>>>>>> $GIT_HASH"
BUILD_NUM="${env.BUILD_NUMBER}"
echo "build no is >>>>>>>> ${BUILD_NUM}"
}
stage('Build Project') {
sh "mvn clean install -PrunInstallPackage -Duser=admin -Dpass=guessing -Dip=200.0.0.1 -Dport=4444 -Dhash=${GIT_HASH} -Dbuildnumber=${BUILD_NUM}"
}
}
Problem 2
env.BUILD_NUMBER is a Groovy statement, instead mvn gets executed inside a shell instance by the sh command.
For that reason I suggest you to use the BUILD_NUM variable way.
I'm running a declarative Jenkins pipeline on Ubuntu 18.04 slave. My issue is that whenever I'm trying to provide arguments from a sh step to a bash script, the arguments are not there. When running the exact same commands from either a terminal directly or another script file (similarly how Jenkins does it via temp file) the arguments work fine.
The Jenkinsfile looks something like
pipeline {
agent { label "ubuntu" }
options { timeout(time: 1, unit: 'HOURS') }
stages {
stage('Build') {
steps {
sh """
#!/bin/bash
...
. ./Scripts/install_tools.sh "force"
"""
}
}
}
}
The pipeline itself runs smooth and does what I need it to do. The problem is that when calling install_tools.sh no arguments are found. The script looks something like
#!/bin/bash
echo "Running $0"
echo "Args: $#"
...
The line echo "Args: $#" I have tried also with $* and $1 - Each time the arguments are returned as empty, but only when running from the pipeline. It seems to me that this is related to some Groovy stuff, but I have no clue what.
How do I call a bash script during the pipeline and get the arguments passed properly?
put #!/bin/bash into the first line
sh """#!/bin/bash
. ./Scripts/install_tools.sh "force"
"""
otherwise . (dot) command could have different meaning
I've done a lot of Android dev never needing to understand gradle well.
Ideally I'd be running a python script after setting up a virtualenv, but for this question I'm happy with a MWE.
task rem(type: Exec) {
doLast {
exec {
workingDir '.'
commandLine 'dir'
}
}
}
Results in:
> Task :app:rem FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':app:rem'.
> execCommand == null!
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 2s
1 actionable task: 1 executed
Let's say I go back to how I was declaring my tasks before shelling out:
task rem {
without (type: Exec) I get:
> A problem occurred starting process 'command 'dir''
where before I had
> execCommand == null!
I have only got the following commandLine to work:
task rem(type: Exec) {
commandLine 'python', '--version'
}
replacing body with commandLine 'dir' fails with
...
> A problem occurred starting process 'command 'dir''
...
Eventually stumbled on https://stackoverflow.com/a/37871837/866333 for running windows scripts and found chaining, https://stackoverflow.com/a/50394682/866333, more easily.
task rem(type: Exec) {
commandLine 'cmd', '/c', "cd && virtualenv --python=\"C:\Program Files\Python38\python.exe\" C:\Users\XXX\AndroidStudioProjects\YYY\virtualenvs\ZZZ"
}
Had good success with the above snippet (using double quotes so I could substitute XXX YYY ZZZ from earlier computations. It isn't pretty but the python script will be.
Hopefully this will save another gradle newbie the hours I needed.
Note the cd to begin the (nominal) chain. I was not in the directory I expected.
I have the following in a Jenkinsfile as a discrete stage:
stage("Check if Postman scripts are already cloned, delete") {
steps {
sh "[ -d postman-scripts ] && rm -rf postman-scripts && echo 'Directory cleared.'"
}
}
However when I look at my Jenkins pipeline it looks like it fails and doesn't run the entire script:
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Check if Postman scripts are already cloned, delete)
[Pipeline] sh
+ '[' -d postman-scripts ']'
[Pipeline] }
Is that because it's not finding the postman-scripts directory, so it's not executing the rest of the script? And is that being counted as a build failure? It skips all following stages, so logically I think it's doing that.
Also any help with the bash scripting would be nice, I'm not entirely sure how I can string this all together on one line to do what I want (IF directory exists, remove directory. Otherwise do nothing).
Your shell test condition [ -d postman-scripts ] will FAIL if the directory does not exist. That is a failure like any other command, so it exits. #jordanm is correct, using -f will return success even if nothing exists to remove, so your next step will continue. Exception if unable to remove files because of a system lock or similar, which is probably a good thing,
My Jenkins build job indicates success but none of the scripts I am executing seem to do anything. I put a simple "pwd" command to echo current working directory but nothing happens. All Jenkins reports is "Running shell script" and then nothing is seen.
Snippet of Jenkins log:
First time build. Skipping changelog.
[Pipeline] sh
[an_example_test3-26UQSYQ445FSBP4QRKEWLPQCJH545MLQVD6B552CIGPOXLZO4G5A] Running shell script
[Pipeline] sh
[an_example_test3-26UQSYQ445FSBP4QRKEWLPQCJH545MLQVD6B552CIGPOXLZO4G5A] Running shell script
[Pipeline] sh
[an_example_test3-26UQSYQ445FSBP4QRKEWLPQCJH545MLQVD6B552CIGPOXLZO4G5A] Running shell script
[Pipeline] sh
[an_example_test3-26UQSYQ445FSBP4QRKEWLPQCJH545MLQVD6B552CIGPOXLZO4G5A] Running shell script
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
Jenkinsfile that is used to define the builds:
node {
stage 'build'
sh "echo ${env.BRANCH_NAME}"
git url: 'git#hd1:testing', branch: "${env.BRANCH_NAME}"
sh "ls -rtl"
sh "pwd"
sh "csh ${workspace}/simple.csh"
sh "csh ${workspace}/source.csh"
}