I am using the Jenkins Azure VM Agents Plugin with a Linux Master, to launch jobs on Windows agents.
I have been through all the configuration steps and everything works fine until I try to use Docker on the agents.
My pipeline script:
pipeline {
agent {
docker {
image 'myurl.io/myimage:latest'
registryUrl 'https://myurl.io/'
registryCredentialsId '123456789abcdefg'
}
}
The pipeline appears to fails on when it runs this command:
docker pull myurl.io/myimage:latest
The error reported comes down to this:
Caused: java.io.IOException: Cannot run program "nohup" (in directory "C:\Jenkins\workspace\Test Pipeline Docker"): CreateProcess error=2, The system cannot find the file specified
Some notes:
I have ticked the box to install git on the image:
The Git tools appear to be successfully installed on the agent VM
This question seems to be related but is it not exactly the same
I am not running the sh command directly, it is being run by the plugin.
I do not think I have access to set the PATH at this stage
This issue on JIRA https://issues.jenkins-ci.org/browse/JENKINS-36776 is related, but it does not seems to be fixed and the suggested workarounds don't seem to apply to my situation
My question
Is there a way to get my pipeline script to work? Maybe there are some extra commands I can somehow execute on the agent after it launches - but before the docker pull command - to add the required directories to the PATH?
Or is there some other workaround?
I think you were on the right track with the question you already found:
Jenkins pipeline sh fail with "cannot run program nohup" on windows
But, according to the wiki page of the docker-pipeline plugin, running docker on windows workers is not supported (a bit hidden though...):
For Jenkins environments which have macOS, Windows, or other agents, which are unable to run the Docker daemon, this default setting may be problematic. [https://www.jenkins.io/doc/book/pipeline/docker/#specifying-a-docker-label]
As far as I can see, there were several tries to add that feature, but it doesn't seem to be added (yet): https://github.com/jenkinsci/docker-workflow-plugin/pull/148
In the last link it is also stated, that fixing the sh/nohup issue will not be your only problem, for example the docker plugin will try to run id to get the user.
Nevertheless, you could try to make linux commands available by editing the path in your pipeline declaration:
https://stackoverflow.com/a/45101214/12338776
EDIT:
Just saw this question is 3 years old... Well. But since there was no answer so far, and a lot of people still seem to get here, it might still help someone.
Related
I am trying to setup a Jenkins pipeline using scripted pipeline with a windows 2019 server, however I ran into this error while trying to build
> webpack --config ./config/webpack-cli-prod.config.js
C:\myProject\node_modules\webpack\lib\javascript\JavascriptModulesPlugin.js:143
throw new TypeError(
^
TypeError: The 'compilation' argument must be an instance of Compilation
at Function.getCompilationHooks (C:\myProject\node_modules\webpack\lib\javascript\JavascriptModulesPlugin.js:143:10)
at SourceMapDevToolModuleOptionsPlugin.apply (C:\myProject\node_modules\webpack\lib\SourceMapDevToolModuleOptionsPlugin.js:50:27)
at C:\myProject\node_modules\webpack\lib\SourceMapDevToolPlugin.js:163:53
at Hook.eval [as call] (eval at create (C:\myProject\node_modules\tapable\lib\HookCodeFactory.js:19:10), <anonymous>:100:1)
at Hook.CALL_DELEGATE [as _call] (C:\myProject\node_modules\tapable\lib\Hook.js:14:14)
at Compiler.newCompilation (C:\myProject\node_modules\webpack\lib\Compiler.js:1122:26)
at C:\myProject\node_modules\webpack\lib\Compiler.js:1166:29
at Hook.eval [as callAsync] (eval at create (C:\myProject\node_modules\tapable\lib\HookCodeFactory.js:33:10), <anonymous>:6:1)
at Hook.CALL_ASYNC_DELEGATE [as _callAsync] (C:\myProject\node_modules\tapable\lib\Hook.js:18:14)
at Compiler.compile (C:\myProject\node_modules\webpack\lib\Compiler.js:1161:28)
I tried to run same command/step using freestyle Jenkins job, and it works without this error.
I tried to run the same command on the Jenkins agent locally, and it works without this error.
I looked up on google, and came across this link here, I tried to use newer version of html-webpack-plugin, and we also tried to build without the plugin. All come to the same result, that the error would occur only when running from Jenkins scripted pipeline.
I also tried with a different server, while keeping the same agent and job configuration, and I also get the same error.
The version of npm is 8.11.0, the node version is 16.16.0. The agent is connected through running the agent.jar file from the agent.
The only difference I see between the freestyle job and the scripted pipeline job is the freestyle job appears to be run as SYSTEM by the Jenkins server, whereas the pipeline job is probably run with lower privilege (I am not entirely sure though). I saw also this post, where it says
in the Freestyle job everything is executed in the agent, but for the Scripted Pipeline Job, the pipeline code is translated in the controller to atomic commands that are sent to the agents.
But I have no idea how to make the scripted pipeline job run just like the freestyle job.
On one hand, it appears to have to do with webpack, and on the other it appears to be related to Jenkins since running freestyle and locally on the server is without errors.
This is how my Jenkins scripted pipeline looks like (with sensitive information removed)
node("My-Server"){
dir("C:\\MyProject"){
stage('Pre-Test Build Client (Web)') {
dir("aFolder"){
bat 'npm run build-all-prod' // This is the script that invoke the webpack build command
}
}
}
}
I have run out of options, and do not where to go from here, and I couldn't find any more information on google that would be helpful. Any help here would be really appreciated. Thank you.
I'm not sure if jenkins creates the same environment variables, and command line tools (you can configure some on the node's configuration page). I would check if the node's environment variables and tools are the same in freestyle vs pipeline job by running something like this in each job, and comparing the output:
bat 'echo %PATH%'
bat 'which webpack'
bat 'npm list webpack'
Another thing worth checking is whether you're using batch script in both jobs, and not eg shell in freestyle.
Lastly I found a gh issue whith the same error as you, caused by having 2 different installations of webpack, one on v5. May be worth looking into, if everything else fails.
I have a couple bash scripts to automate the setup process for a new devices e.g. installing packages, configuring environment variables, etc.
I'm working on making the process more automated with autoexpect, and adding a few thing other things; however, it's difficult to test since every time I run the install script I have to manually go back an undo the changes that were made from running the script. Is there a way to run the scripts without actually installing anything so I can observe the behaviour for testing? something like the --dry-run option with rsync
for configuring your machine and being able to test this quickly and knowing you won't cause problems to your PC locally, create a VM using Virtual box or VMwWare player and then snapshot the VM so you can revert back to the state before you run the script, and then you can run your script on this VM, and check what configuration has been applied successfully.
I'm new to Jenkins and I have been searching around but I couldn't find what I was looking for.
I'd like to know how to run docker command in Jenkins (Build - Execute Shell):
Example: docker run hello-world
I have set Docker Installation for "Install latest from docker.io" in Jenkins Configure System and also have installed several Docker plugins. However, it still didn't work.
Can anyone help me point out what else should I check or set?
John
One of the following plugins should work fine:
CloudBees Docker Custom Build Environment Plugin
CloudBees Docker Pipeline Plugin
I normally run my builds on slave nodes that have docker pre-installed.
I came across another generic solution. Since I'm not expert creating a Jenkins-Plugin out of this, here the manual steps:
Create/change your Jenkins (I use portainer) with environment variables DOCKER_HOST=tcp://192.168.1.50 (when using unix protocol you also have to mount your docker socket) and append :/var/jenkins_home/bin to the actual PATH variable
On your docker host copy the docker command to the jenkins image "docker cp /usr/bin/docker jenkins:/var/jenkins_home/bin/"
Restart the Jenkins container
Now you can use docker command from any script or command line. The changes will persist an image update.
I have Jenkins running on Windows, and I have a build that works fine under CygWin bash from the CygWin terminal, so I now want to automate it. However, using this script:
#!C:\cygwin\bin\bash.exe
whoami
make
The system reports me as nt authority\system, not the ken that I get when using an interactive shell. Is there an easy way to persuade Jenkins or CygWin to run as me?
Most likely you are running jenkins with default installation. You have two options. First is mentioned in the comment. Change the "Service account" to be same as yours.
Second option is derived from best practices. Run the jenkins master on a system with backup etc. Configure slave node with your account credentials. Change the project configuration to build on the specific node.
(It is possible to run slave and master on same machine with different credentials - just in case you want to try out things)
The real problem I was having was not that the shell script was running as the wrong user, but that the shell script was not executing the default /etc/profile. So, the solution was simply:
#!C:\cygwin\bin\bash.exe -l
whoami
make
I was still nt authority\system, but now I had the correct environment set up and could run make successfully.
Note also that if I create a /home/system directory I can add .bash_profile, etc, to that directory to further customise the build environment.
Here is my problem:
I have a python build script building a .air package and deploying it on a server. If I run this script from a cmd.exe window, there is no problem, the files are correctly created and deployed.
But I would like to use Bamboo to be able to build my app everytime I commit/push changes. I configured it but have a serious problem: the .air package is created but the deployment (with a fabric script) doesn't finish... I noticed that the process is launched by 'Administrator' in a cmd.exe and by SYSTEM trough Bamboo. I think that this is the problem (maybe SYSTEM cannot find the ssh host or whatever), so I wanted to know if someone knows how to change the user executing the script to 'Administrator'.
Yes, you can change the user that your bamboo server runs as. I recently posted an answer with details on how do just that in Windows Mercurial global keychain . All the documentation for the windows stuff in the wrapper file can found at http://wrapper.tanukisoftware.com/doc/english/props-nt.html .