I'm trying to implement docker with jenkins and am not sure if I am on the right track.
Given:
Running jenkins on docker from Windows
Plan on fetching code from github, building the solution, running functional tests, etc on a container somehow
What I've currently done:
(1) Installed Docker on Windows
(2) Successfully launched Jenkins on Docker with the command
"docker run –name myJenkins -p 8080:8080 -p 50000:50000 -v ~/Jenkins:/var/jenkins_home/ jenkins/jenkins:lts"
I believe this step binds the docker volume to my host machine's directory. This allows me to view and access the Jenkins content.
(3) In my host machine's Jenkins directory, I've created a plugin.txt (containing a variety of Jenkins plugins I want installed) and a Dockerfile. The Dockerfile installs the specified plugins in the plugins.txt file.
FROM jenkins/jenkins:lts
COPY plugins.txt /usr/share/jenkins/ref/plugins.txt
RUN /usr/local/bin/install-plugins.sh < /usr/share/jenkins/ref/plugins.txt
(4) In the windows command prompt, I built the Dockerfile with the command "docker build -t new_jenkins_image ."
(5) I stop my current container "myJenkins" and create a new container with the command "docker run –name myJenkins2 -p 8080:8080 -p 50000:50000 -v ~/Jenkins:/var/jenkins_home/ new_jenkins_image". This loads up Jenkins with the newly installed jenkins plugins.
What I'm stuck/confused on
(1) Do I have to create a new container with a new name every time I want to install new jenkins plugins through the Dockerfile? This seems like a manual process as well... There has to be a better way.
(2) I started a basic jenkins pipeline job with the "Pipeline script from SCM" option. I entered in the correct repository URL and credentials but left the "Script Path" blank for now (I do not have a Jenkinsfile yet). When I execute the build, Jenkins did not fetch the code from github.
java.lang.IllegalArgumentException: Empty path not permitted.
at org.eclipse.jgit.treewalk.filter.PathFilter.create(PathFilter.java:80)
at org.eclipse.jgit.treewalk.TreeWalk.forPath(TreeWalk.java:205)
at org.eclipse.jgit.treewalk.TreeWalk.forPath(TreeWalk.java:249)
at org.eclipse.jgit.treewalk.TreeWalk.forPath(TreeWalk.java:281)
at jenkins.plugins.git.GitSCMFile$3.invoke(GitSCMFile.java:171)
at jenkins.plugins.git.GitSCMFile$3.invoke(GitSCMFile.java:165)
at jenkins.plugins.git.GitSCMFileSystem$3.invoke(GitSCMFileSystem.java:193)
at org.jenkinsci.plugins.gitclient.AbstractGitAPIImpl.withRepository(AbstractGitAPIImpl.java:29)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.withRepository(CliGitAPIImpl.java:72)
at jenkins.plugins.git.GitSCMFileSystem.invoke(GitSCMFileSystem.java:189)
at jenkins.plugins.git.GitSCMFile.content(GitSCMFile.java:165)
at jenkins.scm.api.SCMFile.contentAsString(SCMFile.java:338)
at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:110)
at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:67)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:293)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Finished: FAILURE
I believe it's because the docker container does not have git installed? The container cannot access the Git or MSBuild from my host machine... Do I have to create a new container here to simply fetch the code?
Can someone explain to me what I'm missing or where I went wrong?
From my understanding, the process goes like this: Create new pipeline job -> select pipeline script from scm -> enter repo URL, credentials, branch to build and Jenkinsfile -> Jenkinsfile will execute instructions to compile, test, and deploy.
Where does the Dockerfile come into play here? Is my thought process on the right track?
you need to create container every time if you change/update the image. but it not required to give a new name each time. Did you stopped and removed previously running container? if not so docker gives errors like same name container cannot start. So stop and remove your previous container. and you will be able to start a new container with an updated image.
Yes, you need to install git in the same container to pull the code. it cannot access git on the host machine. But the error you are showing is like validation error. (i mean Jenkins validates input even before trying to pull the code. if you add some fake name it will throw next error like git not found)
Your thought is on correct track. Create new pipeline job -> select pipeline script from scm -> enter repo URL, credentials, branch to build and Jenkinsfile -> Jenkinsfile will execute instructions to compile, test, and deploy.
At the end of the question, you mentioned about different Dockerfile, i assume you are talking about Dockerfile in your repository (git). you can run your pipeline in docker agent. This removes to setup everything on jenkins host means you dont need to install dependencies to run your pipeline code on host, for example, if you are trying to execute some nodejs code in pipe, you need to setup nodejs on Jenkins host before you run the pipe, to get rid of this you can run pipe in container where everything is pre-setup. But I don't think you can use this feature if you are running Jenkins itself in docker. you need to setup Jenkins on the host directly in that case.
Related
My goal is to run a docker container that will simulate a physical device that sends requests to an external API.
The simulator lives in a GitHub repo and after cloning it is run by bash scripts that:
Set up a virtualenv.
Run maven build.
Execute other commands via Python that runs a docker container.
It can also remove docker container.
How can I simplify these dependencies and make sure everything is packed in a working container (application that otherwise would be run through shell scripts)?
I'm trying to use the Docker plugin in Jenkins to commit a docker container when the build running on it fails. Currently I have a Jenkins server with ~15 nodes, each with its own docker cloud. The nodes all have the latest version of docker-ce installed. I have a build set up to run on a docker container. What I want to do is commit the container when the build fails. Below are the things I have tried:
Adding a post build task, where I obtain the container ID and the hostname of the node running the container. I then SSH into the node and then commit the container.
The problem: Not able to SSH from inside a container as it requires a password and there's no way adding the node to the container's list of known hosts
Checking the "commit container" box in the build's general configurations
The problem: This is probably working but I don't know where the container is being committed to. Also this happens every time, and not just when the build fails.
Using the build script
Same problem as using the post build task
Execute a docker command (Build step)
This option asks for the container ID, which I have no way of knowing as it is new every time a build is run.
Please let me know if I have misunderstood any of the above ways! I am still new to Jenkins and Docker so I am learning as I go. :)
I am using the Jenkins Azure VM Agents Plugin with a Linux Master, to launch jobs on Windows agents.
I have been through all the configuration steps and everything works fine until I try to use Docker on the agents.
My pipeline script:
pipeline {
agent {
docker {
image 'myurl.io/myimage:latest'
registryUrl 'https://myurl.io/'
registryCredentialsId '123456789abcdefg'
}
}
The pipeline appears to fails on when it runs this command:
docker pull myurl.io/myimage:latest
The error reported comes down to this:
Caused: java.io.IOException: Cannot run program "nohup" (in directory "C:\Jenkins\workspace\Test Pipeline Docker"): CreateProcess error=2, The system cannot find the file specified
Some notes:
I have ticked the box to install git on the image:
The Git tools appear to be successfully installed on the agent VM
This question seems to be related but is it not exactly the same
I am not running the sh command directly, it is being run by the plugin.
I do not think I have access to set the PATH at this stage
This issue on JIRA https://issues.jenkins-ci.org/browse/JENKINS-36776 is related, but it does not seems to be fixed and the suggested workarounds don't seem to apply to my situation
My question
Is there a way to get my pipeline script to work? Maybe there are some extra commands I can somehow execute on the agent after it launches - but before the docker pull command - to add the required directories to the PATH?
Or is there some other workaround?
I think you were on the right track with the question you already found:
Jenkins pipeline sh fail with "cannot run program nohup" on windows
But, according to the wiki page of the docker-pipeline plugin, running docker on windows workers is not supported (a bit hidden though...):
For Jenkins environments which have macOS, Windows, or other agents, which are unable to run the Docker daemon, this default setting may be problematic. [https://www.jenkins.io/doc/book/pipeline/docker/#specifying-a-docker-label]
As far as I can see, there were several tries to add that feature, but it doesn't seem to be added (yet): https://github.com/jenkinsci/docker-workflow-plugin/pull/148
In the last link it is also stated, that fixing the sh/nohup issue will not be your only problem, for example the docker plugin will try to run id to get the user.
Nevertheless, you could try to make linux commands available by editing the path in your pipeline declaration:
https://stackoverflow.com/a/45101214/12338776
EDIT:
Just saw this question is 3 years old... Well. But since there was no answer so far, and a lot of people still seem to get here, it might still help someone.
I'm using the current Jenkins Maven Project tutorial using Docker:
https://jenkins.io/doc/tutorials/build-a-java-app-with-maven/
I keep getting this error at the Build stage:
[simple-java-maven-app] Running shell script
sh: can't create
/var/jenkins_home/workspace/simple-java-maven-app#tmp/durable-bae402a9/jenkins-log.txt:
nonexistent directory
sh: can't create
/var/jenkins_home/workspace/simple-java-maven-app#tmp/durable-bae402a9/jenkins-result.txt:
nonexistent directory
I've tried setting least restrictive permissions with chmod -R 777, chown -R nobody and chown -R 1000 on the listed directories, but nothing seems to work.
This is happening with the jenkins image on Docker version 17.12.0-ce, build c97c6d6 on Windows 10 Professional.
As this is happening with the Maven project tutorial on the Jenkins site, I'm wondering how many others have run into this issue.
I had also the same problem on MacOSX.
After few hours of research, I have finally find the solution.
To solve the problem, it's important to understand that Jenkins is inside a container and when the docker agent inside this container talk to your docker engine, it give path to mount volume matching inner the container. But your docker engine is outer. So to allow to work correctly path inner the container must match the same path outer the container in your host.
To allow working correctly, you need to change 2 things.
docker run arguments
Jenkinsfile docker agent arguments
For my own usage, I used this
docker run -d \
--env "JENKINS_HOME=$HOME/Library/Jenkins" \
--restart always \
--name jenkins \
-u root \
-p 8080:8080 \
-v /var/run/docker.sock:/var/run/docker.sock \
-v $HOME/Library/Jenkins:$HOME/Library/Jenkins \
-v "$HOME":/home \
jenkinsci/blueocean
In the Jenkinsfile
Replace the agent part
agent {
docker {
image 'maven:3-alpine'
args '-v /root/.m2:/root/.m2'
}
by
agent {
docker {
image 'maven:3-alpine'
args '-v <host_home_path>/.m2:/root/.m2'
}
It's quite likely that this issue resulted from a recent change in Docker behaviour, which was no longer being handled correctly by the Docker Pipeline plugin in Jenkins.
Without going into too much detail, the issue was causing Jenkins to no longer be able to identify the container it was running in, which results in the errors (above) that you encountered with these tutorials.
A new version (1.15) of the Docker Pipeline plugin was released yesterday (https://plugins.jenkins.io/docker-workflow).
If you upgrade this plugin on your Jenkins (in Docker) instance (via Manage Jenkins > Manage Plugins), you'll find that these tutorials should start working again (as documented).
The error message means that the directory durable-bae402a9 was not created.
Walk back through the tutorial to find the step that should have created that directory, and make whatever changes are needed to make sure it succeeds.
I'm new to Jenkins and I have been searching around but I couldn't find what I was looking for.
I'd like to know how to run docker command in Jenkins (Build - Execute Shell):
Example: docker run hello-world
I have set Docker Installation for "Install latest from docker.io" in Jenkins Configure System and also have installed several Docker plugins. However, it still didn't work.
Can anyone help me point out what else should I check or set?
John
One of the following plugins should work fine:
CloudBees Docker Custom Build Environment Plugin
CloudBees Docker Pipeline Plugin
I normally run my builds on slave nodes that have docker pre-installed.
I came across another generic solution. Since I'm not expert creating a Jenkins-Plugin out of this, here the manual steps:
Create/change your Jenkins (I use portainer) with environment variables DOCKER_HOST=tcp://192.168.1.50 (when using unix protocol you also have to mount your docker socket) and append :/var/jenkins_home/bin to the actual PATH variable
On your docker host copy the docker command to the jenkins image "docker cp /usr/bin/docker jenkins:/var/jenkins_home/bin/"
Restart the Jenkins container
Now you can use docker command from any script or command line. The changes will persist an image update.