Build inside docker container in jenkins - maven

I am using the docker-custom-build-environment-plugin (https://wiki.jenkins-ci.org/display/JENKINS/CloudBees+Docker+Custom+Build+Environment+Plugin) to build a maven job inside a docker container.
I have done the following setup as shown in this image:
This is the result in the console log:
$ docker inspect maven:3.3.3-jdk-8
[{
"Architecture": "amd64",
"Author": "",
"Comment": "",
"Config": {
"AttachStderr": false,
"AttachStdin": false,
"AttachStdout": false,
"Cmd": [
"mvn"
],
"CpuShares": 0,
"Cpuset": "",
"Domainname": "",
"Entrypoint": null,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"LANG=C.UTF-8",
"JAVA_VERSION=8u66",
"JAVA_DEBIAN_VERSION=8u66-b17-1~bpo8+1",
"CA_CERTIFICATES_JAVA_VERSION=20140324",
"MAVEN_VERSION=3.3.3",
"MAVEN_HOME=/usr/share/maven"
],
"ExposedPorts": null,
"Hostname": "",
"Image": "b7928b8a133284ec65b4790aa3d013bd3e266ea3e257c6a6d0254b9708a133e8",
"Labels": {},
"MacAddress": "",
"Memory": 0,
"MemorySwap": 0,
"NetworkDisabled": false,
"OnBuild": [],
"OpenStdin": false,
"PortSpecs": null,
"StdinOnce": false,
"Tty": false,
"User": "",
"Volumes": null,
"WorkingDir": ""
},
"Container": "7393955c8b148c5d2b6da0a5fdf5623aa699ce2ed9ab1f43c2ba79913654bf0b",
"ContainerConfig": {
"AttachStderr": false,
"AttachStdin": false,
"AttachStdout": false,
"Cmd": [
"/bin/sh",
"-c",
"#(nop) CMD [\"mvn\"]"
],
"CpuShares": 0,
"Cpuset": "",
"Domainname": "",
"Entrypoint": null,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"LANG=C.UTF-8",
"JAVA_VERSION=8u66",
"JAVA_DEBIAN_VERSION=8u66-b17-1~bpo8+1",
"CA_CERTIFICATES_JAVA_VERSION=20140324",
"MAVEN_VERSION=3.3.3",
"MAVEN_HOME=/usr/share/maven"
],
"ExposedPorts": null,
"Hostname": "",
"Image": "b7928b8a133284ec65b4790aa3d013bd3e266ea3e257c6a6d0254b9708a133e8",
"Labels": {},
"MacAddress": "",
"Memory": 0,
"MemorySwap": 0,
"NetworkDisabled": false,
"OnBuild": [],
"OpenStdin": false,
"PortSpecs": null,
"StdinOnce": false,
"Tty": false,
"User": "",
"Volumes": null,
"WorkingDir": ""
},
"Created": "2015-11-11T08:06:10.345733061Z",
"DockerVersion": "1.9.0",
"Id": "642c36d39ed754e55c15a382531bc60591154b7280356708dca833d346891415",
"Os": "linux",
"Parent": "b7928b8a133284ec65b4790aa3d013bd3e266ea3e257c6a6d0254b9708a133e8",
"Size": 0,
"VirtualSize": 651375578
}
]
$ docker run --rm --entrypoint /bin/true maven:3.3.3-jdk-8
$ docker run --tty --detach --workdir /example/apphome/jenkins/workspace/IntegrationTesting-Components --volume /tmp:/tmp:rw --volume /example/apphome/jenkins:/example/apphome/jenkins:rw --net bridge --add-host dockerhost:172.17.42.1 --env BUILD_DISPLAY_NAME=#1733 --env BUILD_ID=1733 --env BUILD_NUMBER=1733 --env BUILD_TAG=jenkins-IntegrationTesting-Components-1733 --env BUILD_URL=http://tools.example.com/jenkins/job/IntegrationTesting-Components/1733/ --env buildInfoConfig.propertiesFile=/tmp/buildInfo6572943319531125678.properties --env CLASSPATH= --env EXECUTOR_NUMBER=1 --env extractor.used=true --env GIT_BRANCH=origin/feature/java8 --env GIT_COMMIT=8569f85875d85c7181dcf9bf261819a1c55279cf --env GIT_PREVIOUS_COMMIT=8569f85875d85c7181dcf9bf261819a1c55279cf --env GIT_PREVIOUS_SUCCESSFUL_COMMIT=49617fb1a913a30c7111f60d2ace4f9aca0d8aa9 --env GIT_URL=ssh://git#git.example.com/int/components.git --env HUDSON_HOME=/example/apphome/jenkins --env HUDSON_SERVER_COOKIE=cb467bbe36d08ed4 --env HUDSON_URL=http://tools.example.com/jenkins/ --env JAVA_HOME=/usr/lib/jvm/java-1.7.0 --env JENKINS_HOME=/example/apphome/jenkins --env JENKINS_SERVER_COOKIE=cb467bbe36d08ed4 --env JENKINS_URL=http://tools.example.com/jenkins/ --env JOB_NAME=IntegrationTesting-Components --env JOB_URL=http://tools.example.com/jenkins/job/IntegrationTesting-Components/ --env M2_HOME=/example/apps/apache-maven-3.0.3 --env MAVEN_HOME=/example/apps/apache-maven-3.0.3 --env "MAVEN_OPTS=-D-Xms256m -Xmx1024m -XX:MaxPermSize=512m -Dm3plugin.lib=D:\data\Jenkins\plugins\artifactory\WEB-INF\lib" --env "NODE_LABELS=deployqueue docker linux maven jenkins-server" --env NODE_NAME=jenkins-server --env PATH+JDK=/usr/lib/jvm/java-1.7.0/bin --env PATH+MAVEN=/example/apps/apache-maven-3.0.3/bin --env POM_ARTIFACTID=components --env "POM_DISPLAYNAME=Integration Testing Components" --env POM_GROUPID=com.example.integrationtesting --env POM_PACKAGING=jar --env POM_VERSION=java8-SNAPSHOT --env WORKSPACE=/example/apphome/jenkins/workspace/IntegrationTesting-Components maven:3.3.3-jdk-8 /bin/cat
Docker container 66994ba8669ba42203e0ced67fcf16674bceaab2d0477e04ed92466eb05858e1 started to host the build
Parsing POMs
maven3-agent.jar already up to date
maven3-interceptor.jar already up to date
maven3-interceptor-commons.jar already up to date
$ docker exec --tty 66994ba8669ba42203e0ced67fcf16674bceaab2d0477e04ed92466eb05858e1 env
[IntegrationTesting-Components] $ docker exec --tty --user 501216: 66994ba8669ba42203e0ced67fcf16674bceaab2d0477e04ed92466eb05858e1 env BUILD_DISPLAY_NAME=#1733 BUILD_ID=1733 BUILD_NUMBER=1733 BUILD_TAG=jenkins-IntegrationTesting-Components-1733 BUILD_URL=http://tools.example.com/jenkins/job/IntegrationTesting-Components/1733/ buildInfoConfig.propertiesFile=/tmp/buildInfo6572943319531125678.properties CA_CERTIFICATES_JAVA_VERSION=20140324 CLASSPATH= EXECUTOR_NUMBER=1 extractor.used=true GIT_BRANCH=origin/feature/java8 GIT_COMMIT=8569f85875d85c7181dcf9bf261819a1c55279cf GIT_PREVIOUS_COMMIT=8569f85875d85c7181dcf9bf261819a1c55279cf GIT_PREVIOUS_SUCCESSFUL_COMMIT=49617fb1a913a30c7111f60d2ace4f9aca0d8aa9 GIT_URL=ssh://git#git.example.com/int/components.git HOME=/root HOSTNAME=66994ba8669b HUDSON_HOME=/example/apphome/jenkins HUDSON_SERVER_COOKIE=cb467bbe36d08ed4 HUDSON_URL=http://tools.example.com/jenkins/ JAVA_DEBIAN_VERSION=8u66-b17-1~bpo8+1 JAVA_HOME=/usr/lib/jvm/java-1.7.0 JAVA_VERSION=8u66 JENKINS_HOME=/example/apphome/jenkins JENKINS_SERVER_COOKIE=cb467bbe36d08ed4 JENKINS_URL=http://tools.example.com/jenkins/ JOB_NAME=IntegrationTesting-Components JOB_URL=http://tools.example.com/jenkins/job/IntegrationTesting-Components/ LANG=C.UTF-8 M2_HOME=/example/apps/apache-maven-3.0.3 MAVEN_HOME=/example/apps/apache-maven-3.0.3 "MAVEN_OPTS=-D-Xms256m -Xmx1024m -XX:MaxPermSize=512m -Dm3plugin.lib=D:\data\Jenkins\plugins\artifactory\WEB-INF\lib" MAVEN_VERSION=3.3.3 "NODE_LABELS=deployqueue docker linux maven jenkins-server" NODE_NAME=jenkins-server PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PATH+JDK=/usr/lib/jvm/java-1.7.0/bin PATH+MAVEN=/example/apps/apache-maven-3.0.3/bin POM_ARTIFACTID=components "POM_DISPLAYNAME=Integration Testing Components" POM_GROUPID=com.example.integrationtesting POM_PACKAGING=jar POM_VERSION=java8-SNAPSHOT TERM=xterm WORKSPACE=/example/apphome/jenkins/workspace/IntegrationTesting-Components /usr/lib/jvm/java-1.7.0/bin/java -D-Xms256m -Xmx1024m -XX:MaxPermSize=512m -Dm3plugin.lib=D:\data\Jenkins\plugins\artifactory\WEB-INF\lib -Djava.awt.headless=true -cp /example/apphome/jenkins/maven3-agent.jar:/example/apps/apache-maven-3.0.3/boot/plexus-classworlds-2.4.jar org.jvnet.hudson.maven3.agent.Maven3Main /example/apps/apache-maven-3.0.3 /example/apphome/jenkins/slave.jar /example/apphome/jenkins/maven3-interceptor.jar /example/apphome/jenkins/maven3-interceptor-commons.jar 44922
flag provided but not defined: --user
See 'docker exec --help'.
run PrePostClean
running on jenkins-server
clean on master
Stopping Docker container after build completion
$ docker kill 66994ba8669ba42203e0ced67fcf16674bceaab2d0477e04ed92466eb05858e1
66994ba8669ba42203e0ced67fcf16674bceaab2d0477e04ed92466eb05858e1
$ docker rm --force 66994ba8669ba42203e0ced67fcf16674bceaab2d0477e04ed92466eb05858e1
66994ba8669ba42203e0ced67fcf16674bceaab2d0477e04ed92466eb05858e1
ERROR: Failed to launch Maven. Exit code = 2
Any idea what i am doing wrong?

The advice from https://www.cloudbees.com/blog/templating-jenkins-build-environments-docker-containers on using their plugin is to base your slaves off evarga/jenkins-slave
The Dockerfile that you've based it off has some key differences such as the creation of the creation of the jenkins user and starting the sshd daemon. So if you're going to start off from your own image, base it off the evarga image first.

Related

Container is not accessible if the user is not logged in

I am using Docker Desktop on a Windows 10 client that is a member of an Active Directory domain. The users on our network want GitLab as a Docker container.
The Gitlab container runs on the Win 10 Client without any issues using the following command when i am logged in:
docker run `
--detach `
--hostname gitlab.companydomain.de `
--ip 192.168.178.35 `
--publish 443:443 --publish 80:80 --publish 22:22 `
--name gitlab `
--restart always `
--volume $env:GITLAB_HOME/config:/etc/gitlab `
--volume $env:GITLAB_HOME/logs:/var/log/gitlab `
--volume $env:GITLAB_HOME/data:/var/opt/gitlab `
--shm-size 256m `
gitlab/gitlab-ce:15.4.0-ce.0 `
My problem is the following: The GitLab container doesn't run if the system has started and i am not logged in.
I have tried several configurations that i found in the internet.
First, I registered dockerd as a service with the following command:
dockerd --register-service
Then I enabled ports 22, 80 and 443 for incoming and outgoing packets in the Windows firewall.
Then I tried to adapt the daemon.json file to my needs. But I don't know if I have set the values for hosts correctly or if some values are missing. Maybe the value "hosts": ["tcp://0.0.0.0:2375", "npipe://"] needs to be set to port 80 because the gitlab container is listening on port 80?
{
"allow-nondistributable-artifacts": [],
"authorization-plugins": [],
"bridge": "",
"cluster-advertise": "",
"cluster-store": "",
"data-root": "",
"debug": true,
"default-ulimits": {},
"dns": [],
"dns-opts": [],
"dns-search": [],
"exec-opts": [],
"experimental": false,
"features": {},
"fixed-cidr": "",
"group": "",
"hosts": ["tcp://0.0.0.0:2375","npipe://"]
"insecure-registries": [],
"labels": [],
"log-driver": "",
"log-level": "",
"max-concurrent-downloads": 3,
"max-concurrent-uploads": 5,
"max-download-attempts": 5,
"mtu": 0,
"pidfile": "",
"raw-logs": false,
"registry-mirrors": [],
"shutdown-timeout": 15,
"storage-driver": "",
"storage-opts": [],
"swarm-default-advertise-addr": "",
"tlscacert": "",
"tlscert": "",
"tlskey": "",
"tlsverify": false
}

Docker Image Download with download-frozen-image-v2.sh on Windows

I am working on downloading a Docker Image on an internet-connected Windows machine that does not have (and cannot have) Docker installed on it, to transfer to an non-internet-connected Linux machine that does have Docker. I'm using git-bash to run download-frozen-image-v2.sh. Everything is working as expected until the script begins to download the final layer of any given image. On the final layer the json file is being returned empty. Through echo statements, I'm able to see that everything is working flawlessly until lines 119-142
jq "$addJson + ." > "$dir/$layerId/json" <<-'EOJSON'
{
"created": "0001-01-01T00:00:00Z",
"container_config": {
"Hostname": "",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": null,
"Cmd": null,
"Image": "",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": null,
"OnBuild": null,
"Labels": null
}
}
EOJSON
Only on the final layer, this code is resulting in an empty json file, which in-turn creates an error in line 173
jq --raw-output "$imageOldConfig + del(.history, .rootfs)" "$dir/$configFile" > "$dir/$imageId/json"
jq: error: syntax error, unexpected '+', expecting $end (Windows cmd shell quoting issues?) at <top-level>, line 1:
+ del(.history, .rootfs)
jq: 1 compile error
Update
Exact steps to replicate
Perform on Windows 10 computer.
1) Install scoop for Windows https://scoop.sh/
2) in Powershell scoop install git curl jq go tar
3) git-bash
4) in git-bash curl -o download-frozen-image-v2.sh https://raw.githubusercontent.com/moby/moby/master/contrib/download-frozen-image-v2.sh
5) bash download-frozen-image-vs.sh ubuntu ubuntu:latest
The above will result in the aforementioned error.
in response to #peak below
The command I'm using is bash download-frozen-image-v2.sh ubuntu ubuntu:latest which should download 5 layers. The first 4 download flawlessly, it is only the last layer that fails. I tried this process for several other images, and it always fails on the final layer.
addJson:
{ id: "ee6b1042efee4fb07d2fe1a5079ce498567e6f5ac849413f0e623d4582da5bc9", parent: "80a2fb00dfe137a28c24fbc39fde656650cd68028d612e6f33912902d887b108" }
dir/configFile:
ubuntu/113a43faa1382a7404681f1b9af2f0d70b182c569aab71db497e33fa59ed87e6.json
dir/configFile contents:
{
"architecture": "amd64",
"config": {
"Hostname": "",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": [
"/bin/bash"
],
"ArgsEscaped": true,
"Image": "sha256:c2775c69594daa3ee360d8e7bbca93c65d9c925e89bd731f12515f9bf8382164",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": null,
"OnBuild": null,
"Labels": null
},
"container": "6713e927cc43b61a4ce3950a69907336ff55047bae9393256e32613a54321c70",
"container_config": {
"Hostname": "6713e927cc43",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": [
"/bin/sh",
"-c",
"#(nop) ",
"CMD [\"/bin/bash\"]"
],
"ArgsEscaped": true,
"Image": "sha256:c2775c69594daa3ee360d8e7bbca93c65d9c925e89bd731f12515f9bf8382164",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": null,
"OnBuild": null,
"Labels": {}
},
"created": "2018-06-05T21:20:54.310450149Z",
"docker_version": "17.06.2-ce",
"history": [
{
"created": "2018-06-05T21:20:51.286433694Z",
"created_by": "/bin/sh -c #(nop) ADD file:28c0771e44ff530dba3f237024acc38e8ec9293d60f0e44c8c78536c12f13a0b in / "
},
{
"created": "2018-06-05T21:20:52.045074543Z",
"created_by": "/bin/sh -c set -xe \t\t&& echo '#!/bin/sh' > /usr/sbin/policy-rc.d \t&& echo 'exit 101' >> /usr/sbin/policy-rc.d \t&& chmod +x /usr/sbin/policy-rc.d \t\t&& dpkg-divert --local --rename --add /sbin/initctl \t&& cp -a /usr/sbin/policy-rc.d /sbin/initctl \t&& sed -i 's/^exit.*/exit 0/' /sbin/initctl \t\t&& echo 'force-unsafe-io' > /etc/dpkg/dpkg.cfg.d/docker-apt-speedup \t\t&& echo 'DPkg::Post-Invoke { \"rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true\"; };' > /etc/apt/apt.conf.d/docker-clean \t&& echo 'APT::Update::Post-Invoke { \"rm -f /var/cache/apt/archives/*.deb /var/cache/apt/archives/partial/*.deb /var/cache/apt/*.bin || true\"; };' >> /etc/apt/apt.conf.d/docker-clean \t&& echo 'Dir::Cache::pkgcache \"\"; Dir::Cache::srcpkgcache \"\";' >> /etc/apt/apt.conf.d/docker-clean \t\t&& echo 'Acquire::Languages \"none\";' > /etc/apt/apt.conf.d/docker-no-languages \t\t&& echo 'Acquire::GzipIndexes \"true\"; Acquire::CompressionTypes::Order:: \"gz\";' > /etc/apt/apt.conf.d/docker-gzip-indexes \t\t&& echo 'Apt::AutoRemove::SuggestsImportant \"false\";' > /etc/apt/apt.conf.d/docker-autoremove-suggests"
},
{
"created": "2018-06-05T21:20:52.712120056Z",
"created_by": "/bin/sh -c rm -rf /var/lib/apt/lists/*"
},
{
"created": "2018-06-05T21:20:53.405342638Z",
"created_by": "/bin/sh -c sed -i 's/^#\\s*\\(deb.*universe\\)$/\\1/g' /etc/apt/sources.list"
},
{
"created": "2018-06-05T21:20:54.091704323Z",
"created_by": "/bin/sh -c mkdir -p /run/systemd && echo 'docker' > /run/systemd/container"
},
{
"created": "2018-06-05T21:20:54.310450149Z",
"created_by": "/bin/sh -c #(nop) CMD [\"/bin/bash\"]",
"empty_layer": true
}
],
"os": "linux",
"rootfs": {
"type": "layers",
"diff_ids": [
"sha256:db9476e6d963ed2b6042abef1c354223148cdcdbd6c7416c71a019ebcaea0edb",
"sha256:3a89e0d8654e098e949764b1cb23018e27f299b0931c5fd41c207d610ff356c4",
"sha256:904d60939c360b5f528b886c1b534855a008f9a7fd411d4977e09aa7de74c834",
"sha256:a20a262b87bd8a00717f3b30c001bcdaf0fd85d049e6d10500597caa29c013c5",
"sha256:b6f13d447e00fba3b9bd10c1e5c6697e913462f44aa24af349bfaea2054e32f4"
]
}
}
Any help in figuring out what is occurring here would be greatly appreciated.
Thank you.
I can't tell you why this happens but it appears to be a problem with how jq parses the input file. It's segfaulting when reading the file. It's a known issue in the windows builds where the problem is triggered by the length of the paths to the files.
Fortunately, there is a way around this issue by modifying the script to go against all conventional wisdom and cat the file to jq.
The script isn't utilizing jq very well and builds some of the json manually so some additional fixes would be needed. It will have errors regarding INVALID_CHARACTER when parsing. It's probably a manifestation of this issue since the script is manually building a lot of the jq programs.
I put up a gist with the updated file that at least doesn't error out, check to see if it works as expected.
Changes start at line 172 and 342.
The way it builds the manifest is just messy. I've cleaned it up a bit removing all the string interpolations instead passing all parameters in as arguments to jq.
# munge the top layer image manifest to have the appropriate image configuration for older daemons
local imageOldConfig="$(cat "$dir/$imageId/json" | jq --raw-output --compact-output '{ id: .id } + if .parent then { parent: .parent } else {} end')"
cat "$dir/$configFile" | jq --raw-output "$imageOldConfig + del(.history, .rootfs)" > "$dir/$imageId/json"
local manifestJsonEntry="$(
jq --raw-output --compact-output -n \
--arg configFile "$configFile" \
--arg repoTags "${image#library\/}:$tag" \
--argjson layers "$(IFS=$'\n'; jq --arg a "${layerFiles[*]}" -n '$a | split("\n")')" \
'{
Config: $configFile,
RepoTags: [ $repoTags ],
Layers: $layers
}'
)"
(1) I have verified that using bash, the sequence:
addJson='{ id: "ee6b1042efee4fb07d2fe1a5079ce498567e6f5ac849413f0e623d4582da5bc9",
parent: "80a2fb00dfe137a28c24fbc39fde656650cd68028d612e6f33912902d887b108" }'
jq "$addJson + ." configFile > layerId.json
succeeds, where configFile has the contents shown in the updated question.
(2) Similarly, I have verified that the following also succeeds:
imageOldConfig="$(jq --raw-output --compact-output '{ id: .id } + if .parent then { parent: .parent } else {} end' layerId.json)"
jq --raw-output "$imageOldConfig + del(.history, .rootfs)" <<-'EOJSON'
<JSON as in the question>
EOJSON
where <JSON as in the question> stands for the JSON shown in the question.
(3) In general, it is not a good idea to pass shell $-variables into jq programs by shell string interpolation.
For example, rather than writing:
jq --raw-output "$imageOldConfig + del(.history, .rootfs)"
it would be much better to write something like:
jq --raw-output --argjson imageOldConfig "$imageOldConfig" '
$imageOldConfig + del(.history, .rootfs)'

How to edit Linux containers configs and observe it hashes in Docker Windows?

I installed Docker on Windows. It's switched to Switched to Linux containers.
When I type in my console: docker inspect e3a934c54979 I see an information:
[
{
...
"Image": "sha256:2359fa12fdedef2af79d9b836a26175808d4b1433b5e7022d2d73c72b2a43b60",
"ResolvConfPath": "/var/lib/docker/containers/e3a934c549799d9ec45d65ad6aa73bba8fad924215087a9c9c60535ef2a5c2e8/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/e3a934c549799d9ec45d65ad6aa73bba8fad924215087a9c9c60535ef2a5c2e8/hostname",
"HostsPath": "/var/lib/docker/containers/e3a934c549799d9ec45d65ad6aa73bba8fad924215087a9c9c60535ef2a5c2e8/hosts",
"LogPath": "/var/lib/docker/containers/e3a934c549799d9ec45d65ad6aa73bba8fad924215087a9c9c60535ef2a5c2e8/e3a934c549799d9ec45d65ad6aa73bba8fad924215087a9c9c60535ef2a5c2e8-json.log",
"Name": "/festive_edison",
...
"HostConfig": {
"Binds": null,
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "default",
"PortBindings": {
"80/tcp": [
{
"HostIp": "",
"HostPort": "80"
}
]
},
...
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/10f5348d5bfa76612ab30d1a253f17a6989fcd3f7ce23642b313c49f99a95f44-init/diff:/var/lib/docker/overlay2/028eac1b0f37fd3be798d222f7d1da48a40f0ef9c4470709e63c4c8f322a477f/diff:/var/lib/docker/overlay2/d15e7ce0f29f82d6d3b9537980b766c32e7f6ffc81374cdb26fede3872afed1e/diff:/var/lib/docker/overlay2/efab543606225e581832ef6e2b732a78c82b2f6d9fe662babe09b188f600dd72/diff:/var/lib/docker/overlay2/263366359e8a86cc6c009f70fa00a158dbcbcfd2a4e31d9538c559dd82e29b10/diff:/var/lib/docker/overlay2/32ea6c48b53f4846284e1baac83dffcfb039a53a8d2f33ac2728691160f5d100/diff:/var/lib/docker/overlay2/685745d44609453debf484b2ccf63035532b334e75b9f18a00c5e1253e18841a/diff:/var/lib/docker/overlay2/e30c0a304544255bc9eba90dfb720c332e168b4972df926a79ef27df707889fd/diff:/var/lib/docker/overlay2/a5743532bc060895f0a495249182787322400a1a33fd187b3210895e1ca83129/diff",
"MergedDir": "/var/lib/docker/overlay2/10f5348d5bfa76612ab30d1a253f17a6989fcd3f7ce23642b313c49f99a95f44/merged",
"UpperDir": "/var/lib/docker/overlay2/10f5348d5bfa76612ab30d1a253f17a6989fcd3f7ce23642b313c49f99a95f44/diff",
"WorkDir": "/var/lib/docker/overlay2/10f5348d5bfa76612ab30d1a253f17a6989fcd3f7ce23642b313c49f99a95f44/work"
},
"Name": "overlay2"
},
...
}
]
But Windows doesn't have those directories. It only has "MobyLinuxVM.vhdx" which, I think, contains this stuff.
My question is how to edit "config.json" and "hostconfig.json" in this case? How do I view a GUID-json.log? How do I view container's hashes (/var/lib/docker/aufs/diff)?
Information from https://blog.jongallant.com/2017/11/ssh-into-docker-vm-windows/
In a Windows command prompt enter:
docker run --privileged -it -v
/var/run/docker.sock:/var/run/docker.sock
jongallant/ubuntu-docker-client
docker run --net=host --ipc=host --uts=host --pid=host -it
--security-opt=seccomp=unconfined --privileged --rm -v /:/host alpine /bin/sh
chroot /host
From here you'll have access to the /var/lib/Docker/containers/ directories for the hostconfig.json and other files.

Docker disconnect all containers from docker network

I have docker network "my_network". I want to remove this docker network with docker network rm my_network. Before it I should disconnect all my containers from this network. I can use docker network inspect and get output like
[
{
"Name": "my_network",
"Id": "aaaaaa",
"Scope": "some_value",
"Driver": "another_value",
"EnableIPv6": bool_value,
"IPAM": {
"Driver": "default",
"Options": {},
"Config": [
{
"Subnet": "10.0.0.0/1"
}
]
},
"Internal": false,
"Containers": {
"bbb": {
"Name": "my_container_1",
"EndpointID": "ENDPOITID1",
"MacAddress": "MacAddress1",
"IPv4Address": "0.0.0.0/1",
"IPv6Address": ""
},
"ccc": {
"Name": "my_container_2",
"EndpointID": "ENDPOINTID2",
"MacAddress": "MacAddress2",
"IPv4Address": "0.0.0.0/2",
"IPv6Address": ""
}
},
"Options": {},
"Labels": {}
}
]
It is okay to manual disconnect if I have only several containers but if I have 50 containers I have problem.
How can I disconnect all containers from this network with single or several command?
docker network inspect has a format option.
That means you can list all Container names with:
docker network inspect -f '{{range .Containers}}{{.Name}}{{end}}' network_name
That should then be easy, by script, to read each name and call docker network disconnect.
wwerner proposes below in the comments the following command:
for i in ` docker network inspect -f '{{range .Containers}}{{.Name}} {{end}}' network_name`; do docker network disconnect -f network_name $i; done;
In multiple line for readability:
for i in ` docker network inspect -f '{{range .Containers}}{{.Name}} {{end}}' network_name`;\
do \
docker network disconnect -f network_name $i; \
done;
Adding:
Note that there is a space in the format as opposed to the answer to split the names by a space.

What's the container of a base docker image?

I am learning to use docker. I know that each docker image is build on a base image which doesn't have a parent.
Then from a base image, we can customize some stuff via containers (maybe a very short life container) to commit a new image.
So I understand that the process is like this: base image -> container -> new image 1 -> container -> new image 2
However, I inspect the json data of the base image. I still can see that it has a container's information:
[{
"Architecture": "amd64",
"Author": "",
"Comment": "",
"Config": {
"AttachStderr": false,
"AttachStdin": false,
"AttachStdout": false,
"Cmd": null,
"CpuShares": 0,
"Cpuset": "",
"Domainname": "",
"Entrypoint": null,
"Env": null,
"ExposedPorts": null,
"Hostname": "3f37dbc61890",
"Image": "",
"Labels": null,
"MacAddress": "",
"Memory": 0,
"MemorySwap": 0,
"NetworkDisabled": false,
"OnBuild": null,
"OpenStdin": false,
"PortSpecs": null,
"StdinOnce": false,
"Tty": false,
"User": "",
"Volumes": null,
"WorkingDir": ""
},
"Container": "3f37dbc61890b0bb37cc8479db94602bcc2d6e177d76c0f3d7d53346c0dc580c",
"ContainerConfig": {
"AttachStderr": false,
"AttachStdin": false,
"AttachStdout": false,
"Cmd": [
"/bin/sh",
"-c",
"#(nop) ADD file:777fad733fc954c0c161670c48c10ea1787a6e5d544daa20e55d593279df3fa3 in /"
],
"CpuShares": 0,
"Cpuset": "",
"Domainname": "",
"Entrypoint": null,
"Env": null,
"ExposedPorts": null,
"Hostname": "3f37dbc61890",
"Image": "",
"Labels": null,
"MacAddress": "",
"Memory": 0,
"MemorySwap": 0,
"NetworkDisabled": false,
"OnBuild": null,
"OpenStdin": false,
"PortSpecs": null,
"StdinOnce": false,
"Tty": false,
"User": "",
"Volumes": null,
"WorkingDir": ""
},
"Created": "2015-04-21T22:18:45.67739694Z",
"DockerVersion": "1.6.0",
"Id": "706766fe101906a1a6628173c2677173a5f8c6c469075083f3cf3a8f5e5eb367",
"Os": "linux",
"Parent": "",
"Size": 188104128,
"VirtualSize": 188104128
}]
3f37dbc61890b0bb37cc8479db94602bcc2d6e177d76c0f3d7d53346c0dc580c is
the container id of the base image.
what is a container of a base image? I feel like it becomes the hostname.
What is the hostname actually?
I'm not sure of your terminology. If a base image is by definition an image that has no parent, then the image in your example is not a base image.
But an image may have no parent. scratch, for example, has no parent:
$ docker inspect -f '{{.Container}}' ubuntu
a4c15f8c80978475a53f96721f935de5823bc8c29aff14eb00a15f9b9d96cddd
$ docker inspect -f '{{.Container}}' scratch
$
You can also create an image that has no parent using import:
$ echo hello world > foo && tar -cf- foo | docker import -
3e8fc0cb69fae0bd3f9711031df6d3b7bf6a7e8c9745657d9261e7b803718c67
$ docker inspect -f '{{.Container}}' 3e8fc0c
$
Unlike scratch, this image may have files in it. In fact, you can flatten a complex image using this technique.
$ docker create ubuntu # create container from image
6b90bf145c193ef8e4ecb789372d2fd619769a20d96c8f3f586dcfbc501b0611
$ docker export 6b90bf1 > ubuntu.tar # export container fs to tarball
$ docker import - flat_ubuntu < ubuntu.tar
4ef4ffb9514212acf6a19b2eeda8855b8c0445924311043ba5cba6574d40d772
$ docker inspect -f '{{.Container}}' 4ef4ffb
$
It's important to note that while this new image has the exact same files as the original image, it doesn't have the other docker features like environment, volume, entrypoint, etc.
I would not necessarily call this a "base" image. I would call it a "flat" image. I would say a base image is the image you indicate in a Dockerfile FROM directive. In my terminology a base image need not be flat.

Resources