I want to use Hadoop from CDH docker image. CDH image is already installed on my machine and I can run it.
docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
07a55a9d4cb9 4239cd2958c6 "/usr/bin/docker-quickstart" 18 minutes ago Up 18 minutes 0.0.0.0:32774->7180/tcp, 0.0.0.0:32773->8888/tcp container
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' container
172.17.0.2
Local, I am writing an ansible playbook and I need to set Hadoop conf dir in CDH which is: "/etc/hadoop/conf".
How can I set the running docker image in my ansible playbook?
I tried:
- name: run cloudera
docker_container:
name: "container"
image: quickstart/cloudera
command: /usr/bin/docker-quickstart"
state: started
ports:
- 8888:8888
- 7180:7180
But this command runs another docker image and I would like to connect to the running one.
inventory.ini
container ansible_connection=docker
Note: I suggest for the future that you rename your container to something more distinct than container....
example playbook.yml
---
- hosts: container
tasks:
- name: I am a dummy task, write your own
file:
path: /tmp/helloContainer
state: file
Running the playbook
ansible-playbook -i inventory.ini playbook.yml
Related
is there a way to start a podman container with ansible periodically?
I'd like to mix the ansible's cron, and podman modules, like:
- name: Run the DB Migrations docker image
ansible.builtin.cron:
hours: '10'
job:
containers.podman.podman_container:
name: my-podman-container
image: registry.gitlab.com/vilmosnagy/my-container
recreate: true
Yeah, I could write the podman run --rm --name my-podman-container registry.gitlab.com/vilmosnagy/my-container command in the cron module I'm looking for a better way to do this.
Thanks,
I am on my MacBook terminal. I try to have a jenkins container up and running on my local machine.
I firstly created a docker-compose.yml :
version: '3'
services:
jenkins:
container_name: jenkins
image: jenkins/jenkins
ports:
- "8080:8080"
volumes:
- $PWD/jenkins_home:/var/jenkins_home
networks:
- net
networks:
net:
As you can see in the volumes section, I have defined the jenkins_home folder under my current directory as the volume for jenkins data.
Then under my current directory of my machine, I created a folder named jenkins_home. Here is my current directory:
-rw-r--r-- 1 john 1349604816 220 Sep 4 00:08 docker-compose.yml
drwxr-xr-x 2 john 1349604816 64 Sep 4 00:06 jenkins_home
As you can see, I need to change the ownership of jenkins_home folder in order to have jenkins container be able to write data in it (because the uid is not 1000). So, I executed command:
sudo chown 1000:1000 jenkins_home/
Then, my current directory looks like this:
-rw-r--r-- 1 john 1349604816 220 Sep 4 00:08 docker-compose.yml
drwxr-xr-x 2 1000 1000 64 Sep 4 00:06 jenkins_home
After that I run my container by command: docker-compose up. But I ended up with error:
Starting jenkins ... done
Attaching to jenkins
jenkins | touch: cannot touch '/var/jenkins_home/copy_reference_file.log': Permission denied
jenkins | Can not write to /var/jenkins_home/copy_reference_file.log. Wrong volume permissions?
jenkins exited with code 1
Why I still get the permission error after I changed the ownership of the jenkins_home folder under my current directory on my machine?
P.S. I understand there could be other way to purely have a jenkins container running but still I would like to understand what is wrong with my approach and hopefully could also get it work.
Jenkins needs to create or to use existing jenkins_home directory,
When Docker sees that jenkins_home volume in your machine doesn't exists then it will create it with your osx UID & GID.
If you create the jenkins_home folder you must stay with your current directory permissions and not changed them,
Docker running UID isn't the same as your machine, they may have different UID and GID.
Linux namespaces provide isolation for running processes, limiting
their access to system resources without the running process being
aware of the limitations. For more information on Linux namespaces,
see Linux namespaces.
The best way to prevent privilege-escalation attacks from within a
container is to configure your container’s applications to run as
unprivileged users. For containers whose processes must run as the
root user within the container, you can re-map this user to a
less-privileged user on the Docker host. The mapped user is assigned a
range of UIDs which function within the namespace as normal UIDs from
0 to 65536, but have no privileges on the host machine itself.
There a wonderful video explaining how docker works with namespaces
Does the actual jenkins user/group exist on the Mac?
This is what I do on my linux servers where:
ARG user=jenkins
ARG group=jenkins
ARG uid=1000
ARG gid=1000
On my alpine server:
addgroup -g ${gid} ${group}
adduser -u ${uid} -G ${group} -s /bin/bash -D ${user}
to become
addgroup -g 1000 jenkins
adduser -u 1000 -G jenkins -s /bin/bash -D jenkins
On my centos8 server
groupadd -g ${gid} ${group}
useradd -u ${uid} -g ${group} -s /bin/bash -d ${user}
to become
groupadd -g 1000 jenkins
useradd -u 1000 -g jenkins -s /bin/bash -d jenkins
then:
sudo chown jenkins:jenkins jenkins_home/
I do not use Mac, but I presume it is similar
UPDATE
Based on all the above, try the following:
docker-compose.yml
version: '3'
services:
jenkins:
container_name: jenkins
image: jenkins/jenkins
ports:
- 8080:8080
- 50000:50000
volumes:
- $PWD/jenkins_home:/var/jenkins_home
- /var/run/docker.sock:/var/run/docker.sock
networks:
- net
networks:
net:
I have added the following:
port 50000 (only if you want to attach build slave servers, opposed to just running builds on the master)
volume /var/run/docker.sock (to be able to use the docker daemon with Jenkins, you need to mount the volume)
!!DO THE FOLLOWING!! Delete the original jenkins_home directory that you created before. Now run 'docker-compose up', since the host volume directory does not exist, docker will now create the required directory on the host which is based on the configuration in the docker-compose.yml (in this case '$PWD/jenkins_home'), thus it will now have the correct ownership and permissions for the jenkins container to use it.
If that doesn't work, make the jenkins container run in privileged mode, see below:
version: '3'
services:
jenkins:
container_name: jenkins
image: jenkins/jenkins
privileged: true
user: root
ports:
- 8080:8080
- 50000:50000
volumes:
- $PWD/jenkins_home:/var/jenkins_home
- /var/run/docker.sock:/var/run/docker.sock
networks:
- net
networks:
net:
I want to replace this command with an ansible playbook
'docker build -q --network host -t "ubuntu" . '
I have been going through docker_image module of ansible but couldn't figure it out. Any idea on how to proceed further?
Thanks in advance.
The nearest you can have it is:
---
- name: build the image
docker_image:
name: docker
tag: ubuntu
path: "/yourpath"
state: present
For the --network host, there is a request open in Github to have it.
I have 2 containers mgmt-app who is a Meteor container and mgmt-mongo who is the MongoDB.
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7b65be4ac454 gitlab-lab:5005/dfc/mongo:latest "/entrypoint.sh mongo" About an hour ago Up About an hour 27017/tcp mgmt-mongo
dff0b3c69c5f gitlab-lab:5005/dfc/mgmt-docker-gui:lab "/bin/sh -c 'sh $METE" About an hour ago Up 42 minutes 0.0.0.0:80->80/tcp mgmt-app
From my Docker host I want to run docker run gitlab-lab:5005/dfc/mgmt-docker-gui:lab ls -al
but I have this error:
=> Starting meteor app on port:80
/app/programs/server/node_modules/fibers/future.js:280
throw(ex);
^
Error: MONGO_URL must be set in environment
So I tried:
docker run -e "MONGO_URL=mongodb://mgmt-mongo:27017/meteor" gitlab-lab:5005/dfc/mgmt-docker-gui:lab ls -al
and then the error was:
/app/programs/server/node_modules/fibers/future.js:313
throw(ex);
^
MongoError: failed to connect to server [mgmt-mongo:27017] on first connect
I really don't understand because when I do a docker-compose up -d with this file:
mgmt-app:
image: gitlab-lab:5005/dfc/mgmt-docker-gui:latest
container_name: mgmt-app
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- $HOME/.docker:/root/.docker
- /home/dockeradm/compose/area:/home/dockeradm/compose/area
environment:
- ROOT_URL=http://localhost:80
- MONGO_URL=mongodb://mgmt-mongo:27017/meteor
ports:
- 80:80
restart: always
mgmt-mongo:
image: gitlab-lab:5005/dfc/mongo:latest
container_name: mgmt-mongo
volumes:
- mgmt_mongo_data_config:/data/configdb
- mgmt_mongo_data_db:/data/db
restart: always
everything go well.
So my request is, how should I do my docker run to execute my command ? (the command is not a simple ls -al but it's ok for the demo)
When you run the containers separately with docker run, they are not linked on the same docker network so the mongo container is not accessible from the app container. To remedy this, you should use either:
--link to mark the app container as linked to the mongo container. This works, but is deprecated.
a defined docker network for both containers to be linked by; this is more complex, but is the recommended architecture
By contrast, docker-compose automatically adds both containers to the same docker network, so they are immediately connectable without any extra configuration required:
By default Compose sets up a single network for your app. Each container for a service joins the default network and is both reachable by other containers on that network, and discoverable by them at a hostname identical to the container name.
With docker compose i launch a jenkins container and i want to have the possibility to execute docker command with(docker installed on the server).
But when i tried to make a simple test run hello-world image i have the following error :
docker: Cannot connect to the Docker daemon. Is the docker daemon running on this host?.
I set the user on the docker group, what's wrong with my docker compose file ?
in other post i see if i add this line :
/var/run/docker.sock:/var/run/docker.sock
my container with jenkins can communicate with docker
my docker compose file
jenkins:
image: jenkins:2.32.3
ports:
- 8088:8080
- 50000:50000
volumes:
- /home/my-user-name/docker-jenkins/jenkins_home:/var/jenkins_home
- /var/run/docker.sock:/var/run/docker.sock
- /usr/bin/docker:/usr/bin/docker
- /tmp:/tmp
To access the docker.sock file, you must run with a user that has filesystem access to read and write to this socket. By default that's with the root user and/or the docker group on the host system.
When you mount this file into the container, that mount keeps the same uid/gid permissions on the file, but those id's may map to different users inside your container. Therefore, you should create a group inside the container as part of your Dockerfile that maps to the same gid that exists on the host, and assign your jenkins user to this group, so that it has access to the docker.sock. Here's an example from a Dockerfile where I do this:
...
ARG DOCKER_GID=993
RUN groupadd -g ${DOCKER_GID} docker \
&& useradd -m -d /home/jenkins -s /bin/sh jenkins \
&& usermod -aG docker jenkins
...
In the above example, 993 is the docker gid on my host.