I'm using Windows 10 with native docker installation.
I'm looking for the location where docker save the containers logs.
In Linux, the Docker containers log files are in this location:
/var/lib/docker/containers/container-id/container-id-json.log
But where can I find it in windows 10 ?
For Windows 10 + WSL 2 (Ubuntu 20.04), Docker version 20.10.2, build 2291f61
Lets DOCKER_ARTIFACTS == \\wsl$\docker-desktop-data\version-pack-data\community\docker
Container logs can be found in the following location
DOCKER_ARTIFACTS\containers\[Your_container_ID]\[Your_container_ID]-json.log
Here is an example :
Check first if those logs are in (as suggested here):
C:\ProgramData\docker\containers\[container_ID]\[container_ID]-json.log
The Docker C:\ProgramData\docker is the Root Dir reported by docker info.
Regarding Docker Linux through Hyper-v, check if "How to Delete Docker Container Log Files (Windows or Linux) " can help (from Jon Gallant):
Run docker inspect to find your Docker log file location
Find the “Docker Root Dir” Value, mine is /var/lib/docker
Your docker log file path should be /var/lib/docker, but if it isn’t, then change it in the command below.
find /var/lib/docker/containers/ -type f -name "*.log"
The command you see in this image is based on "How to SSH into the Docker VM (MobyLinuxVM) on Windows"
We aren’t technically going to SSH into the VM, we’ll create a container that has full root access and then access the file system from there.
Get container with access to Docker Daemon
Run container with full root access
Switch to host file system
Open a Command prompt and execute the following:
docker run --privileged -it -v /var/run/docker.sock:/var/run/docker.sock jongallant/ubuntu-docker-client
docker run --net=host --ipc=host --uts=host --pid=host -it --security-opt=seccomp=unconfined --privileged --rm -v /:/host alpine /bin/sh
chroot /host
Execute the find command there, and you should find the logs.
For Windows 10 + Docker Desktop version 3.6.0, the virtual path for logs and data (artifacts) is \\wsl$\docker-desktop-data\version-pack-data\community\docker (you can copy/paste it in Explorer navigation bar).
The logs are at \\wsl$\docker-desktop-data\version-pack-data\community\docker\containers\[containerID]\[containerID]-json.logs
and the data is under \\wsl$\docker-desktop-data\version-pack-data\community\docker\volumes\[volumeID]\_data
How to get containerID:
You can find container ID (truncated) by running docker ps in a command prompt. You can also find it by clicking the CLI button on DockerDesktop next to the container name, the id will be in the title of the cmd window that will pop up.
Once you have the id, you can navigate to containers\[containerID] under the artifacts directory (\\wsl$\docker-... above). The log file will have the .log extension and its name will have the containerID in it. Keep in mind that it will be an enriched json format though, so not easily readable.
How to get volumeID:
To find a container's data (for example kafka broker topics), you need to find the id of the volumes where the data is stored. For that you will need to click on the container in DockerDesktop, then click INSPECT (top right). You can then scroll down to find Mounts configuration entries. Each mount will have a volumeID (different from containerID), and that volumeID will be part of a path similar to this one /var/lib/docker/volumes/71f7a5992c58fdcf229c3848acb014712f34fab380bc7c712cf5a0a632fe9110/_data. volumeID here being 71f7a5992c58fdcf229c3848acb014712f34fab380bc7c712cf5a0a632fe9110.
You can then take volumeID and navigate to volumes\[volumeID] under the artifacts directory (\\wsl$\docker-... above) where the data will be located.
For me, using Docker Desktop for Windows on version 4.9.1 (81317), Windows 10 21H2, WSL 2 mode, the containers' folders were at
\\wsl$\docker-desktop-data\data\docker\containers
Slightly different from the others.
For Windows Users who wants to delete all Docker log files on WSL 2.
The path to the docker container is correct thx to #craftsmannadeem
\\wsl$\docker-desktop-data\version-pack-data\community\docker\containers
Here a command to execute on Windows to delete all log files:
del /s \\wsl$\docker-desktop-data\version-pack-data\community\docker\containers\*-json.log
Bye bye Docker logs:
File was deleted - \\wsl$\docker-desktop-data\version-pack-data\community\docker\containers\2012efd0ccfb8aed6291dd9a3b7b5aef507b6af4fce5b85e9306f45980db9531\2012efd0ccfb8aed6291dd9a3b7b5aef507b6af4fce5b85e9306f45980db9531-json.log
File was deleted - \\wsl$\docker-desktop-data\version-pack-data\community\docker\containers\9e627f1fe8f3c3ab85c64f85f93942d1f077e9a6e2896b51df782b0c0c3777d1\9e627f1fe8f3c3ab85c64f85f93942d1f077e9a6e2896b51df782b0c0c3777d1-json.log
File was deleted - \\wsl$\docker-desktop-data\version-pack-data\community\docker\containers\6ea8f3cb354c199bc719701f8f1e75c333f81cd2f03dca0c7a626cbcbf9ed5a0\6ea8f3cb354c199bc719701f8f1e75c333f81cd2f03dca0c7a626cbcbf9ed5a0-json.log
...
For window system logs are located at C:\ProgramData\Docker\containers or %APPDATA%\Docker
For Linux system logs are located at /var/lib/docker/containers
I couldn't find where the logs were stored locally. (Good chance they aren't plain text any more. However, if you just need the output of the logs, you can run a command like this:
docker logs --details [container-name] > container-name.log
This will grab the logs for the container and write them to a log file in the current directory.
Note: The --details adds additional info to the logs like environment variables and the like, but is not required for the command to work.
If you use docker-compose with windows+wsl, In my case the log monitoring agent (also running as a container in docker) was not able to find the log files eventhough the path for logs was mounted as volume.
volumes:
- /var/run/docker.sock:/var/run/docker.sock
-/var/lib/docker:/var/lib/docker
The log monitoring agent could not find logs in
/var/lib/docker/containers/**/*.log
The problem in my case was, I was running docker-compose up command for the log monitoring agent from within wsl shell. When I ran it from windows powershell or cmd, the agent was able to find the logs in mounted path.
I have a similar issue to the following link, but in powershell as I am running a clickhouse docker container in windows 10.
Data directory permissions on host for Clickhouse installation via docker
My setup is run as such:
docker run -d -p 8124:8124 --name my_database --ulimit nofile=262144:262144 --volume=E:/:/var/lib/clickhouse yandex/clickhouse-server
E drive is one of the drives on my windows computer.
I cannot seem to access /var/lib/clickhouse/data when running a mergetree table creation. It seems that clickhouse client is not being given adequate permissions to reach this file system. The error looks as such:
Access to file denied: /var/lib/clickhouse/data/default/test_mergetree/tmp_insert_20150731_20150731_8_8_0
Since I am in powershell I am unsure how I might approach solving this. I am attempting to access the file system to give powershell permissions:
Something like this
ICACLS "var/lib/clickhouse/data" /setowner "administrator"
But then since clickhouse is dockerized it seems I cannot find the path:
The system cannot find the path specified.
Would I have to run docker compose? Or am I approaching this the wrong way?
ATTEMPT 1
I've tried running the following:
docker run --rm -i --entrypoint /bin/sh yandex/clickhouse-server -c id clickhouse
#got back:
uid=0(root) gid=0(root) groups=0(root)
#went into the system and ran
docker exec -it container-id bash
chown -R 0:0 /var/lib/clickhouse
#got back
chown: cannot read directory '/var/lib/clickhouse/System Volume Information': Permission denied
You should run docker and clickhouse in Linux instead of Windows.
Turns out this is an issue which has not been repaired in windows docker desktop:
https://github.com/docker/for-win/issues/39
Volume mounts are necesarry. But I got around it by changing the disk image to the target host drive. Under settings -> advanced -> change the virtual hard disk image to the drive you want and you can write within that drive. Note you still won't have access to raw data.
Still an issue but the practical solution is to move to Docker Volumes.
With bind mount there is a problem with wsl2
What is the equivalent to "Connect To Server" on the mac for the command line?
I would like to automate the process rather than summon the dialog every time I need to connect to a server.
There are multiple ways to connect to remote server!
I am assuming you have server running on windows OS. Mac OS X includes the SMB protocol for networking with Windows computers; some Linux and Unix systems can also share files with this protocol as well. You can easily mount a shared SMB volume using the mount command. Steps to be followed as below
1) mkdir /temp
2) chmod 777
3) mount -t smbfs //username#ip/nameOfSharefolder /temp
After this you can browse to/temp directory and browse
You can also use sshssh or ftp command to access the remote server but you need to run the ftp server in case of ftp command or remote access must be enable in case of ssh
I am using docker on windows. With the use of kitematic, I have created an ubuntu container. This ubuntu image has postgresql installed on it.
Wondering if there is any possibility to access the postgres configuration files available in the container from the host (windows machine)?
Where exactly does the container store its file system on the host machine?
I hope it would be part of image file with format VMDK.
Please correct me if I'm wrong.
Wondering if there is any possibility to access the postgres configuration files available in the container from the host (windows machine)
That is not how Docker would allow you to modify a file in a container.
For that, you should mount a host (Windows) folder when starting (docker run -v) your container.
See "Mount a host directory as a data volume"
docker run -d -P --name web -v /c/Users/<myACcount>/src/webapp:/opt/webapp training/webapp python app.py
Issue 247 mentions ~/Library/Application Support/Kitematic for App data, and ~/Kitematic "for easy access to volume data".
I'm working on wrapping some scientific software by docker image using boot2docker on Mac OS X. And that software (https://github.com/voutcn/megahit.git) is using named pipes (in python code, but it's not important) to wire different parts (written in C) to each other. I mount temporary folder from host Mac OS X machine to provide scratch area in docker container (because temporary output of software could be huge) with something like this:
docker run -v /external/folder:/tmp/scratch <image> <args>
It gives me this mount line inside container:
none on /tmp/scratch type vboxsf (rw,nodev,relatime)
And inside this mounted folder named pipe creation fails when it runs inside container. It's not even related to python, C or any particular language. I double checked with linux command mkfifo pipe1 in this folder with an error:
mkfifo: cannot create fifo 'pipe1': Operation not permitted
It works well for any internal not mounted folder inside container though. Why does it happen and how could it be fixed?
PS: Here is what I do to easily reproduce the problem.
1) Mac OS X with boot2docker
2) Dockerfile is:
FROM ubuntu:14.04
#WORKDIR /tmp <- this one would work
WORKDIR /tmp/scratch
ENTRYPOINT [ "mkfifo" ]
CMD [ "pipe1" ]
3) Image building:
docker build --rm -t mine/namedpipes:latest .
4) Running (being in external host folder to be mounted):
docker run -v $(pwd):/tmp/scratch mine/namedpipes:latest
Upgrade to a recent version of Docker for Mac, and your problem will likely be solved: https://docs.docker.com/docker-for-mac/release-notes/#beta-2-release-2016-03-08-1102-beta2
The issue is that FIFOs are actually kernel objects you access using the filesystem, and so you would need extra work to support cross-kernel FIFOs (or unix domain sockets) - a fifo is either valid inside the Linux guest running the docker daemon or in the OS X host, not in both, and it makes sense that you can't create an OS X fifo from inside the linux box. It would be sort of like trying to create a fifo on a network drive, it doesn't make sense as a local IPC mechanism.
Current support for special files is detailed in https://docs.docker.com/docker-for-mac/osxfs/#file-types
The issue for cross-hypervisor support is located at https://github.com/docker/for-mac/issues/483