Is there any way to share a .bash_history volume with a docker container so that everytime I go into a shell I have my bash history available for scrolling through?
Would be awesome to be able to do the same thing with IPython too.
It is the example from the documentation about volume: Mount a host file as a data volume:
docker run --rm -it -v ~/.bash_history:/root/.bash_history ubuntu /bin/bash
This will drop you into a bash shell in a new container, you will have your bash history from the host and when you exit the container, the host will have the history of the commands typed while in the container.
In your docker-compose.override.yml:
version: '2'
services:
whatever:
…
volumes:
- …
- ~/.bash_history:/root/.bash_history
To keep IPython history, you can set the IPYTHONDIR environment variable to somewhere within your mapped volume.
The docker-compose.override.yml would look like this:
version: '2'
services:
some-service:
environment:
- IPYTHONDIR=/app/.ipython
volumes:
- .:/app
My solution is useful when:
you don't want to share your local .bash_history with .bash_history in your container
you use other shell (like fish shell) but you want to save .bash_history between your builds
you don't want to commit .bash_history to git repo but you want to create it automatically inside same directory when a container starts
I assume file structure to be:
docker-compose.yml
docker/
\--> bash/
\--> .bashrc
\--> .bash_history
docker-compose.yml
web-service:
build: .
volumes:
- ./docker/bash/.bashrc:/home/YOUR_USER_NAME/.bashrc
- ./docker/bash:/home/YOUR_USER_NAME/bash
./docker/bash/.bashrc - it will automatically create .bash_history:
export HISTFILE=~/bash/.bash_history
touch $HISTFILE
Optionally, you can add to .gitignore:
docker/bash/.bash_history
You can also achieve this with a named volume and tell bash where he can find the bash history file by defining the HISTFILE environment variable. I explained a bit more here:
https://antistatique.net/en/we/blog/2019/11/12/tips-docker-keep-your-bash-history
For bash
volumes:
- ./.data/shell_history/php_bash_history.txt:/home/www-data/.bash_history #bash
For sh
volumes:
- ./.data/shell_history/nginx_bash_history.txt:/root/.ash_history #sh
Related
I am trying to run a local script with docker bash in windows PowerShell but not working.
My script part is another program, but the finally goal is to process a media file and zip it with the shell script.
The cmd: docker exec -it containername /bin/bash < myscript.sh -f fileone.mp4 -h output
I have an error in ps:
The '<' operator is reserved for future use.
The parameters (and also the files) are changing, if rerun the shell script, and after the script, processing is done it will create a zip file (what I need) with the output name, but random strings will be placed to the zipped filename too.
Anyone tried to use docker in that way in windows?
I figure out a solution for my own question. I just leave it here, if someone needs it.
The docker-compose file:
version: '3.8'
services:
somename:
build:
context: .
dockerfile: Dockerfile
container_name: 'name_if_you_need'
The dockerfile:
FROM debian:latest
# Install and/or config anything what you need
ADD . /newfolder
WORKDIR /newfolder
ENTRYPOINT [ "/newfolder/myscript.sh" ]
To call (with arguments and/or flags if your script need it): docker run --rm -v ${PWD}:/newfolder image_name -flag1 sample.mp4 -flag2 sample (no tty error, not need winpty)
Please note, if your script working with file or files, and you pass it via arguments like me, you need to copy them in your current folder before docker run
With this solution, if your script generates a file or files when/after executing, you will see them automatically in your current folder.
I'm using the bash shell provided by Git for Windows for Docker toolbox for Windows. I want to export a string representing a unix path to a environment variable to then use in a docker container. Something like:
export MY_VAR=/my/path; docker-compose up
The problem is that in my container the variable will be something like:
echo $MY_VAR # prints c:/Program Files/Git/my/path
So it seems the shell (my guess) recognizes the string as a path and converts it to windows format. Is there a way to stop this?
I've attempted to use MSYS_NO_PATHCONV=1:
MSYS_NO_PATHCONV=1; export LOG_PATH=/my/path; docker-compose up
But it did not have any effect.
I don't think it's an issue with my docker-compose and dockerfile but I'll attach them if someone is interested.
My Dockerfile:
FROM node:8-slim
RUN mkdir /test \
&& chown node:node /test
USER node
ENTRYPOINT [ "/bin/bash" ]
My docker-compose.yml:
version: '2'
services:
test:
build:
context: .
image: test
environment:
- MY_VAR
volumes:
- ${MY_VAR}:/test
command: -c 'sleep 100000'
The Final goal here is to make a directory on the host machine accessible from the docker container (for logs and such). The directory should be set by an environment variable. Setting the directory in the docker-compose.yml does work, just not for my use case.
If you want your command docker-compose up to be run with MSYS_NO_PATHCONV=1; you have two options:
export LOG_PATH=/c/Windows; export MSYS_NO_PATHCONV=1; docker-compose up This will affect your bash session as the variable is exported
export LOG_PATH=/c/Windows; MSYS_NO_PATHCONV=1 docker-compose up; (note I removed one semi-colon intentionally) This will set MSYS_NO_PATHCONV only in the context of the command to run
Test it with:
$ export LOG_PATH=/c/Windows ; cmd "/c echo %LOG_PATH%";
C:/Windows --> Fails
$ export LOG_PATH=/c/Windows ; MSYS_NO_PATHCONV=1 cmd "/c echo %LOG_PATH%"
/c/Windows --> Success
$ export LOG_PATH=/c/Windows ; export MSYS_NO_PATHCONV=1; cmd "/c echo %LOG_PATH%";
/c/Windows --> Success but MSYS_NO_PATHCONV is now "permanently" set
Seems a workaround is to remove the first / from the string and add it in the docker-compose.yml instead.
new docker-compose.yml:
version: '2'
services:
test:
build:
context: .
image: test
environment:
- MY_VAR
volumes:
- /${MY_VAR}:/test # added '/' to the beginning of the line
command: -c 'sleep 100000'
and then starting the container with:
export MY_VAR=my/path; docker-compose up # removed the '/' from the beginning of the path.
This does seem more like a "lucky" workaround than a perfect solution as when I'll build this on other systems I'll have to remind myself to remove the /. Doable but a bit annoying. Maybe someone has a better idea.
Here's my docker-compose:
version: '2'
services:
couchpotato:
build:
context: ./couchpotato
dockerfile: Dockerfile
ports:
- 5050:5050
volumes:
- "${PWD}/couchpotato/data:/home/CouchPotato/data/"
- "${PWD}/couchpotato/config:/home/CouchPotato/config/"
When I run it inside the shell, in the directory of the docker-compose.yml, I get:
WARNING: The PWD variable is not set. Defaulting to a blank string.
and the compose starts with PWD being empty.
I don't see any error in the file, as seen here: https://docs.docker.com/compose/environment-variables/
You don't need ${PWD} for this, you can just make the path relative and compose will expand it (one major difference between compose paths and those processed by docker run).
version: '2'
services:
couchpotato:
build:
context: ./couchpotato
dockerfile: Dockerfile
ports:
- 5050:5050
volumes:
- "./couchpotato/data:/home/CouchPotato/data/"
- "./couchpotato/config:/home/CouchPotato/config/"
As for why compose doesn't see this variable, that depends on your shell. Compose looks for an exported environment variable, contents of the .env file, and command line flags to the docker-compose command. If each of those comes up empty for the variable, you'll get that warning.
My advice: change all $PWD to .
$PWD will not work if you are running using sudo. Try the recommended settings from Docker for Linux https://docs.docker.com/engine/install/linux-postinstall/.
Sudo will run as a different user, with a different env.
$ sudo env | grep -i pwd
$ env | grep -i pwd
PWD=/home/user
OLDPWD=/
If you really need absolute paths, then call this before calling docker-compose up:
set PWD=%CD%
I had the same issue with one of my env vars. On looking at my bashrc file more closely, I found out that I hadn't exported that variable.
Before:
VAR=<value>
After:
export VAR=<value>
I recently upgraded my Docker Toolbox on Windows 10, and now my volume mounts no longer work. I've tried everything. Here is the current mount path:
volumes:
- C:\Users\Joey\Desktop\backend:/var/www/html
I receive an invalid bind mount error.
Use:
volumes:
- "C:/Users/Joey/Desktop/backend:/var/www/html"
Putting the whole thing in double quotes and using forward slashes worked for me.
I was on windows 10 in windows 10 using Linux containers through WSL2
This answer was from Spenhouet given here.
Share nfs path using docker settings
2. execute following command
docker run --rm -v c:/Users:/data alpine ls /data
Set path in docker compose file as shown below
File copied to windows
I think you have to set COMPOSE_CONVERT_WINDOWS_PATHS=1, see here.
Docker Machine should do it automatically: https://github.com/docker/machine/pull/3830
I faced with same issue (I'm using Docker Desktop).
My steps were:
1) Place your folder under drive "C"
2) Open "Settings" in Docker Desktop -> "Shared Drives" -> "Reset Credentials" -> select drive "C" -> "Apply"
3) Open terminal and run (as proposed by Docker Desktop):
docker run --rm -v c:/Users:/data alpine ls /data
4) Open your docker-compose.yml and update path in -volumes:
volumes:
- /data/YOUR_USERNAME/projects/my_project/jssecacerts:/usr/lib/jvm/java-1.8-openjdk/jre/lib/security/jssecacerts/
5) restart docker container
This solution worked for me, in docker-compose.yml :
volumes:
- c/Users/Cyril/django:/mydjango
(Windows 10 with WSL2 and Docker Desktop)
It seems you are using an absolute path located inside C:\Users dir, that didn't work for me either, and if you are using Docker-Toolbox see below.
Overview
Forwarding the ./ relative path in volumes section will automatically get resolved by docker-compose to the directory containing docker-compose.yml file (for example, if your project is in %UserProfile%/my-project then ./:/var/www/html gets /c/Users/my-name/my-project:/var/www/html).
The problem is that currently (using DockerToolbox-19.03.1) only the /c/Users directory gets shared with the Virtual-Machine (toolbox puts docker itself in the VM, which means it has no access to your file system, except mounted shared-directories).
Conclusion
So, basically placing your project there (C:\Users\YOUR_USER_NAME) should make ./ work.
But not even that worked for me, and we ended up with below _prepare.sh script:
#!/bin/bash
VBoxManage='/c/Program Files/Oracle/VirtualBox/VBoxManage'
# Defines variables for later use.
ROOT=$(dirname $0)
ROOT=$(cd "$ROOT"; pwd)
MACHINE=default
PROJECT_KEY=shared-${ROOT##*/}
# Prepares machine (without calling "docker-machine stop" command).
#
if [ $(docker-machine status $MACHINE 2> /dev/null) = 'Running' ]; then
echo Unmounting volume: $ROOT
eval $(docker-machine env $MACHINE)
docker-compose down
docker-machine ssh $MACHINE <<< '
sudo umount "'$ROOT'";
'
"$VBoxManage" sharedfolder remove $MACHINE --name "$PROJECT_KEY" -transient > /dev/null 2>&1
else
docker-machine start $MACHINE
eval $(docker-machine env $MACHINE)
fi
set -euxo pipefail
"$VBoxManage" sharedfolder add $MACHINE --name "$PROJECT_KEY" --hostpath "$ROOT" -automount -transient
docker-machine ssh $MACHINE <<< '
echo Mounting volume: '$ROOT';
sudo mkdir -p "'$ROOT'";
sudo mount -t vboxsf -o uid=1000,gid=50 "'$PROJECT_KEY'" "'$ROOT'";
'
docker-compose up -d
docker-machine ssh $MACHINE
bash
Usage:
Place a copy of it beside each project's docker-compose.yml file.
Run it each time the system is turned on (simply double-click it or its shortcut).
Done! relative paths should now work even if your project is in another drive (far away and outside of C:\Users dir).
Note:
With a little edit, it should work without docker-compose being required.
Consider running docker system prune to free disk-space (or simply add docker system prune --force to the above script, on a new line right after mount command).
On windows 10, solved the problem with adding the last one / at the end of host and mount path, like that:
volumes:
- '/c/work/vcs/app/docker/i18n/:/usr/app/target/i18n/'
Without adding the last one / mounted path contained some docker system folders and symlinks.
If you're using the new Docker WSL2 backend, some drives may not be mounted in any WSL (and so Docker won't be able to see them either). For example, D: or E: or usb drives. See
https://github.com/docker/for-win/issues/2151
https://superuser.com/questions/1114341/windows-10-ubuntu-bash-shell-how-do-i-mount-other-windows-drives
To rule out this problem, try running docker-compose from a wsl command line.
I solved it by replacing : and '' in the windows path with / at the first of the line.
to be like that:
volumes:
-/c/Users/Joey/Desktop/backend:/var/www/html
Please note: c should be small.
Is there any way to start an interactive shell in a container using Docker Compose only? I've tried something like this, in my docker-compose.yml:
myapp:
image: alpine:latest
entrypoint: /bin/sh
When I start this container using docker-compose up it's exited immediately. Are there any flags I can add to the entrypoint command, or as an additional option to myapp, to start an interactive shell?
I know there are native docker command options to achieve this, just curious if it's possible using only Docker Compose, too.
You need to include the following lines in your docker-compose.yml:
version: "3"
services:
app:
image: app:1.2.3
stdin_open: true # docker run -i
tty: true # docker run -t
The first corresponds to -i in docker run and the second to -t.
The canonical way to get an interactive shell with docker-compose is to use:
docker-compose run --rm myapp
(With the service name myapp taken from your example. More general: it must be an existing service name in your docker-compose file, myapp is not just a command of your choice. Example: bash instead of myapp would not work here.)
You can set stdin_open: true, tty: true, however that won't actually give you a proper shell with up, because logs are being streamed from all the containers.
You can also use
docker exec -ti <container name> /bin/bash
to get a shell on a running container.
The official getting started example (https://docs.docker.com/compose/gettingstarted/) uses the following docker-compose.yml:
version: "3.9"
services:
web:
build: .
ports:
- "8000:5000"
redis:
image: "redis:alpine"
After you start this with docker-compose up, you can shell into either your redis container or your web container with:
docker-compose exec redis sh
docker-compose exec web sh
docker-compose run myapp sh should do the deal.
There is some confusion with up/run, but docker-compose run docs have great explanation: https://docs.docker.com/compose/reference/run
If anyone from the future also wanders up here:
docker-compose exec service_name sh
or
docker-compose exec service_name bash
or you can run single lines like
docker-compose exec service_name php -v
That is after you already have your containers up and running.
The service_name is defined in your docker-compose.yml file
Using docker-compose, I found the easiest way to do this is to do a docker ps -a (after starting my containers with docker-compose up) and get the ID of the container I want to have an interactive shell in (let's call it xyz123).
Then it's a simple matter to execute
docker exec -ti xyz123 /bin/bash
and voila, an interactive shell.
This question is very interesting for me because I have problems, when I run container after execution finishes immediately exit and I fixed with -it:
docker run -it -p 3000:3000 -v /app/node_modules -v $(pwd):/app <your_container_id>
And when I must automate it with docker compose:
version: '3'
services:
frontend:
stdin_open: true
tty: true
build:
context: .
dockerfile: Dockerfile.dev
ports:
- "3000:3000"
volumes:
- /app/node_modules
- .:/app
This makes the trick: stdin_open: true, tty: true
This is a project generated with create-react-app
Dockerfile.dev it looks this that:
FROM node:alpine
WORKDIR '/app'
COPY package.json .
RUN npm install
COPY . .
CMD ["npm", "run", "start"]
Hope this example will help other to run a frontend(react in example) into docker container.
I prefer
docker-compose exec my_container_name bash
If the yml is called docker-compose.yml it can be launched with a simple $ docker-compose up. The corresponding attachment of a terminal can be simply (consider that the yml has specified a service called myservice):
$ docker-compose exec myservice sh
However, if you are using a different yml file name, such as docker-compose-mycompose.yml, it should be launched using $ docker-compose -f docker-compose-mycompose.yml up. To attach an interactive terminal you have to specify the yml file too, just like:
$ docker-compose -f docker-compose-mycompose.yml exec myservice sh
A addition to this old question, as I only had the case last time. The difference between sh and bash. So it can happen that for some bash doesn't work and only sh does.
So you can:
docker-compose exec CONTAINER_NAME sh
and in most cases: docker-compose exec CONTAINER_NAME bash
use.
If you have time. The difference between sh and bash is well explained here:
https://www.baeldung.com/linux/sh-vs-bash
You can do docker-compose exec SERVICE_NAME sh on the command line. The SERVICE_NAME is defined in your docker-compose.yml. For example,
services:
zookeeper:
image: wurstmeister/zookeeper
ports:
- "2181:2181"
The SERVICE_NAME would be "zookeeper".
According to documentation -> https://docs.docker.com/compose/reference/run/
You can use this docker-compose run --rm app bash
[app] is the name of your service in docker-compose.yml