I can't figure out how to read content of a file from a Docker container. I want to execute content of a SQL file into my PGSQL container. I tried:
docker exec -it app_pgsql psql --host=127.0.0.1 --username=foo foo < /usr/src/app/migrations/*.sql
My application is mounted in /usr/src/app. But I got an error:
bash: /usr/src/app/migrations/*.sql: No such file or directory
It seems that Bash interprets this path as an host path, not a guest one. Indeed, executing the command in two times works perfectly:
docker exec -it app_pgsql
psql --host=127.0.0.1 --username=foo foo < /usr/src/app/migrations/*.sql
I think that's more a Bash issue than a Docker one, but I'm still stuck! :)
Try and use a shell to execute that command
sh -c 'psql --host=127.0.0.1 --username=foo foo < /usr/src/app/migrations/*.sql'
The full command would be:
docker exec -it app_pgsql sh -c 'psql --host=127.0.0.1 --username=foo foo < /usr/src/app/migrations/*.sql'
try with sh -c "your long command"
Also working when piping backup to the mysql command:
cat backup.sql | docker exec -i CONTAINER /usr/bin/mysql -u root --password=root DATABASE
You can use the database client in order to connect to you container and redirect the database file, then you can perform the restore.
Here is an example with MySQL: a container running MySQL, using the host network stack. Since that the container is using the host network stack (if you don't have any restriction on your MySQL or whatever database), you can connect via localhost and performing the commands transparently
mysql -h 127.0.0.1 -u user -pyour_passwd database_name < db_backup.sql
You can do the same with PostgresSQL (Restore a postgres backup file using the command line?):
pg_restore --host 127.0.0.1 --port 5432 --username "postgres" --dbname "mydatabase" --no-password --clean "/home/dinesh/db/mydb.backup"
Seems like that "docker exec" does not support input redirection.. I will verify this and maybe open an issue for Docker Community at GitHub, if it is applicable.
Related
I am currently developing an application, in which I want to automate a testing process to speed up my development time. I use a postgres db container, and I then want to check that the preparation of the database is correct.
My process is currently as follows:
docker run -p 5432:5432 --env-file=".db_env" -d postgres # Start the postgres db
# Prep the db, do some other stuff
# ...
docker exec -it CONTAINER_NAME psql -U postgres
Currently, I have to to docker ps to get the container name and then paste it and replace CONTAINER_NAME. The container is the only one running, so I am thinking I could easily find the container id or the container name automatically instead of using docker ps to manually retrieve it, but I don't know how. How do I do this using bash?
Thank you!
The container id is being returned from the docker run command:
CONTAINER_ID=$(docker run -p 5432:5432 --env-file=".db_env" -d postgres)
You can choose the name of your container with docker run --name CONTAINER_NAME.
https://docs.docker.com/engine/reference/run/#name---name
You can get its ID using:
docker ps -aqf "name=postgres"
If you're using Bash, you can do something like:
docker exec -it $(docker ps -aqf "name=postgres") psql -U postgres
In the end, I took use of #mrcl's answer, from which I developed a complete answer. Thank you for that #mrcl!
CONTAINER_ID=$(docker run -p 5432:5432 --env-file=".db_env" -d postgres)
# Do some other stuff
# ...
docker exec -it $CONTAINER_ID psql -U postgres
I have recently started using Docker. However, while I was able to run a postgres container and run a bash command "psql" inside it. Now, I am facing error in trying to do the same after sometime.
Here is what worked for me sometime back and now it does not work anymore:
docker run --rm -it postgres bash
The above command opens a bash inside the postgres container. When I type psql inside this container, it shows error:
root#3615146cf679:/# psql
psql: error: could not connect to server: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket “/var/run/postgresql/.s.PGSQL.5432”?
You need to use these commands in order:
start the container with:
$ sudo docker run --name some-postgres -e POSTGRES_PASSWORD=mysecretpassword -d postgres
enter the container with:
$ sudo docker exec -it some-postgres /bin/bash
when you entered the container, run:
$ psql -U postgres
I myself figured it out that using "bash" at the time of starting the container was causing the problem. Once we run it using:
docker run --rm postgres
Above command says that we need to provide a Password or Auth Method. Hence, we do so.
Anyone of below 3 commands can start a postgres container:
docker run --rm -e POSTGRES_PASSWORD=postgres postgres
or
docker run --rm -e POSTGRES_HOST_AUTH_METHOD=trust postgres
or
docker run --rm -e POSTGRES_HOST_AUTH_METHOD=trust -e POSTGRES_USER=postgres -e
POSTGRES_PASSWORD=postgres postgres
Then, we can execute:
docker exec -it <container_id> bash
psql -U postgres
CREATE TABLE tutorials (id int, tutorial_name text);
INSERT INTO tutorials VALUES (1, 'C++');
select * from tutorials;
Let's assume I run the following command inside a script:
#!/usr/bin/env bash
docker run --name mydb --rm -e POSTGRES_PASSWORD=kgalli -e POSTGRES_USER=kgalli -p "9999:5432" -v $PWD/db:/opt -d postgres
When I then run the following command to create a database it works fine.
docker exec -e PGPASSWORD=kgalli mydb psql -U kgalli -d template1 -c "CREATE DATABASE kgalli_test WITH OWNER kgalli ENCODING 'UTF8' LC_COLLATE = 'en_US.utf8' LC_CTYPE = 'en_US.utf8';"
However when I add this line to the script above, so the script not only starts the postgres server but also creates the database it fails.
I do not really understand why I get the following error:
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
I know I can instruct docker postgres image to create a database on start. But this is actually not what I want to achieve. I just using this as an example to understand the problem.
When you're running it in a script, it's most likely just happening too quickly. The docker run … command returns immediately, and then docker exec … is attempting to use PostgreSQL while the database server is still starting up. You need to wait for it to be ready before creating the extra database.
That said, the postgres image has functionality in its entrypoint script to run custom initialization scripts. You can put your CREATE DATABASE … statement into a .sql file or config and mount it into /docker-entrypoint-initdb.d in the container. The postgres container will automatically run it when the database server is ready.
The docs for this seems to have disappeared, but you can see the implementation in docker-entrypoint.sh.
Using docker run, you are starting a new container, using docker exec, you are executing a command in already running container
The docker run command first creates a writeable container layer over the specified image, and then starts it using the specified command.
The docker exec command runs a new command in a running container.
If the container is paused, then the docker exec command will fail with an error
$ docker pause test
test
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1ae3b36715d2 ubuntu:latest "bash" 17 seconds ago Up 16 seconds (Paused) test
$ docker exec test ls
FATA[0000] Error response from daemon: Container test is paused, unpause the container before exec
$ echo $?
1
(ref.1)
(ref.2)
I am trying to run pg_dump in a Docker container via kubectl and save the output to my local machine.
Here's what I have so far:
kubectl exec -it MY_POD_NAME -- pg_dump -h DB_HOST -U USER_NAME SCHEMA_NAME > backup.sql
However this just hangs currently. I am fairly certain it's due to the -- ignoring the >
kubectl exec -it MY_POD_NAME -- pg_dump -h DB_HOST -U USER_NAME SCHEMA_NAME outputs to the console as expected.
Use kubectl port-forward POD_NAME 6000:5342 to forward your pod port (assumed to be exposed on 5432) onto localhost:6000.
And then run pg_dump directly with hostname as localhost and port as 6000
$ pg_dump -h DB_HOST -U USER_NAME SCHEMA_NAME > backup.sql
Managed to solve myself - not the most elegant solution but it works.
First I open a shell on a pod in the cluster which has network access to the RDS instance:
kubectl exec -it psql-xxx-xxx sh
Once connected to the shell, run pg_dump to backup the database:
pg_dump -h db.internal.dns -U user schema_name > backup.sql
Once the backup completes, exit the container and copy the file from the pod to my local:
kubectl cp psql-xxx-xxx:/backup.sql ./backup.sql
Will continue searching for a streamlined way to do this.
If the database is indeed running as a docker image in remote kubernetes instance, I was successful with
kubectl exec -it POD_NAME -- pg_dump -h localhost -U DB_USER DB_NAME > backup.sql
Where localhost points to localhost in remote instance. Running directly from local terminal, this saves database dump to my local machine, no matter the database itself is running in the cloud.
I need to log in to a bash console within a docker container, which runs in a remote host.
The following commands work:
(local)$ ssh -i myKey user#remoteHost
(remote)$ docker exec -it myContainer /bin/bash
Note that I use passwordless authentication with SSH. My scenario is a bit more involved, including a script to get into a single command (which would actually also figure out docker container ID), this is enough to show the problem. When I try to run in a single command, I get the following error:
(local)$ ssh -i myKey user#remoteHost "docker exec -it myContainer /bin/bash"
cannot enable tty mode on non tty input
How can I run this SSH and get past the "cannot enable tty" error?
Use the -t option (twice) with ssh:
ssh -tt -i myKey user#remoteHost docker exec -it myContainer /bin/bash
you can use the command (from your pc) docker-machine with this you are able to connect to the docker server (if the api are exposed) an manage the docker like your local environment (docker ps, docker run etc etc)
documentation:
https://docs.docker.com/machine/