how to periodically dump a set of database tables to a local machine from a psql database in a single bash script run on Mac OS X? - macos

I tried this but it does not work.
#!/bin/bash
TABLENAMES="user_stats"
ssh -t railsapps#xxx.xxx.xxx.xx -p xxx bash -c "'
for TABLENAME in $TABLENAMES
do
psql -d mydb -P format=unaligned -P tuples_only -P fieldsep=\, -c "SELECT * FROM $TABLENAME" > /tmp/$TABLENAME
done
'"

You should use the Postgres COPY command, and stuff them all in to a single sql file, that you then run with psql. Then you can take that script and either feed it to cron on Mac OS X, or launchd to run the script periodically.

Related

Bash script, remote psql command over ssh, pipe sql file, password fails

I have a bash script. I want to run a postgres command with ssh that pipes a local file. The problem is the psql command prompts for a password, and my sql file gets piped into that. How do I write a command that pipes after I type in the password?
ssh server "psql -W -h db_host -p 5432 -U db_user -d postgres" < staging.sql
I suggest to break it down into multiple steps:
# Transfer the sql file to the server
scp staging.sql server
# Excute the queries in that file with psql over ssh
# Notes:
# - ssh -t enforces terminal allocation. You may try it without this option and see if it still works.
# - psql -f FILENAME reads commands from file
#
ssh -t server \
'psql -W -h db_host -U db_user -d postgres -f staging.sql; rm staging.sql'

Docker: "pg_restore input file does not appear to be a valid archive" error

I backup a PostgreSql database using the following command and it creates an sql file:
cmd /c 'docker exec -t <container-name> pg_dump <db_name> -U postgres -c
-v > C:\\backup\\<db_name>.sql'
However, I cannot restore the sql backup file using the following command:
first I drop and create an empty db:
docker exec <container-name> bash -c "dropdb -U postgres <db_name>"
docker exec <container-name> bash -c "createdb -U postgres <db_name>"
then restore:
cmd /c "docker exec -i <container-name> pg_restore -C -U postgres -d
<db_name> -v < C:\\<db_name>.sql"
gives "pg_restore: error: input file does not appear to be a valid archive" error. So,
1. how can I restore the database with sql file?
2. how can I backup PostgreSql db in Docker on Windows?
A plain-text pg_dump is restored with psql, not with pg_restore.
I don't understand why you want to run that inside the container. Install a PostgreSQL client on the host operating system and simplify the procedure. Besides, a backup should be on a different machine than the database.

Ways to add one more quote nesting level (docker + bash + psql)

I need to modify a value in postgreSQL table in docker using a single command. I want to run something like this inside a container:
su - postgres -c "psql -c \"\c database_name\" -c \"UPDATE table_name SET value_name = 'value';\""
It turned out to be the only way to do this because echo "\c database_name \\ UPDATE table_name SET value_name = 'value';" | psql doesn't work in my postgreSQL resulting in
invalid command \
Is there anything I can do to run it in a container via command sudo docker exec -it container_name bash -c "*command above*"? None of the quote types surrounding command above make it work. Thank you in advance!
You should be able to avoid using su by telling docker exec itself which user should run psql. You can also run psql directly, rather than executing a bash shell that executes it.
sudo docker exec --user postgres \
psql -c "\c database_name" -c "UPDATE table_name SET value_name = 'value';"

Limited printed rows when running select query using postgres bash on linux

I have a CentOS linux machine that has postgres DB installed on.
I connected to this machine via ssh and I ran the next command in order to print a certain table.
sudo -u postgres bash -c "psql -d db -c \"SELECT *
FROM accounts;\"" 2>/dev/null
this table contains 800 rows BUT the output of this command only prints 38 rows.
Can anyone tell me how I can fix it?
The solution was to change the end of the command from 2>/dev/null to >/dev/null

bash script returns error "ERROR: syntax error at end of input LINE 1: SELECT" for psql request to copy the table to an external file

What should I do for making it work?
#!/bin/bash
TABLENAMES="user_stats"
ssh -t railsapps#xxx.xxx.xxx.xx -p xxx bash -c "'
for TABLENAME in $TABLENAMES
do
psql -d mydb -P format=unaligned -P tuples_only -P fieldsep=\, -c "SELECT * FROM $TABLENAME" > /tmp/$TABLENAME
done
'"
General problem: how to periodically dump the database tables to a local machine from a psql database in a single bash script run on Mac OS X?
Firstly, you should test your SQL and bash scripts remotely (do SSH interactively).
I think your problem is caused by a bad mix of quote / double-quote. I think the star (*) and $TABLENAME are expensed before the SSH call, so too early. Try to put a backslash before the $ sign.
You should use the verbose or the debug option, to help to understand what is really executed:
ssh -t railsapps#xxx.xxx.xxx.xx -p xxx bash -vxc "'
for TABLENAME in \$TABLENAMES; do
psql -d mydb -P format=unaligned -P tuples_only -P fieldsep=\, -c "SELECT \* FROM \$TABLENAME" > /tmp/\$TABLENAME
done
'"

Resources