I have a script which backups multiple tables in a single line as follow:
/usr/local/pgsql/bin/pg_dump --quote-all-identifiers --username=postgres -p 5432 -t schema.table1 -t schema.table2 -t schema.table3 -t schema.table4 -h localhost mydb | gzip -1 > file.dmp.gz
I've created a new sh script to be able to re-utilize the command as follow:
backup_table.sh
$TABLE=$1
$DESTINATION=$2
/usr/local/pgsql/bin/pg_dump --quote-all-identifiers --username=postgres -p 5432 -t $TABLE -h localhost mydb | gzip -1 > $DESTINATION
As you can see, this works for only 1 table, I'm not sure how to pass multiple tables to the sh script (-t table1 -t table2 -t table3 etc)
I could use arrays, but still, not sure how to code this.
Thanks!
If you are willing to make DESTINATION the first expected argument, then something like this should work for you:
DESTINATION=$1
TABLES=`echo ${#:2}|sed "s/\s/ -t /g"`
/usr/local/pgsql/bin/pg_dump --quote-all-identifiers --username=postgres -p 5432 -t $TABLES -h localhost mydb | gzip -1 > $DESTINATION
Related
I have a folder with many *.backup files to be imported to a Postgresql database. I've created a bash script to do it:
#!/usr/bin/env sh
echo "Importing data..."
psql -h $DB_HOST -p $DB_PORT -U $DB_USER -c "DROP DATABASE IF EXISTS $DB_DATABASE;"
psql -h $DB_HOST -p $DB_PORT -U $DB_USER -c "CREATE DATABASE $DB_DATABASE;"
cd /App/data/backup
for f in *.backup; do
echo "- $f"
pg_restore -h $DB_HOST -p $DB_PORT -U $DB_USER -d $DB_DATABASE -v "$f"
done
cd -
exit 0
Is there an easier/faster way to do this? Is it possible to import files massively?
Note: this script runs in a docker. Don't worry about the "drop database" statement :)
Assuming that you system has multiple CPUs, you can get some speedup by running the import in parallel. Since pg_restore is usually disk-bound process - the gains will be limited. Network performance will also has big impact.
Two options: using pg_restore -j, and processing multiple files in parallel:
Multiple jobs for pg_restore
for f in *.backup; do
echo "- $f"
pg_restore -j4 -h $DB_HOST -p $DB_PORT -U $DB_USER -d $DB_DATABASE -v "$f"
done
Restore multiple backup files in parallel:
ls *.backup | xargs -P4 -I{} pg_restore -h $DB_HOST -p $DB_PORT -U $DB_USER -d $DB_DATABASE -v "{}"
You want to experiments with various levels of -P, as it is hard to predict the optimal value.
I am trying to create a shell script to bootstrap new DBs.
I am able to create users, grant privileges and do all actions, except running any queries with passwords. The single quotes in shell script creates statements which postgres is not accepting.
Because of this, we cannot completely automate this process.
Below is one of the postgres line used in shell script.
PGPASSWORD=change123 psql -h $DB -p 5432 -d postgres -U root -c \"CREATE USER $(echo "$j" | cut -d "_" -f1)dbuser WITH PASSWORD \'$(echo $DBPASSWD|base64 --decode)\';\"
When executing the above script, the command is converted as
psql -h testdb -p 5432 -d postgres -U root -c '"CREATE' USER admindbuser WITH PASSWORD ''\''ZnuLEmu72R'\'''
where I want the command to be like
psql -h testdb -p 5432 -d postgres -U root -c "CREATE USER admindbuser WITH PASSWORD 'ZnuLEmu72R';"
Any help is very much appreciated. I want some help in guiding how to modify the line in shell so as to achieve the required command.
Change
PGPASSWORD=change123 psql\
-h $DB \
-p 5432 \
-d postgres \
-U root \
-c \"CREATE USER $(echo "$j" | cut -d "_" -f1)dbuser WITH PASSWORD \'$(echo $DBPASSWD|base64 --decode)\';\"
to
PGPASSWORD=change123 psql \
-h "$DB" \
-p 5432 \
-d postgres \
-U root \
-c "CREATE USER ${j%%_*}dbuser WITH PASSWORD '$(printf '%s' "$DBPASSWD" | base64 --decode)';"
I have this code $(echo "psql -U postgres -d mydb -c "SELECT * FROM table_name;" " | ssh $REMOTE_IP)
I need to run that query in the remote host, but i can't apply the query part in the echo
Any help?
Your syntax is incorrect and you don't need to use pipe. Try this:
ssh "$REMOTE_IP" 'psql -U postgres -d mydb -c "SELECT * FROM table_name;"'
Im trying to make a backup in a folder C:\Users\Marko Petričević\Documents\Radni_sati_Backup\proba where "proba" is the name of backup file.
My command looks like this:
pg_dump -h 192.168.130.240 -p 5433 -U postgres -F c postgres > C:\Users\Marko Petričević\Documents\Radni_sati_Backup\proba
and then i get an error: " pg_dump: too many command-line arguments (first is "Petričević\Documents\Radni_sati_Backup\proba") "
But, when I write a command like:
pg_dump -h 192.168.130.240 -p 5433 -U postgres -F c postgres >C:\radni_sati_backup\radni_sati_proba
Everything works, and I get that "radni_sati_proba" file in the directory as I listed in command.
Why is this happening?
Found out what the problem was:
pg_dump -h 192.168.130.240 -p 5433 -U postgres -F c postgres > C:\Users\Marko Petričević\Documents\Radni_sati_Backup\proba
needs to be like this:
pg_dump -h 192.168.130.240 -p 5433 -U postgres -F c postgres > "C:\Users\Marko Petričević\Documents\Radni_sati_Backup\proba"
Problem was the space in path.
How to atore a scalar postgresql-value on a bash-variable like in script below?
dbname="testlauf"
username="postgres"
vartest='psql -c -d $dbname -U $username -h localhost -p 5432 "SELECT gid FROM testtable WHERE aid='1';"'
echo "$vartest"
I tried several different writings, but nothing seems to work. Thanks in advance.
Put the -c option just before its argument - the query. Mind also using the additional -t option to get just the tuple value. And of course, use the backticks (`) operator.
Using the -X option is also recommended, as sometimes a .psqlrc file might add some redundant output, as well as the -A option, which disables column aligning (whitespaces).
In order to skip NOTICE or other additional messages, include the -q flag.
vartest=`psql -d $db -U $user -AXqtc "SELECT gid FROM testtable WHERE aid='1'"`
Using -t option or --tuples-only will give you the rows only, so it will easier to store them in array variable (if the result from query more than one)
vartest =(`psql -t -d $dbname -U $username -c "SELECT gid FROM testtable WHERE aid='1';"`)
echo $vartest
example:
query result
ubuntu#ratnakri:~$ psql -h localhost -p 5432 -t -U postgres -d postgres -c "select slot_name from pg_replication_slots"
barman
barman2
make it into array variable
ubuntu#ratnakri:~$ RESULT=(`psql -h localhost -p 5432 -t -U postgres -d postgres -c "select slot_name from pg_replication_slots"`)
ubuntu#ratnakri:~$ echo ${RESULT[0]}
barman
ubuntu#ratnakri:~$ echo ${RESULT[1]}
barman2