I would like to dump postgres database and make it periodical with crontab.
so I tried to run the following in a bash script:
sudo -u postgres pg_dump --dbname=postgresql://django:mypass#127.0.0.1:5432/django
I get this:
-bash: !: event not found
probably because there are special characters in the password. how to escape from special characters? how can I pass the password parameter? is there any other way to auto dump periodically from postgresql.
You must encode the special characters.
If your password is a!b#c , the command would be
sudo -u postgres pg_dump --dbname=postgresql://django:a%21b%40c#127.0.0.1:5432/django
Related
I am trying to use a one word alias to access a postgres database, the problem I am having is it needs a carriage return to work. Can anyone help me to get it to log in without the carriage return?
This is my alias which is in an environment file I am dotting:
alias pgres='psql --host cloudpoc.test.com.au -p 8367 -U kevin -W kev_db'
This is my .pgpass file which is in the users home directory:
cloudpoc.test.com.au:8367:kev_db:kevin:kevpass123
If I type "pgres" then I get a prompt for a password. I have to hit enter again to log in.
Shouldn't it just log me straight in?
Prompting for a password is what -W is for. If you don't want that behavior, then don't use -W.
You can specify the database connection parameters in a postgresql service file called ~/.pg_service.conf in you home. Based on your connection string:
cloudpoc.test.com.au:8367:kev_db:kevin:kevpass123
You would enter the following in ~/.pg_service.conf:
# Kevin's database service
[kev]
host=cloudpoc.test.com.au
port=8367
user=kevin
dbname=kev_db
password=kevpass123
And you would then connect to the database with:
psql service=kev
I'm trying to connect to a postgres database using the URL format in a bash script:
psql "postgresql://larryq:password!#localhost:5432/postgres"
..however when I do so I get a message about a !#localhost event not found.
I've looked at the documentation but am not sure what's going on or how to escape the exclamation point. I've tried single quoting around it, I've tried a backslash before it but nothing has worked.
Using the same format, but with a user without an exclamation point in their password, works great. How can I pull it off with an exclamation in there?
You could store the password in PGPASSWORD and call psql:
edb=# create role foouser with password 'abc123!';
CREATE ROLE
edb=# alter user foouser with login;
ALTER ROLE
edb=> \q
[root#dba /]# PGPASSWORD='abc123!' psql "postgresql://foouser#localhost:5432/edb"
psql.bin (10.10.18)
Type "help" for help.
edb=>
or
[root#dba /]# PD='abc123!'
[root#dba /]# echo "postgresql://foouser:${PD}#localhost:5432/edb"
postgresql://foouser:abc123!#localhost:5432/edb
[root#dba /]# psql "postgresql://foouser:${PD}#localhost:5432/edb"
psql.bin (10.10.18)
Type "help" for help.
edb=>
I can connect to a mongo server using this command:
mongo host:port/admin --username=user --password=pass
and I get a mongo shell where I can execute any type of mongo commands after
I switch to the dev db (use dev).
I would like to execute a mongo command from the unix shell / command line
on the dev database, but I have access only to admin.
How can I do that? How can I specify the db I would like to use?
If you wish to execute just a single command, you could do it with:
mongo server:22021/my_dbname --eval "printjson(db.serverStatus())"
If you wish to execute multiple commands say residing in a file called command.js, you could do something like:
mongo server:22021/my_dbname --quiet commands.js
If you want pure shell script (SQL) style, then you could do something like:
function testMongoScript {
mongo server:22021/my_dbname <<EOF
use mydb
db.leads.findOne()
db.leads.find().count()
EOF
}
For further details, you could refer here as well.
You can use output redirection to run whatever command you need:
mongo localhost:27017 <<< $'rs.slaveOk()\nshow dbs'
or
echo $'rs.slaveOk()\nshow dbs' | mongo localhost:27017
just remember the \n between commands.
Use --authenticationDatabase to authenticate to the database where your user exists even though you are connecting to a different database:
mongo localhost:27017/dev -u user -p **** --authenticationDatabase admin
Naturally, you won't be able to do anything in the dev database your user doesn't have permissions to do.
I am just getting started with shell scripts to save me typing in the same commands over and over. This command is used to copy a database over to a slave server as part of setting up MySQL database replication.
It works when typed into the command prompt directly:
mysqldump --host=192.168.1.1 –uUSER –pPASSWORD --opt database_name | mysql --host=192.168.1.2 –uUSER –pPASSWORD -C database_name
USER, PASSWORD and database_name all are replaced with their actual values in the real script.
When I type this command into a scripts.sh file, give it the execute permission, and then run it with ./scripts.sh I get:
'RROR1102 (42000): Incorrect database name 'database_name
mysqldump: Got errno 32 on write
What could be causing this error? Do I need to modify the command somehow when it is contained in a shell script?
The variable your database name is in has a CR at the end. You may need to run your script through dos2unix, or use one of the solutions on this site for stripping CRs from data if you're getting the database name from an external source.
I am trying to automate a set of procedures that create TEMPLATE databases.
I have a set of files (file1, file2, ... fileN), each of which contains a set of pgsql commands required for creating a TEMPLATE database.
The contents of the file (createdbtemplate1.sql) looks roughly like this:
CREATE DATABASE mytemplate1 WITH ENCODING 'UTF8';
\c mytemplate1
CREATE TABLE first_table (
--- fields here ..
);
-- Add C language extension + functions
\i db_funcs.sql
I want to be able to write a shell script that will execute the commands in the file, so that I can write a script like this:
# run commands to create TEMPLATE db mytemplate1
# ./groksqlcommands.sh createdbtemplate1.sql
for dbname in foo foofoo foobar barbar
do
# Need to simply create a database based on an existing template in this script
psql CREATE DATABASE $dbname TEMPLATE mytemplate1
done
Any suggestions on how to do this? (As you may have guessed, I'm a shell scripting newbie.)
Edit
To clarify the question further, I want to know:
How to write groksqlcommands.sh (a bash script that will run a set of pgsql cmds from file)
How to create a database based on an existing template at the command line
First off, do not mix psql meta-commands and SQL commands. These are separate sets of commands. There are tricks to combine those (using the psql meta-commands \o and \\ and piping strings to psql in the shell), but that gets confusing quickly.
Make your files contain only SQL commands.
Do not include the CREATE DATABASE statement in the SQL files. Create the db separately, you have multiple files you want to execute in the same template db.
Assuming you are operating as OS user postgres and use the DB role postgres as (default) Postgres superuser, all databases are in the same DB cluster on the default port 5432 and the role postgres has password-less access due to an IDENT setting in pg_hba.conf - a default setup.
psql postgres -c "CREATE DATABASE mytemplate1 WITH ENCODING 'UTF8'
TEMPLATE template0"
I based the new template database on the default system template database template0. Basics in the manual here.
Your questions
How to (...) run a set of pgsql cmds from file
Try:
psql mytemplate1 -f file
Example script file for batch of files in a directory:
#! /bin/sh
for file in /path/to/files/*; do
psql mytemplate1 -f "$file"
done
The command option -f makes psql execute SQL commands in a file.
How to create a database based on an existing template at the command line
psql -c 'CREATE DATABASE my_db TEMPLATE mytemplate1'
The command option -c makes psql execute a single SQL command string. Can be multiple commands, terminated by ; - will be executed in one transaction and only the result of the last command returned.
Read about psql command options in the manual.
If you don't provide a database to connect to, psql will connect to the default maintenance database named "postgres". In the second answer it is irrelevant which database we connect to.
you can echo your commands to the psql input:
for dbname in foo foofoo foobar barbar
do
echo """
CREATE DATABASE $dbname TEMPLATE mytemplate1
""" | psql
done
If you're willing to go the extra mile, you'll probably have more success with sqlalchemy. It'll allow you to build scripts with python instead of bash, which is easier and has better control.
As requested in the comments: https://github.com/srathbun/sqlCmd
Store your sql scripts under a root dir
Use dev,tst,prd parametrized dbs
Use find to run all your pgsql scripts as shown here
Exit on errors
Or just git clone the whole tool from here
For that use case where you have to do it....
Here is a script I've used for importing JSON into PostgreSQL (WSL Ubuntu), which basically requires that you mix psql meta commands and SQL in the same command line. Note use of the somewhat obscure script command, which allocates a pseudo-tty:
$ more update.sh
#!/bin/bash
wget <filename>.json
echo '\set content `cat $(ls -t <redacted>.json.* | head -1)` \\ delete from <rable>; insert into <table> values(:'"'content'); refresh materialized view <view>; " | PGPASSWORD=<passwd> psql -h <host> -U <user> -d <database>
$