I want to be able to clone the contents of our postgre production database to an ownerless local database efficiently. I've successfully done this, but it was a laborious process with the following steps
$ pg_dump [prod_db] > tempfile
[Go through tempfile manually removing all 60ish references to the owner, named 'postgres']
$ cat tempfile > psql [local_db]
Otherwise when I ran the last step, I got a bunch of SQL error messages saying ERROR: role "postgres" does not exist. I tried recreating the local db with a matching 'postgres' owner, but a) I still got the same type of errors, and b) I don't want to have an owner set for my local database if it means I'll have to log into it.
Is there a best practice/efficient way of doing this if I want to re-clone it in future?
Use the -O switch to not have an owner defined in the dump.
Not having an owner set is not normal Postgres design. To avoid having to login to your postgres database you can setup a .pgpass file. This is a plain text file and should be set with 600 permissions. The contents will look like:
hostname:port:database:username:password
Each connection can be on one line.
Additionally, for local db connections, assuming you have setup the rest of your security properly (ssh certs, etc) you can edit your pg_hba.conf file and set local connection authentication method to "trust." This is obviously not recommended for production or sensitive data. The line would look like this:
local all all trust
The default method is "peer". Unless you set your username in a .pgpass file you will still need to connect with psql -U postgres but you will not need to enter a password.
Related
I am trying to use a one word alias to access a postgres database, the problem I am having is it needs a carriage return to work. Can anyone help me to get it to log in without the carriage return?
This is my alias which is in an environment file I am dotting:
alias pgres='psql --host cloudpoc.test.com.au -p 8367 -U kevin -W kev_db'
This is my .pgpass file which is in the users home directory:
cloudpoc.test.com.au:8367:kev_db:kevin:kevpass123
If I type "pgres" then I get a prompt for a password. I have to hit enter again to log in.
Shouldn't it just log me straight in?
Prompting for a password is what -W is for. If you don't want that behavior, then don't use -W.
You can specify the database connection parameters in a postgresql service file called ~/.pg_service.conf in you home. Based on your connection string:
cloudpoc.test.com.au:8367:kev_db:kevin:kevpass123
You would enter the following in ~/.pg_service.conf:
# Kevin's database service
[kev]
host=cloudpoc.test.com.au
port=8367
user=kevin
dbname=kev_db
password=kevpass123
And you would then connect to the database with:
psql service=kev
I have a database that is owned by role postgres.
Using the psql command in bash I want to transfer the ownership of a database and all everything in it (schema, tables, views, sequences, triggers etc) to the new role.
I have:
sudo -u postgres psql << EOF
ALTER DATABASE ${database} OWNER TO ${role};
GRANT ALL ON DATABASE ${database} TO ${role};
EOF
Is that enough?
No, that won't do the trick.
It would be nice if you could use REASSIGN OWNED, but you cant't do that with the installation superuser, because the system objects cannot change ownership.
You'll have to identify all objects whose ownership you want to change and use ALTER ... OWNER TO ... for each one.
I'm trying to execute several different PostgreSQL commands inside of different bash scripts. I thought I had the .pgpass file properly configured, but when I try to run pg_dump, vacuumdb, or reindexdb, I get errors about how a password isn't being supplied. For my bash script to execute properly, I need these commands to return an exit code of 0.
I'm running PostgreSQL 9.5.4 on macOS 10.12.6 (16G1408).
In an admin user account [neither root nor postgres], I have a .pgpass file in ~. The .pgpass file contains:
localhost:5432:*:postgres:DaVinci
The user is indeed postgres and the password is indeed DaVinci.
Permissions on the .pgpass file are 600.
In the pg_hba.conf file, I have:
# pg_hba.conf file has been edited by DaVinci Project Server. Hence, it is recommended to not edit this file manually.
# TYPE DATABASE USER ADDRESS METHOD
local all all md5
host all all 127.0.0.1/32 md5
host all all ::1/128 md5
So, for example, from a user account [neither root nor postgres], I run:
/Library/PostgreSQL/9.5/pgAdmin3.app/Contents/SharedSupport/pg_dump --host localhost --username postgres testworkflow13 --blobs --file /Users/username/Desktop/testdestination1/testworkflow13_$(date "+%Y_%m_%d_%H_%M").backup --format=custom --verbose --no-password
And I get the following error:
pg_dump: [archiver (db)] connection to database "testworkflow13" failed: fe_sendauth: no password supplied
I get the same result if I run this with sudo as well.
Curiously, pg_dump does execute, and does export out a .backup file to the testdestination1 directory, but since it throws an error, if it's in a bash script, the script is halted.
Where am I going wrong? How can I make sure that the .pgpass file is being properly read so that the --no-password flag in the command works?
Please start with a read to official docs.
Also, even this topic is more than 2 years also, i strongly suggest to update to at least to version 10, anyhow nothing relevant has been changed around .pgpass
.pgpass need to be chmod 600, fine, the user that uses that must can read, so that must be the owner of that file.
Please remove the --no-password that just confuse and is not needed.
Using 127.0.0.1 instead of localhost clarify where you are going, "usually" are the same.
... from a user account [neither root nor postgres] ...
The user you are using for must have read access to .pgpass, as said, so you have to clarify that and provide that file to that user, maybe using the PGPASSFILE env variable could be useful for you.
Another way is the use of .pg_service.conf file with or without the .pgpass, for what you have written it looks like that may be more appropriate
Also you could set the PGPASSWORD in the env of the user.
Think about security, some choices look the simpliest but can expose accesses .. and as DBA I'm frankly tired about peoples that store password in visible places, printed in logs or on github or set "trust" in pg_hba and finally comes to me to say "postgreSQL is insecure".. hahaha!
Final note, you do not have a pg_hba error, in case you will have a "pg_hba" error message.
Turns out that changing all three lines in the pg_hba.conf file to the trust method of authentication solved this.
local all all trust
host all all 127.0.0.1/32 trust
host all all ::1/128 trust
Since the method is trust, the .pgpass file may be entirely irrelevant--I'm not sure, but at least I got it working.
i try to get an auto backup script running.
The script works perfectly, if i run it by typing "bash backup.su" into the terminal.
For every query and operation I have to input the password (database user), but it works fine.
But as soon as I don't use the terminal (eg via cron), it doesn't. It just prints out my logfile without any information.
I think it has something to do with user rights
Terminal output:
Making backup directory in /scripts/2013-09-26-daily
Performing schema-only backups
------------------------------------------
Password for user backupuser: <- Query getting all databases with certain string in name
The following databases were matched for schema-only backup:
Performing full backups
------------------------------------------
Password for user backupuser: <- Query getting all databases
Plain backup of DB1
Password:
Plain backup of DB2
Password:
Plain backup of AndSoOn
Password:
All database backups complete!
Cron Log:
Making backup directory in /scripts/2013-09-26-daily/
Performing schema-only backups
------------------------------------------
The following databases were matched for schema-only backup:
Performing full backups
------------------------------------------
All database backups complete!
As you can see there seems to be no query executed. Right now i'm using a special backup user, but it also doesn't work with postgres. (With the postgres user he also ask for a password... but an empty one stops the script)
Does anybody have a clue for me? As i said it works perfectly manually, bot doesn't with cron.
Greetings
Martin
You can create a .pgpass in the user's home directory that you want to use for the cron job. The .pgpass format should be:
hostname:port:database:username:password
In practice, I found that you need to use a wildcard ("*") for the hostname, port, and database for cron jobs to work. For example, if you are running the cron job as the postgres user and the user home directory is /home/postgres, then create a file called ".pgpass" in /home/postgres with the following content:
*:*:*:postgres:password
Your cron jobs running under the postgres user should then work without needing a password.
Official documentation for this password file is here:
http://www.postgresql.org/docs/8.4/static/libpq-pgpass.html
When running interactively you have a tty, but when running from cron there is no tty attached, i.e. your script has no stdin to read from (that would block the script while waiting for input that can never be entered from anywhere).
You should make your script runnable without a password, e.g. run it under a specific user account and configure postgres to allow that user to connect without a password (i.e. edit pg_hba.conf).
My goal is to be able to fire off a command without having to be prompted for my password. Is there any way to achieve this from the windows command line? In Linux I feel I could send the password to standard in or something, but I am not sure if I could do this for windows.
Thanks!
There are two ways:
Set environment variable PGPASSWORD e.g. set PGPASSWORD=yoursecretpassword
Use password file %APPDATA%\postgresql\pgpass.conf as described in documentation
Within password file (my location is C:\Users\Grzesiek\AppData\Roaming\postgresql\pgpass.conf) use specified in doc format. For example to connect database postgres as role postgres on local 5432 server add:
localhost:5432:postgres:postgres:12345
I checked this and it works well (for me), but don't use 127.0.0.1).
Another handy option (specially if your PG server runs in your own client machine, and if this does not poses any security problem for you) is to allow login without password in the server ("trust" authentication mode).
For example, this line in pg_hba.conf (in your DATA dir, a typical location: C:\Program Files\PostgreSQL\9.0\data\ ) grants access without password from your local machine.
host all all 127.0.0.1/32 trust
Then, connect with
psql.exe -h 127.0.0.1 -U postgres -w [YOUR_DB_NAME]
I know it is an old question but for anyone looking like I was, it is hard to find a solution for windows. stick this in a .bat file and it will work (at least for me it did). change director to postres directory, set environment variable PGPASSWORD execute copy command to a csv file and then clear environment variable, then go back to root directory.
cd C:\Program Files\PostgreSQL\9.5\bin\
set PGPASSWORD=yourpassword
psql -d databasename -U yourusername -w -c "\COPY (select * from yourtable) TO 'c:/Users/yourdirectory/yourcsvfilename.csv' DELIMITER '|' CSV HEADER;"
set PGPASSWORD=
cd c:\
I realize this question is a bit old now, but I believe there is a better means for secure, password-free PostgreSQL logon on Windows - SSPI.
Create a local or domain user account and PostgreSQL login with the same name.
In pg_ident.conf, create a mapping:
# MAPNAME SYSTEM-USERNAME PG-USERNAME
SSPI username#AUTHORITY loginname
Replace username with the Windows user name (this is the sAMAccountName attribute for domain accounts), and replace AUTHORITY with the computer name or short domain name. If you're not sure what to use for AUTHORITY, check the PostgreSQL log file. For a local account, this will be the computer name; for a domain account, it's probably the short domain name. Lastly, replace loginname with the PostgreSQL login name (to reduce confusion, I would recommend using the same name for both username and loginname).
In pg_hba.conf, allow the logon; e.g.:
# TYPE DATABASE USER ADDRESS METHOD
host all loginname 127.0.0.1/32 sspi map=SSPI
host all loginname ::1/128 sspi map=SSPI
If a domain account, set a SPN for it; e.g.:
setspn -S POSTGRES/serverfqdn username
Now you can log onto Windows using the specified username and run psql.exe, etc. without needing a password.
Steps:
First ENviroment Var PGPASSWORD
C:\Windows\system32>set PGPASSWORD=clave
before
psql -d basededatos -U usuario
Ready,
I got it working in Postgres 15 with:
psql -U postgres -h 127.0.0.1 -p 54322 -f some_file password=postgres
J
If you're able to use Powershell, you can set environment variables inline, similar to bash.
This should work:
$Env:PGPASSWORD='your-pass'; psql -U postgres
Note the semicolon between setting the variable and actual command, this is important since those are inline but two separate commands.
I found another useful solution that worked for me on this link. It basically sets the PGPASSWORD at the beginning of your command like this:
PGPASSWORD=PASSWORD psql YOUR_CODE