I am supposed to execute same psql command from a bash script on 5 remote machines using a username and password.
I have read that we have to pass the credentials in .pgpass file and use the -w option while executing the psql command.
But how can I execute the same command on the 5 machines using the same .pgpass file?
You can add multiple entries in .pgpass file for e.g.
syntax:
hostname:port:database:username:password
sample file:
test.net:5432:testdb:testuser:testpass
test1.net:5432:testdb1:testuser1:testpass1
test2.net:5432:testdb2:testuser2:testpass2
Make sure the permission of .pgpass file is set to 0600
chmod 0600 .pgpass
You can also use wildcards too (such as *) which is particularly handy for the database.
This means that for the pgpassfile syntax:
hostname:port:database:username:password
Can be used with value such as:
my-host:5432:*:my-username:my-plaintext-password
To enable you to connect to all databases on the server using the same credential. If you need a different credential for specific databases, then use more rows preceding this one.
Related
I'm trying to execute several different PostgreSQL commands inside of different bash scripts. I thought I had the .pgpass file properly configured, but when I try to run pg_dump, vacuumdb, or reindexdb, I get errors about how a password isn't being supplied. For my bash script to execute properly, I need these commands to return an exit code of 0.
I'm running PostgreSQL 9.5.4 on macOS 10.12.6 (16G1408).
In an admin user account [neither root nor postgres], I have a .pgpass file in ~. The .pgpass file contains:
localhost:5432:*:postgres:DaVinci
The user is indeed postgres and the password is indeed DaVinci.
Permissions on the .pgpass file are 600.
In the pg_hba.conf file, I have:
# pg_hba.conf file has been edited by DaVinci Project Server. Hence, it is recommended to not edit this file manually.
# TYPE DATABASE USER ADDRESS METHOD
local all all md5
host all all 127.0.0.1/32 md5
host all all ::1/128 md5
So, for example, from a user account [neither root nor postgres], I run:
/Library/PostgreSQL/9.5/pgAdmin3.app/Contents/SharedSupport/pg_dump --host localhost --username postgres testworkflow13 --blobs --file /Users/username/Desktop/testdestination1/testworkflow13_$(date "+%Y_%m_%d_%H_%M").backup --format=custom --verbose --no-password
And I get the following error:
pg_dump: [archiver (db)] connection to database "testworkflow13" failed: fe_sendauth: no password supplied
I get the same result if I run this with sudo as well.
Curiously, pg_dump does execute, and does export out a .backup file to the testdestination1 directory, but since it throws an error, if it's in a bash script, the script is halted.
Where am I going wrong? How can I make sure that the .pgpass file is being properly read so that the --no-password flag in the command works?
Please start with a read to official docs.
Also, even this topic is more than 2 years also, i strongly suggest to update to at least to version 10, anyhow nothing relevant has been changed around .pgpass
.pgpass need to be chmod 600, fine, the user that uses that must can read, so that must be the owner of that file.
Please remove the --no-password that just confuse and is not needed.
Using 127.0.0.1 instead of localhost clarify where you are going, "usually" are the same.
... from a user account [neither root nor postgres] ...
The user you are using for must have read access to .pgpass, as said, so you have to clarify that and provide that file to that user, maybe using the PGPASSFILE env variable could be useful for you.
Another way is the use of .pg_service.conf file with or without the .pgpass, for what you have written it looks like that may be more appropriate
Also you could set the PGPASSWORD in the env of the user.
Think about security, some choices look the simpliest but can expose accesses .. and as DBA I'm frankly tired about peoples that store password in visible places, printed in logs or on github or set "trust" in pg_hba and finally comes to me to say "postgreSQL is insecure".. hahaha!
Final note, you do not have a pg_hba error, in case you will have a "pg_hba" error message.
Turns out that changing all three lines in the pg_hba.conf file to the trust method of authentication solved this.
local all all trust
host all all 127.0.0.1/32 trust
host all all ::1/128 trust
Since the method is trust, the .pgpass file may be entirely irrelevant--I'm not sure, but at least I got it working.
I want to be able to clone the contents of our postgre production database to an ownerless local database efficiently. I've successfully done this, but it was a laborious process with the following steps
$ pg_dump [prod_db] > tempfile
[Go through tempfile manually removing all 60ish references to the owner, named 'postgres']
$ cat tempfile > psql [local_db]
Otherwise when I ran the last step, I got a bunch of SQL error messages saying ERROR: role "postgres" does not exist. I tried recreating the local db with a matching 'postgres' owner, but a) I still got the same type of errors, and b) I don't want to have an owner set for my local database if it means I'll have to log into it.
Is there a best practice/efficient way of doing this if I want to re-clone it in future?
Use the -O switch to not have an owner defined in the dump.
Not having an owner set is not normal Postgres design. To avoid having to login to your postgres database you can setup a .pgpass file. This is a plain text file and should be set with 600 permissions. The contents will look like:
hostname:port:database:username:password
Each connection can be on one line.
Additionally, for local db connections, assuming you have setup the rest of your security properly (ssh certs, etc) you can edit your pg_hba.conf file and set local connection authentication method to "trust." This is obviously not recommended for production or sensitive data. The line would look like this:
local all all trust
The default method is "peer". Unless you set your username in a .pgpass file you will still need to connect with psql -U postgres but you will not need to enter a password.
I want to write a shell script to run these commands. I usually connect from terminal using commands as below
//first go to the directory
cd /opt/novell/sentinel/3rdparty/postgresql/bin/
// then type following
export LD_LIBRARY_PATH=/opt/novell/sentinel/3rdparty/postgresql/lib/
// then fire following command
./psql --host 127.0.0.1 --port 5432 --dbname=SIEM --username=dbauser
Password for user dbauser: ****
Why don't you update your PATH and export LD_LIBRARY_PATH permanently, by adding to your .profile these lines:
PATH=/opt/novell/sentinel/3rdparty/postgresql/bin/:$PATH
export LD_LIBRARY_PATH=/opt/novell/sentinel/3rdparty/postgresql/lib/
Then use the script to connect DB as simple as following
#!/bin/sh
psql --host=127.0.0.1 --port=5432 --dbname=SIEM --username=dbauser
After you run the script, you will be asked about the password.
If you would like not to enter password every time, you can use the password file .pgpass (see documentation for details), just add to your ~/.pgpass the following line:
127.0.0.1:5432:SIEM:dbauser:your_password
Be safe, dissallow any access to world or group:
chmod 0600 ~/.pgpass.
After this, you can connect to your db by using script above without password prompt.
I have created a .pgpass file in my home directory which looks like this
localhost:5432:somedb:someuser:somepass
I am using a shell script which creates a directory and puts a pg_dump of somedb there :
mkdir directory
pg_dump somedb > directory/somefile.dump
It still prompts for the password.
Where is the mistake here ?
Did you try specifying the host, user, & db?
pg_dump -U someuser -h localhost somedb > directory/somefile.dump
Create .pgpass file with content
host:5432:somedb:someuser:somepass
set the permissions
sudo chmod 600 .pgpass
Set the file owner as the same user using which you logged in :
sudo chown login_username:login_username .pgpass
Set PGPASSFILE environment variable :
export PGPASSFILE='/home/user/.pgpass'
Now check by connecting to database :
psql -h host -U someuser somedb
It did not prompt for a password, and logged-in to postgresql.
Although question has already been answered and accepted, it may also happen that permissions on .pgpass file are not properly set. It has to have the world and group access disallowed:
/bin/chmod 0600 ~/.pgpass
psql (startup.c) calls PQconnectdbParams (fe-connect.c), and then passwordFromFile is called. Here’s a checklist to make sure the pgpass file will be used:
Make sure the flags --password/-W and password= in the connection string are unset. Otherwise, the pgpass file will not be used.
Make sure the environment variable PGPASSWORD is unset (echo $PGPASSWORD). Otherwise, the pgpass file will not be used.
Make sure the pgpass file is in the right location ($PGPASSFILE or default ~/.pgpass or %APPDATA%\postgresql\pgpass.conf)
Make sure the passfile is not readable, writable, or executable by group or other (e.g. chmod 600 ~/.pgpass); otherwise psql will print a warning.
Make sure the passfile is a file (not a symlink); otherwise psql will print a warning.
Make sure the passfile is readable by the psql user (e.g. cat ~/.pgpass); otherwise psql will ignore it without warning. Make sure that it is owned by the user, and that all its ancestor directories are executable by the user.
Make sure that the pgpass file has the correct format hostname:port:database:username:password (The Password File, passwordFromFile). Each field (other than password) can be *. The characters : and \ must be escaped \: and \\ (even in password). The password ends at : or the end of the line and can include any byte other than \r, \n, or \0. Any lines that aren’t formatted right or don’t match the host and user are ignored without warning as if they were comments.
Make sure the hostname, port, dbname, username of the line in the pgpass file match the server or are *. The server’s “pwhost” that is checked is the host name if non-empty, or the hostaddr ip address. Otherwise, the line will be ignored without warning.
Unfortunately, there is no logging within these files, so if this doesn’t help, then you may need to compile psql and libpq yourself and run it in a debugger.
Check the owner of the .pgpass file. Just lost half an hour to find out I had created my .pgpass file with sudo. The command for my user and location was chown postgres:postgres /var/lib/pgsql/.pgpass.
In my case, it was I already set PGPASSWORD environment variable (to a wrong password), so psql was picking that instead of what's inside .pgpass
If your password happens to include a colon, you have to escape it by preceding it with a \ in order for the password to be recognised.
My goal is to be able to fire off a command without having to be prompted for my password. Is there any way to achieve this from the windows command line? In Linux I feel I could send the password to standard in or something, but I am not sure if I could do this for windows.
Thanks!
There are two ways:
Set environment variable PGPASSWORD e.g. set PGPASSWORD=yoursecretpassword
Use password file %APPDATA%\postgresql\pgpass.conf as described in documentation
Within password file (my location is C:\Users\Grzesiek\AppData\Roaming\postgresql\pgpass.conf) use specified in doc format. For example to connect database postgres as role postgres on local 5432 server add:
localhost:5432:postgres:postgres:12345
I checked this and it works well (for me), but don't use 127.0.0.1).
Another handy option (specially if your PG server runs in your own client machine, and if this does not poses any security problem for you) is to allow login without password in the server ("trust" authentication mode).
For example, this line in pg_hba.conf (in your DATA dir, a typical location: C:\Program Files\PostgreSQL\9.0\data\ ) grants access without password from your local machine.
host all all 127.0.0.1/32 trust
Then, connect with
psql.exe -h 127.0.0.1 -U postgres -w [YOUR_DB_NAME]
I know it is an old question but for anyone looking like I was, it is hard to find a solution for windows. stick this in a .bat file and it will work (at least for me it did). change director to postres directory, set environment variable PGPASSWORD execute copy command to a csv file and then clear environment variable, then go back to root directory.
cd C:\Program Files\PostgreSQL\9.5\bin\
set PGPASSWORD=yourpassword
psql -d databasename -U yourusername -w -c "\COPY (select * from yourtable) TO 'c:/Users/yourdirectory/yourcsvfilename.csv' DELIMITER '|' CSV HEADER;"
set PGPASSWORD=
cd c:\
I realize this question is a bit old now, but I believe there is a better means for secure, password-free PostgreSQL logon on Windows - SSPI.
Create a local or domain user account and PostgreSQL login with the same name.
In pg_ident.conf, create a mapping:
# MAPNAME SYSTEM-USERNAME PG-USERNAME
SSPI username#AUTHORITY loginname
Replace username with the Windows user name (this is the sAMAccountName attribute for domain accounts), and replace AUTHORITY with the computer name or short domain name. If you're not sure what to use for AUTHORITY, check the PostgreSQL log file. For a local account, this will be the computer name; for a domain account, it's probably the short domain name. Lastly, replace loginname with the PostgreSQL login name (to reduce confusion, I would recommend using the same name for both username and loginname).
In pg_hba.conf, allow the logon; e.g.:
# TYPE DATABASE USER ADDRESS METHOD
host all loginname 127.0.0.1/32 sspi map=SSPI
host all loginname ::1/128 sspi map=SSPI
If a domain account, set a SPN for it; e.g.:
setspn -S POSTGRES/serverfqdn username
Now you can log onto Windows using the specified username and run psql.exe, etc. without needing a password.
Steps:
First ENviroment Var PGPASSWORD
C:\Windows\system32>set PGPASSWORD=clave
before
psql -d basededatos -U usuario
Ready,
I got it working in Postgres 15 with:
psql -U postgres -h 127.0.0.1 -p 54322 -f some_file password=postgres
J
If you're able to use Powershell, you can set environment variables inline, similar to bash.
This should work:
$Env:PGPASSWORD='your-pass'; psql -U postgres
Note the semicolon between setting the variable and actual command, this is important since those are inline but two separate commands.
I found another useful solution that worked for me on this link. It basically sets the PGPASSWORD at the beginning of your command like this:
PGPASSWORD=PASSWORD psql YOUR_CODE