i try to get an auto backup script running.
The script works perfectly, if i run it by typing "bash backup.su" into the terminal.
For every query and operation I have to input the password (database user), but it works fine.
But as soon as I don't use the terminal (eg via cron), it doesn't. It just prints out my logfile without any information.
I think it has something to do with user rights
Terminal output:
Making backup directory in /scripts/2013-09-26-daily
Performing schema-only backups
------------------------------------------
Password for user backupuser: <- Query getting all databases with certain string in name
The following databases were matched for schema-only backup:
Performing full backups
------------------------------------------
Password for user backupuser: <- Query getting all databases
Plain backup of DB1
Password:
Plain backup of DB2
Password:
Plain backup of AndSoOn
Password:
All database backups complete!
Cron Log:
Making backup directory in /scripts/2013-09-26-daily/
Performing schema-only backups
------------------------------------------
The following databases were matched for schema-only backup:
Performing full backups
------------------------------------------
All database backups complete!
As you can see there seems to be no query executed. Right now i'm using a special backup user, but it also doesn't work with postgres. (With the postgres user he also ask for a password... but an empty one stops the script)
Does anybody have a clue for me? As i said it works perfectly manually, bot doesn't with cron.
Greetings
Martin
You can create a .pgpass in the user's home directory that you want to use for the cron job. The .pgpass format should be:
hostname:port:database:username:password
In practice, I found that you need to use a wildcard ("*") for the hostname, port, and database for cron jobs to work. For example, if you are running the cron job as the postgres user and the user home directory is /home/postgres, then create a file called ".pgpass" in /home/postgres with the following content:
*:*:*:postgres:password
Your cron jobs running under the postgres user should then work without needing a password.
Official documentation for this password file is here:
http://www.postgresql.org/docs/8.4/static/libpq-pgpass.html
When running interactively you have a tty, but when running from cron there is no tty attached, i.e. your script has no stdin to read from (that would block the script while waiting for input that can never be entered from anywhere).
You should make your script runnable without a password, e.g. run it under a specific user account and configure postgres to allow that user to connect without a password (i.e. edit pg_hba.conf).
Related
I'm trying to execute several different PostgreSQL commands inside of different bash scripts. I thought I had the .pgpass file properly configured, but when I try to run pg_dump, vacuumdb, or reindexdb, I get errors about how a password isn't being supplied. For my bash script to execute properly, I need these commands to return an exit code of 0.
I'm running PostgreSQL 9.5.4 on macOS 10.12.6 (16G1408).
In an admin user account [neither root nor postgres], I have a .pgpass file in ~. The .pgpass file contains:
localhost:5432:*:postgres:DaVinci
The user is indeed postgres and the password is indeed DaVinci.
Permissions on the .pgpass file are 600.
In the pg_hba.conf file, I have:
# pg_hba.conf file has been edited by DaVinci Project Server. Hence, it is recommended to not edit this file manually.
# TYPE DATABASE USER ADDRESS METHOD
local all all md5
host all all 127.0.0.1/32 md5
host all all ::1/128 md5
So, for example, from a user account [neither root nor postgres], I run:
/Library/PostgreSQL/9.5/pgAdmin3.app/Contents/SharedSupport/pg_dump --host localhost --username postgres testworkflow13 --blobs --file /Users/username/Desktop/testdestination1/testworkflow13_$(date "+%Y_%m_%d_%H_%M").backup --format=custom --verbose --no-password
And I get the following error:
pg_dump: [archiver (db)] connection to database "testworkflow13" failed: fe_sendauth: no password supplied
I get the same result if I run this with sudo as well.
Curiously, pg_dump does execute, and does export out a .backup file to the testdestination1 directory, but since it throws an error, if it's in a bash script, the script is halted.
Where am I going wrong? How can I make sure that the .pgpass file is being properly read so that the --no-password flag in the command works?
Please start with a read to official docs.
Also, even this topic is more than 2 years also, i strongly suggest to update to at least to version 10, anyhow nothing relevant has been changed around .pgpass
.pgpass need to be chmod 600, fine, the user that uses that must can read, so that must be the owner of that file.
Please remove the --no-password that just confuse and is not needed.
Using 127.0.0.1 instead of localhost clarify where you are going, "usually" are the same.
... from a user account [neither root nor postgres] ...
The user you are using for must have read access to .pgpass, as said, so you have to clarify that and provide that file to that user, maybe using the PGPASSFILE env variable could be useful for you.
Another way is the use of .pg_service.conf file with or without the .pgpass, for what you have written it looks like that may be more appropriate
Also you could set the PGPASSWORD in the env of the user.
Think about security, some choices look the simpliest but can expose accesses .. and as DBA I'm frankly tired about peoples that store password in visible places, printed in logs or on github or set "trust" in pg_hba and finally comes to me to say "postgreSQL is insecure".. hahaha!
Final note, you do not have a pg_hba error, in case you will have a "pg_hba" error message.
Turns out that changing all three lines in the pg_hba.conf file to the trust method of authentication solved this.
local all all trust
host all all 127.0.0.1/32 trust
host all all ::1/128 trust
Since the method is trust, the .pgpass file may be entirely irrelevant--I'm not sure, but at least I got it working.
We have a partial LDAP integration on our RHEL servers. I'm trying to create a setup script to run on new servers. The first thing I need my script to do is log into an LDAP user account so that it's home directory gets created. If I put it in a script like so (and run as root):
#!/bin/bash
su - LDAPaccount
It fails saying the user doesn't exist.
If I just run the su - LDAPacccount command, then it creates the users home directory and switches me to that user. Anyone know why running the su command in a bash script fails and how I can get around this?
For anyone who finds this. My solution was to fully qualify everything:
#!/bin/bash
/bin/su - LDAPaccount -c "/usr/bin/ls"
When I run that, it works fine.
I have installed my Postgresql database on a Windows server environment. I'd like to schedule a job using Windows Task scheduler to run every night so I need to run the following command without asking for password:
psql -U myUserName-d myDBName -c "select MyFunctionName()"
When I run the above query in my cmd shell, it asks me for password. When I enter the password manually, the function is correctly run.
So my solution is to read from the pgpass.conf file so no password is required.
Here are the things I have done to achieve this:
I created the pgpass.conf file in a directory I created in the %appdata% (AppData\Roaming\postgresql to be precise).
Here are the contents of this file:
localhost:5432:myDBName:myUserName:myPassword
I have also tried with the value 127.0.0.1 instead of localhost above.
I, then, added the an environment variable (in the user variables for administrator list) called PGPASSFILE and gave it the pgpass.conf location.
;C:\Users\administrator\AppData\Roaming\postgresql\pgpass.conf
Finally I stopped and restarted my Postgres service on Windows services and re-ran the command. But it is still asking for password.
How can I let my command know from where to read the password?
If you don't want to set the PGPASSFILE environment variable, put the password file in the standard location %APPDATA%\postgresql\pgpass.conf as described by the documentation.
I want to be able to clone the contents of our postgre production database to an ownerless local database efficiently. I've successfully done this, but it was a laborious process with the following steps
$ pg_dump [prod_db] > tempfile
[Go through tempfile manually removing all 60ish references to the owner, named 'postgres']
$ cat tempfile > psql [local_db]
Otherwise when I ran the last step, I got a bunch of SQL error messages saying ERROR: role "postgres" does not exist. I tried recreating the local db with a matching 'postgres' owner, but a) I still got the same type of errors, and b) I don't want to have an owner set for my local database if it means I'll have to log into it.
Is there a best practice/efficient way of doing this if I want to re-clone it in future?
Use the -O switch to not have an owner defined in the dump.
Not having an owner set is not normal Postgres design. To avoid having to login to your postgres database you can setup a .pgpass file. This is a plain text file and should be set with 600 permissions. The contents will look like:
hostname:port:database:username:password
Each connection can be on one line.
Additionally, for local db connections, assuming you have setup the rest of your security properly (ssh certs, etc) you can edit your pg_hba.conf file and set local connection authentication method to "trust." This is obviously not recommended for production or sensitive data. The line would look like this:
local all all trust
The default method is "peer". Unless you set your username in a .pgpass file you will still need to connect with psql -U postgres but you will not need to enter a password.
I wrote a shell script to hook into a password protected database. A couple of others want to use the same shell script to share the database access, but I don't want to expose the password that is being used. Is there a way to lock the script so that no one can read it and only execute it? It'd also be ideal for them to enter their sudo password to run the script
Is there a way to lock the script so that no one can read it and only
execute it? It'd also be ideal for them to enter their sudo password
to run the script
Sure. Let's say the script containing the credentials is /usr/local/bin/myscript. Make this file owned by a privileged user. Let's say we have a user named "credentials":
# chown credentials /usr/local/bin/myscript
# chmod 700 /usr/local/bin/myscript
Now only the credentials user can read and execute this script.
Create another script /usr/local/bin/mywrapper that looks like this:
#!/bin/sh
exec sudo -u credentials /usr/local/bin/myscript
And create the appropriate /etc/sudoers entry:
auser ALL=(credentials) /usr/local/bin/myscript
So now, user auser runs "mywrapper". This in turn uses sudo to call the real script, which will prompt auser for their password.
I think this does everything you want. We use a mechanism very much like this at my office to protect credentials (in our case, ssh private keys).
In order to execute a shell script, you have to be able to read it. There are a couple of things you could do:
Who administers this database? Could the users get their own passwords for read only access? That would be ideal because it could limit their access to a small subset of the database.
If they have to have sudo access to run this script, maybe it's possible to remove the password out of the file and into a file that has 600 permissions set. The problem is that this could leak. For example, what if they run the script with set -xv? Then, they could see the password as the script executes.
Can you create a subset of data they might be interested in outside of the database? Then, have your script access that?
Or, you might want to take a look at several read-only modules in Perl. I'm thinking something like Acme::EyeDrops or Acme::Bleach which remove all those unsightly letters and numbers from your Perl script. (And I wonder why Python programmers say Perl is unreadable?)