Postgres databases and roles disappeared - macos

Yesterday evening I closed my macbook's lid and left work. Came back this morning, turned my computer on, and upon trying to log into psql got a warning that the postgres role didn't exist... Upon further inspection, it seems that all but one of my databases are gone (and the default template0/1, postgres and my user's), as well as every roles but mine (<user>). \du+ in the psql console confirms my user has superuser rights. I still tried $ psql -d database_that_disappeared, to no avail. Tried switching to the other postgres versions I have installed locally (9.5.3 --> 9.6.2 --> 9.5.3), with no luck.
I obviously haven't run any brew update or upgrade, nor has OSX automatically updated anything, as I've turned automatic updates off. I have tried both shutting down and rebooting, to no avail either.
Edit: /usr/local/var/postgres/base shows 26 folders + pgsql_tmp, which makes me feel that the data itself isn't gone?

Just to let you know.
I was facing the very same problem.
Realised I had a process called "Google" that was listening to the port 5432 (found out using the command:
lsof -i -n -P | grep 5432
Besides that, my user (in the OS) disappeared as well!
Well, I found out the problem was that I did have two instances of PostgreSQL installed. When my mac restarted for an update, it started automatically the wrong one!
That looks to have happened due to updating the PGAdmin3 to 4. The 4th installs a new version of the PostgreSQL with it and, instead of replacing the previous version (as I expected) it kept both side by side.
Again, this is what I THINK has happened, because I realised I'm suddenly using a previous version of PGAdmin, with the newest version of Postgres (it even triggers a warning message saying there is a version mismatch).
I hope this has helped someone :)
And if you figure out WHY/HOW that happened, let me know.

Related

XQuartz $DISPLAY problems, cannot open xclock locally, cannot ssh -X

I am trying to solve a cascading series of bugs that started with me not able to copy to my macOS clipboard from remote ssh and has lead me to realize my X11 situation is seriously messed up. I have read a few other stackoverflow threads and they do not address my particular problems.
First my setup is macOS Mojave 10.14.5. I have xquartz 2.7.11 installed from the website. When I run echo $DISPLAY locally (on macOS) I get /private/tmp/com.apple.launchd.waagOnO6Qm/org.macosforge.xquartz:0.
Since I don't know where the error actually is I will list two problems I can identify currently.
Two problems:
If I run xclock locally nothing happens inside my terminal. I do notice that an "active" dot appears under the XQuartz dock icon for a second and then disappears. But after this happens my terminal still just hangs at xclock as though it is running.
If I try to ssh -X remote into a remote machine my terminal is locked out. I cannot keyboard interrupt. I ran this with -vvv to try to debug and I see that it hangs with xauth:
debug2: client_x11_get_proto: /opt/X11/bin/xauth -f /var/folders/jw/ltyk9x9n0_xb61jhdnct27fr0000gn/T//ssh-vcqwT7qh5yk2/xauthfile generate /private/tmp/com.apple.launchd.waagOnO6Qm/org.macosforge.xquartz:0 MIT-MAGIC-COOKIE-1 untrusted timeout 1260 2>/dev/null
Attempts to Solve
Other related stack threads have suggested reinstalling XQuartz, which I have done, both manually and with Homebrew. I have logged back out and in following reinstallation.
This thread suggested I solve my xauth problem by deleting .XAuthority file and recreating it. However, when I
xauth generate :0 . trusted
My XQuartz pops up a window saying XQuartz quit unexpectedly which I can provide the Report for if it helps. Then in the terminal it says
xauth: (argv):1: unable to open display ":0". Also I'm not sure this is the problem anyway because my .XAuthority file already contained an entry that it looks like this is trying to produce:
$HOST/unix:0 MIT-MAGIC-COOKIE-1 db7738324ca3662767b20b97b4a68680
Though it is concerning that running xauth is causing my xquartz to repeatedly quit unexpectedly (this dialog box is appearing multiple times).
This has been very frustrating to debug because I am not sure where the problem is, with xauth or xquartz somehow even though it is newly installed. Further, existing StackOverflow threads I have found detail the problem only with ssh -X but clearly I'm having problems locally, given that I can't even run xclock.
Any help is greatly appreciated.
The dot appearing and disappearing quickly indicates that the server process is terminating. This means that either the server crashed, the managing client (eg ~/.xinitrc) terminated, or we failed to even start xinit.
Almost every case I have ever seen of this has been due to someone doing something wrong in their init scripts (eg: ~/.bash*, ~/.profile, ~/.xinitrc).
Remove those and try again, then bisect to figure out the underlying issue.
However, the crash dialog from your "ssh" case indicates that it is likely the server crashing. You will need to look at the crash log for more information (or provide it here if you want help with that).

I can't run any heroku cli commands - everything is completely unresponsive

I followed the directions to uninstall and then re-install Heroku here (homebrew instructions): https://devcenter.heroku.com/articles/heroku-cli
And now whenever I type heroku <anything> everything simply hangs. I can't ctrl+break out or anything. Oddly enough, I can close the iTerm window without a warning that a process is running.
Any help would be immensely appreciated. No idea what exactly to do about this.
I filed a ticket with Heroku yesterday about the same issue.
It turns out there is currently a bug in the .netrc parser. Try moving or removing the ~/.netrc file on your local machine and then log back in. This should resolve your issue.

Issue with conflicting postgres when using homebrew

I have an issue when attempting to use postgresql along with homebrew. After doing a completely clean install (and after upgrading from postgresql 9.1.3 to 9.2.4 and doing a system update of MacOS X) it appears that the postgresql that comes with Lion is conflicting with the one that homebrew provides.
The conflict means that when OSX launches (and with the ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist), postgresql fails to load properly at startup, which then causes the
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
error. The thing is, if I then manually load and unload the homebrew.mxcl.postgresql.plist agent, postgresql works from this point onwards, does anyone know how to fix this issue (I think it may be a conflict between /usr/bin/psql and the homebrew's /usr/local/bin/psql)
EDIT: after a fresh reboot and running launchctl list | grep postgres, I get this as a result 680 - homebrew.mxcl.postgresql, running launchctl list | grep pg gives nothing, and for some odd reason, it happens to be working now (even though I havn't changed anything after posting this). Will reboot a few more times to figure out what happened
EDIT2: It actually seems to work now, I have no idea why (wasted like 3 hours on it last night), I am going to mark this as answered until it comes back again
It seems like it was actually working, maybe something odd was happening in the boot sequence but postgres is now working fine, as expected, through homebrew

psql: could not connect to server: No such file or directory (Mac OS X)

Upon restarting my Mac I got the dreaded Postgres error:
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
The reason this happened is because my macbook froze completely due to an unrelated issue and I had to do a hard reboot using the power button. After rebooting I couldn't start Postgres because of this error.
WARNING: If you delete postmaster.pid without making sure there are really no postgres processes running you, could permanently corrupt your database. (PostgreSQL should delete it automatically if the postmaster has exited.).
SOLUTION: This fixed the issue--I deleted this file, and then everything worked!
/usr/local/var/postgres/postmaster.pid
--
and here is how I figured out why this needed to be deleted.
I used the following command to see if there were any PG processes running. for me there were none, I couldn't even start the PG server:
ps auxw | grep post
I searched for the file .s.PGSQL.5432 that was in the error message above. i used the following command:
sudo find / -name .s.PGSQL.5432 -ls
this didn't show anything after searching my whole computer so the file didn't exist, but obviously psql "wanted it to" or "thought it was there".
I took a look at my server logs and saw the following error:
cat /usr/local/var/postgres/server.log
at the end of the server log I see the following error:
FATAL: pre-existing shared memory block (key 5432001, ID 65538) is still in use
HINT: If you're sure there are no old server processes still running, remove the shared memory block or just delete the file "postmaster.pid".
Following the advice in the error message, I deleted the postmaster.pid file in the same directory as server.log. This resolved the issue and I was able to restart.
So, it seems that my macbook freezing and being hard-rebooted caused Postgres to think that it's processes were still running even after reboot. Deleting this file resolved. Lots of people have similar issues but most the answers had to do with file permissions, whereas in my case things were different.
None of the above worked for me. I had to reinstall Postgres the following way :
Uninstall postgresql with brew : brew uninstall postgresql
brew doctor (fix whatever is here)
brew cleanup
Remove all Postgres folders :
rm -r /usr/local/var/postgres
rm -r ~/Library/Application\ Support/Postgres
Reinstall postgresql with brew : brew install postgresql
Start server : brew services start postgresql
You should now have to create your databases... (createdb)
If you're on macOS and installed postgres via homebrew, try restarting it with
brew services restart postgresql
If you're on Ubuntu, you can restart it with either one of these commands
sudo service postgresql restart
sudo /etc/init.d/postgresql restart
Maybe this is unrelated but a similar error appears when you upgrade postgres to a major version using brew; using brew info postgresql found out this that helped:
To migrate existing data from a previous major version of PostgreSQL run:
brew postgresql-upgrade-database
Here is my way:
launchctl unload ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
rm /usr/local/var/postgres/postmaster.pid
launchctl load ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
The following commands helped me out. The issue was with the PostgreSQL data version. Once upgraded, it started working fine for me.
brew upgrade postgresql
brew postgresql-upgrade-database
brew services restart postgresql
if your postmaster.pid is gone and you can't restart or anything, do this:
pg_ctl -D /usr/local/var/postgres -l /usr/local/var/postgres/server.log start
as explained here initially
For me, the solution was simply restart my computer. I first tried restarting with Brew services and when that didn't work, restarting seemed like the next best option to try before looking into some of the more involved solutions. Everything worked as it should after.
Another class of reasons why this can happen is due to Postgres version updates.
You can confirm this is a problem by looking at the postgres logs:
tail -n 10000 /usr/local/var/log/postgres.log
and seeing entries like:
DETAIL: The data directory was initialized by PostgreSQL version 12, which is not compatible with this version 13.0.
In this case (assuming you are on Mac and using brew), just run:
brew postgresql-upgrade-database
(Oddly, it failed on run 1 and worked on run 2, so try it twice before giving up)
Hello world :)The best but strange way for me was to do next things.
1) Download postgres93.app or other version. Add this app into /Applications/ folder.
2) Add a row (command) into the file .bash_profile (which is in my home directory):
export PATH=/Applications/Postgres93.app/Contents/MacOS/bin/:$PATH
It's a PATH to psql from Postgres93.app. The row (command) runs every time console is started.
3) Launch Postgres93.app from /Applications/ folder. It starts a local server (port is "5432" and host is "localhost").
4) After all of this manipulations I was glad to run $ createuser -SRDP user_name and other commands and to see that it worked! Postgres93.app can be made to run every time your system starts.
5) Also if you wanna see your databases graphically you should install PG Commander.app. It's good way to see your postgres DB as pretty data-tables
Of, course, it's helpful only for local server. I will be glad if this instructions help others who has faced with this problem.
This problema has many sources, and thus many answers. I've experienced each one of them.
1) If you have a crash of some sort, removing the /usr/local/var/postgres/postmaster.pid file is probably required as postgres may not have handled it properly. But ensure that no process is running.
2) Craig Ringer has pointed out in other posts that Apple's bundling of postgreSQL leads to pg gem installation issues Setting the PATH environment variable is a solution.
3) Another solution, is to uninstall and reinstall the gem. A brew update may be necessary as well.
If you stumble upon this post, if you can pinpoint one of the sources, you'll save time...
I was facing a similar issue here I solved this issue as below.
Actually the postgres process is dead, to see the status of postgres run the following command
sudo /etc/init.d/postgres status
It will says the process is dead`just start the process
sudo /etc/init.d/postgres start
This happened to me after my Mac (High Sierra) froze and I had to manually restart it (press and hold the power button). All I had to do to fix it was do a clean restart.
I had the same issue.
Most of the times, the problem is the fact that there's a leftover file
/usr/local/var/postgres/postmaster.pid
which works for most people, but my case was different - I tried googling this issue for last 3 hours, uninstalled postresql on OSX through brew, purged the database, nothing worked.
Finally, I noticed that I had an issue with brew that whenever I tried to install anything, it popped:
Error: Permission denied # rb_sysopen - /private/tmp/github_api_....
or something like it at the end of an install.
I simply did sudo chmod -R 777 /private/tmp and it finally works!
I'm writing this down because this might be a solution for someone else
I faced the same problem for psql (PostgreSQL) 9.6.11.
what worked for me -
remove postmaster.pid -- rm /usr/local/var/postgresql#9.6/postmaster.pid
restart postgres -- brew services restart postgresql#9.6
If postmaster.pid doesn't exist or the above process doesn't work then run --
sudo chmod 700 /usr/local/var/postgresql#9.6
For those running into this issue on M1 macs, try deleting this file and then restarting the brew service:
rm /opt/homebrew/var/postgres/postmaster.pid
My problem ended up being that I was using Gas Mask (a hosts file manager for Mac), and I didn't have an entry for localhost in the hosts file I was using.
I added:
127.0.0.1 localhost
And that resolved my problem.
I'm not entirely sure why, but my Postgres installation got a little bit screwed and some files were deleted resulting in the error OP is showing.
Despite the fact that I am able to run commands like brew service retart postgres and see the proper messages, this error persisted.
I went through the postgres documentation and found that my file /usr/local/var/postgres was totally empty. So I ran the following:
initdb /usr/local/var/postgres
It seems some configurations took place with that command.
Then it asked me to run this:
postgres -D /usr/local/var/postgres
And that told me a postmaster.pid file already exists.
I just needed to know if brew would be able to pick up the configs I just ran, so I tested it out.
ls /usr/local/var/postgres
That showed me a postmaster.pid file. I then did brew services stop postgresql, and the postmaster.pid file disappeared. Then I did brew services start postgresql, and VIOLA, the file reappeared.
Then I went ahead and ran my app, which did in fact find the server, however my databases seem to be gone.
Although I know that they may not be gone at all - the new initialization I did may have created a new data_area, and the old one isn't being pointed to. I'd have to look at where that's at and point it back over or just create my databases again.
Hope this helps! Reading the postgres docs helped me a lot. I hate reading answers that are like "Paste this in it works!" because I don't know what the hell is happening and why.
I had the same issue and it was due to an incompatible version after upgrading from version 11 to 13.2
Checking error log at:
/usr/local/var/log/postgres.log
Showed me:
DETAIL: The data directory was initialized by PostgreSQL version 11, which is not compatible with this version 13.2.
To fix I ran:
brew postgresql-upgrade-database
And then started postresql with brew:
brew services start postgresql
The causes of this error are many so first locate your log file and check it for clues. It might be at /usr/local/var/log/postgres.log or /usr/local/var/postgres/server.log or possibly elsewhere. If you installed with Homebrew you can find the location in ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist.
I had this same concern when connecting trying to start a PostgreSQL database server on MacOS Monterey.
When I run the command below to restart the PostgreSQL database server:
brew services restart PostgreSQL
It restarts but when I try to check the status of the PostgreSQL database server using the command below, I get an error:
Name Status User File
mysql started promisepreston ~/Library/LaunchAgents/homebrew.mxcl.mysql.plist
nginx started promisepreston ~/Library/LaunchAgents/homebrew.mxcl.nginx.plist
postgresql error 256 root ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
Here's what worked for me:
First, I checked the log file for the PostgreSQL database server to what was the cause of the error using the command below:
cat /usr/local/var/log/postgres.log
OR
nano /usr/local/var/log/postgres.log
The logs showed the following errors:
"root" execution of the PostgreSQL server is not permitted.
The server must be started under an unprivileged user ID to prevent
possible system security compromise. See the documentation for
more information on how to properly start the server.
2022-01-25 19:01:06.051 WAT [29053] FATAL: database files are incompatible with server
2022-01-25 19:01:06.051 WAT [29053] DETAIL: The data directory was initialized by PostgreSQL version 13, which is not compatible with this version 14.1.
For the root execution error I had to run the following command to fix file permissions that when messed us when I ran brew services with the sudo command prefix. Replace your-username with your MacOS username (in my case my username was promisepreston:
# Stop postgresql
sudo brew services stop PostgreSQL
# In case service file is copied to ~/Library/LaunchAgents as well
brew services stop postgresql
# Fix permission of homebrew files
sudo chown -R your-username:admin $(brew --prefix)/*
For the database files are incompatible with server I had to simply upgrade the existing PostgreSQL data files which were created using version 13 to the latest PostgreSQL version on my computer which was 14.1 by running the following command below:
brew postgresql-upgrade-database
Afterwhich, I restarted the PostgreSQL database server:
brew services restart PostgreSQL
And then checked the status using the command below:
brew services list
Then I got the output below showing that everything was working fine:
Name Status User File
mysql started promisepreston ~/Library/LaunchAgents/homebrew.mxcl.mysql.plist
nginx started promisepreston ~/Library/LaunchAgents/homebrew.mxcl.nginx.plist
postgresql started promisepreston ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
References: Brew PostgreSQL Starts But Process Is Not Running
I had a similar error.
All of this is done from the command line (no sudo calls at all)
I verified I had PostgreSQL installed psql -V (note that's a capital "V")
I attempted to connect to the server: psql postgres
THIS IS WHERE I EXPERIENCED THE ERROR OF THIS STACK OVERFLOW QUESTION
After doing some research about possible fixes, I obviously had PostgreSQL installed, but I didn't have a default server in place.
What I had to do was create a Custom Data Directory
As far as I can tell, creating the custom data directory is the same as having a default server in place.
Since this is a new machine (MacBook Pro 2021 using apple m1 chip), I wanted to find the easiest solution possible, and I believe this Custom Data Directory is just that. The remaining steps to fix this issue are as follows:
From the home directory, I created an empty directory mkdir myData
From the home directory, Initialized a server: initdb myData (throws a bunch of files into the myData directory)
pg_ctl -D myData -l logfile start (starts the server)
psql postgres (connects to the server)
So, as someone fairly new to PostgreSQL and databases and SQL in general, couple notes:
It is possible to "quit" the connection to the server, using \q (while connected to the server, it's also possible to type "help")
It is also possible to "stop" the server, as well with pg_ctl -D myData stop
At this point I now am certain I have PostgreSQL installed, have a server I can start and stop, and have the ability to connect to/disconnect from, that server.
Go to /var/log/
and run cat postgres.log
Here you will find the reason for the failure of postgres.
If it is a smart shut down then probably your icu4c version (C++ library for Unicode) is not proper which is linked with postgres. So run the following commands.
brew upgrade
brew cleanup
This should work ;)
I've had to look up this post several times to solve this issue. There's another fix, which will also applies to similar issues with other running programs.
I just discovered you can use the following two commands.
$top
This will show all the currently running actions with their pid numbers. When you find postgres and the associated pid
$kill pid_number
I just got the same issue as I have put my machine(ubuntu) for update and got below error:
could not connect to server: No such file or directory Is the server
running locally and accepting connections on Unix domain socket
"/var/run/postgresql/.s.PGSQL.5432"?
After completing the updating process when I restart my system error gone. And its work like charm as before.. I guess this was happened as pg was updating and another process started.
SUPER NEWBIE ALERT: I'm just learning web development and the particular tutorial I was following mentioned I have to install Postgres but didn't actually mention I have to run it as well... Once I opened the Postgres application everything was fine and dandy.
#Jagdish Barabari's answer gave me the clue I needed to resolve this. Turns out there were two versions of postgresql installed while only one was running. Purging all postgresql files and reinstalling the latest version resolved this issue for me.
I removed /usr/lib from the LD_LIBRARY_PATH and it worked.
I was working in dockerfile postgres:alpine.
This answer worked for me: https://stackoverflow.com/a/45454567/15067545 on my ubuntu system.
Command: sudo service postgresql restart.

PostgreSQL Permission denied Error on Unix domain socket "/var/pgsql_socket/.s.PGSQL.5432" - Lion Server 10.7.3 or Lion Server 10.7.4

I recently had major permission problems using Lion Server where permissions would change on folders at will. During this time I had started getting the following error when trying to do a rake db:migrate command:
rake aborted!
could not connect to server: Permission denied
Is the server running locally and accepting
connections on Unix domain socket "/var/pgsql_socket/.s.PGSQL.5432"?
This had been working before when I first created a Rails 3.2.3 application while running 10.7.2. All of a sudden I started having this error come up. I read many blogs about this error and tried to remove it but with no success.
I decided to call Apple Enterprise Support to see what I could do. I suspected that Lion Server had gotten corrupted but wanted to confirm with the support team before wiping my server clean and starting over. They confirmed that is what I needed to do in order to straighten it out. I did this on Friday and only installed what I needed to rebuild my Rails app. I still got the error. I tried again some of the other solutions that were posted but still none of them worked. We continued to look for solutions.
After continuing to search for blog posts we were able to find one that dealt with Lion 10.7.4. What I did not know that something had changed between 10.7.2 when I started writing my Rails application and now with PostgreSQL. We found the following blog post which helped me solve the problem I had for several days.
http://www.mactasia.co.uk/revisited-using-postgresql-in-lion-server
I ended up logging in as root and doing the following:
Made the following changes in /System/Library/LaunchDaemons/org.postgresql.postgres.plist
listen_addresses=127.0.0.1
unix_socket_permissions=0777
Changed the permissions of /var/psql_socket
sudo chmod 755 /private/var/pgsql_socket
Afterwards I logged out of root and logged in as I normally do as a server admin. I no longer got the permission error.
which psql will most likely show /bin/psql
I'm guessing we want to call /usr/local/bin/psql
So either we change the $PATH to include /local/bin prior to /bin, which I don't think is a very good solution (security and such), or
We remove /usr/bin/psql by either rm-ing (or mv-ing) the file, (and symlinking to /usr/local/bin)
or uninstalling the psql client.

Resources