I have an application with multiple pg databases. Clearly, only one can be DATABASE. We're going to need to upgrade the databases, and the instructions for this include issuing a pg:promote command for the DATABASE addon.
I'm wondering if there is any magic involved with pg:promote beyond what happens with addon:rename and attachment:create/delete?
(Yes, the real issue is that I want to manage things via the API...)
Related
I have provisioned postgres on my heroku app and also installed postgres locally to maintain parity (as the documentation recommends) with the online database but I'm also not understanding how this will work. Am I supposed to be accessing a local copy of a database when running on my own computer (while building and before deploying) and then using heroku's separate postgres database once it is deployed? If it is parity, shouldn't they both be using the heroku postgres database?
In other words, will my local app (during production) and heroku app (deployed and live) be using the same online postgres database?
Thanks.
Am I supposed to be accessing a local copy of a database when running on my own computer (while building and before deploying) and then using heroku's separate postgres database once it is deployed?
Yes, that's exactly it. Without seeing what bit of documentation you're referencing it's hard to say what they mean but perhaps there's another way to explain it.
In your local development environment, you may find that you need to test database schema changes (this is just one example, there are many). If you only had the one heroku postgres database you'd be forced to test these changes in production, which might result in poor usability for your users and that doesn't even account for the possibility of making a mistake and accidentally destroying your production data. There are a number of other shortcomings and challenges with this single database configuration.
For these reasons and more, it's best to keep your production data completely separated from your development/staging/test environment by creating a local/staging database. You might reasonably ask, "What about the data? I need data to test!". There are many ways to put together your test database and which you choose will likely depend on your needs. A shortlist of possibilities:
Use a seed file to generate mock data in your db
Use a model factory (usually runs in conjunction with your testing framework)
Take a dump of your production database, anonymize and redact sensitive information and use that for local testing.
I'm trying to downgrade a heroku postgres db from standard to hobby basic. As I'm not fully using the web app currently but there is still some data in there that needs to be kept. How can I downgrade? (some downtime is fine).
Update: managed to setup and promote a new database based on the inststructions below, but i can't deprovision the old one.
heroku info shows:
Heroku's instructions for upgrading with pg:copy will also work for downgrading. Here's the summary:
Provision a new database
Enter maintenance mode to prevent database writes
Transfer data to the new database
Promote the new database
Exit maintenance mode
If your app isn't live (not being actively written to), you can skip the maintenance mode steps.
Once you've done that, you can deprovision your old database.
I'm fairly new to server administration. I have my Laravel app up and running and I want to make sure it has proper backups. I have researched some backup packages and I have settled on https://github.com/spatie/laravel-backup.
However, once the server fails, I need to know how to use the most recent backup (which will be on AWS S3) to restore the database on the rebuilt server. Are there any suggestions for guides on how to do this? I can't seem to find any unless it doesn't really require much learning and instead just a couple mySQL commands.
Thanks!
I would use replication and within Laravel i would try to switch connection to the replica database server so things can run smoothly until the problem is resolved.
Take a look at this Cross-Region Replication
A typical production environment is automatically running backups on most important things that your deployment needs in order to recover from a failure. Those parts would commonly be your database and storage folder, and configuration files.
Also when you deploy a laravel application there aren't many things that are "worth" backing up , you can choose the entire disk to be mirrored somewhere or you can schedule a backup script which run every N times and backups the things that are more important to your application.
Personally i wouldn't rely on an package from laravel to handle my backups , you can always use other backup utilities, replication and so on.
Update
Take a look at the link below:
User Guide » Amazon RDS DB Instance Lifecycle » Backing Up and Restoring
Backing Up and Restoring
You can call the API function RestoreDBInstanceFromDBSnapshot as showed on example.
But i don't think something automated exists that would auto restore or magically make everything work, you need to do a lot of security checks if something like that would even be attempted. Final word i believe a good solution manually entering or sending the request would be the most solid solution.
What is the recommended way to upgrade a Heroku Postgres production database to 9.2 with minimal downtime? Is it possible to use a follower, or should we take the pgbackups/snapshots route?
Until logical followers in 9.4, you'll have to dump and restore (for the reasons Craig describes). You can simplify this with pgbackups:transfer. The direct transfer is faster than dump and restore, but know that you won't have a snapshot to keep.
The script below is basically Heroku's Using PG Backups to Upgrade Heroku Postgres Databases
with modification for pgbackups:transfer. (If you have multiple instances, say a staging server, add "-a" or "--remote" to each Heroku line to specify which server.)
# get the pgbackups plugin
heroku plugins:install git://github.com/heroku/heroku-pg-extras.git
# provision new db
heroku addons:add heroku-postgresql:crane --version=9.2
# wait for it to come online, make note of new color
heroku pg:wait
# prevent new data from arriving during dump
heroku ps:scale worker=0 web=0
heroku maintenance:on
# copy over the DB. could take a while.
heroku pgbackups:transfer OLDCOLOR NEWCOLOR
# promote new database as default for DATABASE_URL
heroku pg:promote NEWCOLOR
# start everything back up and test
heroku ps:scale worker=N web=N
heroku maintenance:off
heroku open
# remove old database
heroku addons:remove HEROKU_POSTGRESQL_OLDCOLOR
Note that if you compare your data size between them, the new one may be much smaller because of efficiencies in 9.2. (My 9.2 was about 70% of the 9.1.)
Heroku followers are, AFAIK, just PostgreSQL streaming replica servers. This means you can't use them across versions, you must have binary-compatible databases.
The same techniques should apply as ordinary PostgreSQL, except that you may not be able to use pg_upgrade on Heroku. This requires shell (ssh, etc) access as the postgres user on the system that hosts the database, so I doubt it's possible on Heroku unless they've provided a tool to run pg_upgrade for you. I can't find much information on this.
You will probably have to look at using Slony-I, Bucardo, or another trigger-based replication solution to do the upgrade unless you can find a way to run pg_upgrade on a Heroku database instance. The general idea is that you set up a new 9.2 instance, use Slony to clone data from the 9.1 instance into it, then once they're fully in sync you stop the 9.1 instance, remove the Slony triggers, and switch clients over to the 9.2 instance.
Search for more information on "postgresql low downtime upgrade slony" etc, see how you go.
I successfully pushed my Sinatra-powered Ruby app to Heroku.
One of the files I pushed is a Ruby script which scrapes the web and puts data into a PostgreSQL database (that's the non-Sinatra one).
I set up a PostgreSQL add-on for the Heroku app, but I haven't gotten further than that.
What I'm trying to figure out is how I'd edit the scraping script (which uses the Sequel gem) to add the data it scrapes to the Heroku PostgreSQL add-on database.
I took a look this tutorial on it, but I got stuck on the first step. I'm afraid I don't understand the command prompt syntax they listed.
Furthermore, when I tried to follow their alternate instructions using PGAdmin III, I ran into another problem. The Heroku tutorial says:
As an alternative, you can also create an a dump file using the PGAdmin GUI tool. To do so, select a database from the Object Browser and click Tools > Backup. Set the filename to data.dump, use the “COMPRESS” format, and (under “Dump Options #1”) choose not to save Privilege or Tablespace.
The problem here is I see no "COMPRESS" format in PGAdmin. Instead, I just save the file "data.dump" as an "All files" type without any formatting.
I'm not sure if this is correct, and if it is, what exactly I need to do next.
Can anyone confirm that I'm on the right path, and if so, what specifically I must do next?
EDIT: For clarification, I'm trying to get my scraping script to add its scraping data to the Heroku app's PostgreSQL database. Right now, it's still written as if it were on my local machine, scraping to my local PostgreSQL database.
It looks like you can run
heroku pg:credentials DATABASE --app your-app-name
where "DATABASE" is literally the word "DATABASE". Once you have the credentials, configure your script to access that database.