When I run heroku pgbackups:restore DATABASE_URL x123 --confirm app_name
Everything seems to go smoothly in the console but the DB remains empy.
HELP!
Database stats such as table count are populated in the background. For the Starter tier that can be delayed by a few minutes. As #kch mentioned as well, there's some caching involved.
Stats for Production tier are updated more frequently, typically less than 1 minute.
Related
I have an app in Heroku which is reaching the 10K rows cap so I have to upgrade my Heroku DB, but I have a couple of doubts regarding this and I wan't to be completely sure about what I'm doing before doing anything to the production DB.
First of all I'm not 100% of this process and after a lot of time I found this link, so what I'm guessing is that I have to put my billing details in account settings, add a credit card and then after that I would be able to create a hobby-basic DB and automatically 9 dollars will be deducted from my credit card every month, am I right?
Regardin the process to change the DB of my app, the commands would be:
First I would put my app in maintenance mode
heroku maintenance:on --app my_app_name
Then I would make a DB backup and download it
heroku pg:backups:capture --app my_app_name
heroku pg:backups:download --app my_app_name
I would also get my old DB name with
heroku pg:info --app my_app_name
Then I create a new DB
heroku addons:create heroku-postgresql:hobby-basic --app my_app_name
Let's suppose HEROKU_POSTGRESQL_ROSE_URL is the name of thee new DB.
And I copy all the record from my old DB to the new one
heroku pg:copy DATABASE_URL HEROKU_POSTGRESQL_ROSE_URL --app my_app_name
To confirm I write the name of my app
my_app_name
Then I designate my new DB as the prrimary DB
heroku pg:promote HEROKU_POSTGRESQL_ROSE_URL --app my_app_name
Finally I turn off the maintenance mode
heroku maintenance:off --app my_app_name
And then I delete my old DB
heroku addons:destroy old_db_name --app my_app_name
Is everything correct?
EDIT: So I made a test creating a hobby-dev DB, but at the end of the process this new DB had like 150 more rows (before it had like 820 rows and now it have like 970 rows), is this normal?
Also, is it normal that the size of the backup is like 200 KB, while in the Heroku interface, the DB had like 10 MB of size?
I'll try to answer in the order of your questions.
Yes, you'll need to add billing information to your account before you can provision paid addons like the hobby-basic database. Once your billing information is added you will be allowed to provision the database.
The process you've detailed for the upgrade has some inefficiencies. In no particular order:
Don't wait until the middle of your upgrade process to create your new database. It can be done ahead of time.
Step 2 is unnecessary. There's no need to download a backup in the middle of this process.
In Step 5, you have ROSE database listed as the target but the new database may have a different color. This information is available from pg:info
Step 6 is also unnecessary. You can confirm in one line like so, heroku pg:copy DATABASE_URL HEROKU_POSTGRESQL_ROSE_URL --app my_app_name --confirm my_app_name
It is normal for a backup dump file to be much smaller than the existing database. Dump files are compressed binary files that don't include table bloat or indexes. In some cases the Dump file might be 10% the size of the original database.
Is it possible to scale down a Heroku DB from production to hobby (the free Dev plan), as long as we stay within the row limits? A site I'm working on requires a production-grade DB for a few weeks, but then it'll be quiet for a while. Haven't been able to find any info on this.
It shouldn't be a problem. Heroku has a guide to upgrade with backups, so I'd recommend taking a backup of your current database, downloading it to your local computer, then spinning up a development database.
Once the development (free) DB is ready, restore from the pgbackup on your local. As long as you're under the row limit, you should be fine.
Obviously, you'd want to put the site in maintenance mode when you do all of this - but it shouldn't be down for more than 5-10 minutes.
What is the recommended way to upgrade a Heroku Postgres production database to 9.2 with minimal downtime? Is it possible to use a follower, or should we take the pgbackups/snapshots route?
Until logical followers in 9.4, you'll have to dump and restore (for the reasons Craig describes). You can simplify this with pgbackups:transfer. The direct transfer is faster than dump and restore, but know that you won't have a snapshot to keep.
The script below is basically Heroku's Using PG Backups to Upgrade Heroku Postgres Databases
with modification for pgbackups:transfer. (If you have multiple instances, say a staging server, add "-a" or "--remote" to each Heroku line to specify which server.)
# get the pgbackups plugin
heroku plugins:install git://github.com/heroku/heroku-pg-extras.git
# provision new db
heroku addons:add heroku-postgresql:crane --version=9.2
# wait for it to come online, make note of new color
heroku pg:wait
# prevent new data from arriving during dump
heroku ps:scale worker=0 web=0
heroku maintenance:on
# copy over the DB. could take a while.
heroku pgbackups:transfer OLDCOLOR NEWCOLOR
# promote new database as default for DATABASE_URL
heroku pg:promote NEWCOLOR
# start everything back up and test
heroku ps:scale worker=N web=N
heroku maintenance:off
heroku open
# remove old database
heroku addons:remove HEROKU_POSTGRESQL_OLDCOLOR
Note that if you compare your data size between them, the new one may be much smaller because of efficiencies in 9.2. (My 9.2 was about 70% of the 9.1.)
Heroku followers are, AFAIK, just PostgreSQL streaming replica servers. This means you can't use them across versions, you must have binary-compatible databases.
The same techniques should apply as ordinary PostgreSQL, except that you may not be able to use pg_upgrade on Heroku. This requires shell (ssh, etc) access as the postgres user on the system that hosts the database, so I doubt it's possible on Heroku unless they've provided a tool to run pg_upgrade for you. I can't find much information on this.
You will probably have to look at using Slony-I, Bucardo, or another trigger-based replication solution to do the upgrade unless you can find a way to run pg_upgrade on a Heroku database instance. The general idea is that you set up a new 9.2 instance, use Slony to clone data from the 9.1 instance into it, then once they're fully in sync you stop the 9.1 instance, remove the Slony triggers, and switch clients over to the 9.2 instance.
Search for more information on "postgresql low downtime upgrade slony" etc, see how you go.
Trying to reset my Rails app's shared database on Heroku.
Doing the following appears to work.
heroku pg:reset SHARED_DATABASE --confirm rabid-raccoon-2000
I get: Resetting SHARED_DATABASE (DATABASE_URL)... done
And running heroku run rake db:migrate after that appears to work as well. But when I run heroku run console, or try to use the app, it does not reflect the changes (it still uses an ancient db schema- even right after I reset it).
I've tried this with both the free 5mb free db, as well as with the $15 shared db, both to no avail. No idea what db it's working with.
My database.yml is checked into version control, but I don't see how that can be a problem.
Just deleted the app and started over. Explanations are welcome.
Just a thought... I followed the directions here to set up a beta postgresql database. The plus is that it gives me direct access to the database so I can change anything needed by my tables.
I then removed the generated .sql file with "git rm conf/evolutions/default/1.sql," committed and pushed to heroku. Happily, the app is now working!
This issue is very frustrating, especially since it mostly affects people using Heroku for the first time (w/ the shared database). It wasn't the database script since it worked just fine on the local dev database. Hope this helps you out for next time.
A bad side of pushing to Heroku is that I must push the code (and the server restarts automatically) before running my db migrations.
This can obviously cause some 500 errors on users navigating the website having the new code without the new tables/attributes: the solution proposed by Heroku is to use the maintenance mode, but I want a way with no downside letting my webapp running everytime!
Is there a way? For example with Capistrano:
I prepare the code to deploy in a new dir
I run (backward) migrations and the old code continue to work perfectly
I swith mongrel instance to the new dir and restart the server
...and I have no downtime!
You could setup a second Heroku app which points to the same DB as your primary production app and use the secondary app to run your DB migrations without interrupting production (assuming the migrations don't break the previous version of your app).
Let's call the Heroku apps PRODUCTION and STAGING.
Your deploy sequence would become something like:
Deploy new code to STAGING
git push heroku staging
Run database migrations on STAGING (to update PROD db)
heroku run -a staging-app rake db:migrate
Deploy new code to PRODUCTION
git push heroku production
The staging app won't cost you anything since you won't need to exceed Heroku's free tier and it would be pretty trivial to setup a rake deploy script to do this for you automatically.
Good luck!
If you're able to live with two versions of the same app live at the same time, Heroku now has a preboot feature.
https://devcenter.heroku.com/articles/preboot
The only method to improve the process somewhat is what this guy suggests. This is still not a hot deploy scenario though:
http://casperfabricius.com/site/2009/09/20/manage-and-rollback-heroku-deployments-capistrano-style/
One thing I would suggest is pushing only your migrations up to Heroku first and running them before you push your codebase. This would entail committing the migrations as standalone commits and manually pushing them each time (which is not ideal). I'm very surprised there is not a better solution to this issue with all of the large apps hosted on Heroku now.
You actually will have some downtime when Heroku restarts your app. They have a new feature called Preboot that starts up new dynos before taking out the old ones: https://devcenter.heroku.com/articles/labs-preboot/
As for database migrations, that article links to this one on how to deal with that issue: http://pedro.herokuapp.com/past/2011/7/13/rails_migrations_with_no_downtime/
I first commit the migrations, run them, then push the rest of the code. Add a single file like so:
git commit -m 'added migration' -- db/migrate/2012-your-migration.rb
Heroku can't deploy by capistrano. You are block by tool released by heroku.
The no downtime system is impossible in all cases. How change your schema with big change without stop your server. If you don't stop it, you can avoid some change and your database can be inconsistent. SO the using of maintenance page is a normal solution.
If you want a little solution to avoid problem is a balancing in two server. One with only read database during your migration. You can switch to this instance during your migration avoiding the maintenance page. After your migration you come back to your master.
Right now I don't see any possibility to do this without downtime. I hate it too.
This console command does it in the smallest amount of time I can think of
git push heroku master &&
heroku maintenance:on &&
sleep 5 &&
heroku run rails db:migrate &&
sleep 3 &&
heroku ps:restart &&
heroku maintenance:off
git push heroku master to push the master branch to heroku
heroku maintenance:on to put on maintenance so no 500s
sleep 5 to let the dynos start up the new code (without it, the migration might fail)
heroku run rails db:migrate to do the actual migration
heroku ps:restart out of experience the restart makes sure the new dynos have the latest schema
heroku maintenance:off turns of the maintenance
You might have to add -a <app name> behind all heroku commands if you have multiple apps.
Just one command will run these in series in terminal on Mac OSX.