What is the difference between heroku ps:exec and heroku run bash? I am just trying to understand the concept. Both seem to be establishing an SSH-tunnel to a remote container/dyno. So why does heroku ps:exec require a dyno-restart on the first use? It seems this command is more generic (since it uses a default shell), so what needs to be configured/installed for it?
heroku run bash creates a standalone (ie not associated with any particular process) that has your application code available and gives you a bash session. This is helpful for running one-off tasks like a database migration it can also be helpful to debug issues where you need to look at the filesystem.
heroku ps:exec tunnels to a dyno that is already running as part of your formation. For instance, if you had 5 web dynos you could tunnel directly to web.3 for instance. This is useful in situations where a dyno is exhibiting issues (memory pressure or high load for example). Being able to connect to the problematic dyno is very useful for debugging.
You should also note that your config vars (ie environment vars set on the heroku settings tab) are not set in heroku ps:exec session.
I can't say for certain why a restart is required but I imagine that some configuration needs to change to enable a connection to a dyno already running in the fleet.
Related
I inherited an app, and while I intend to redeploy everything correctly shortly with a host of other changes, a client of mine wants new branding up quickly on a Heroku server.
I access the server this way:
heroku run bash --app APPNAME
and I pulled down the file via wget and put it into a static files directory.
However, after doing this, the change is not reflected live. Is there anyway to make a temporary change like this without a full redeploy?
File systems on Heroku are ephemeral - the files get deleted and there is no persistence.
The Heroku filesystem is ephemeral - that means that any changes to
the filesystem whilst the dyno is running only last until that dyno is
shut down or restarted. Each dyno boots with a clean copy of the
filesystem from the most recent deploy. This is similar to how many
container based systems, such as Docker, operate.
Why Are My Files Deleted - Heroku
Is it possible, and if so, how? I'd like to be able to reach it from my existing Heroku infrastructure.
Will I need a Procfile? From what I understand it's just a standalone binary written in Go! so it shouldn't be that hard to deploy it, I'm just curious how to deploy it because I don't think I understand the ins and outs of Heroku deployment.
Heroku Dynos should not be used to deploy a database application like InfluxDB.
Dynos are ephemeral servers. Data does not persist between dyno restarts and cannot be shared with other dynos. Practically speaking, any database application deployed on a dyno is essentially useless. This is why databases on Heroku (e.g. Postgres) are all Add-ons. InfluxDB should be set up on a different platform (like, AWS EC2 or a VPS) since a Heroku Add-on is not available.
That said, it is possible to deploy InfluxDB to a Heroku dyno.
To get started, it is important to understand the concept of a 'slug'. Slugs are containers (similar to a Docker images) which hold everything needed to run a program on Heroku's infrastructure. To deploy InfluxDB, an InfluxDB slug needs to be created.* There are two ways to create a slug for Go libraries:
Create a slug directly from a Go executable as described here.**
Build the slug from source using the Heroku Go buildpack (explained below).
To build the slug from source using a buildpack, first clone the InfluxDB Github repo. Then add a Procfile at the root of the repo, which tells Heroku the command to run when the dyno starts up.
echo 'web: ./influxd' > Procfile
The Go buildpack requires all dependencies be included in the directory. Use the godep dependency tool to vendor all dependencies into the directory.
go get github.com/tools/godep
godep save
Next, commit the changes made above to the git repo.
git add -A .
git commit -m dependencies
Finally, create a new app and tell it to compile with the Go buildpack.
heroku create -b https://github.com/kr/heroku-buildpack-go.git
git push heroku master
heroku open // Open the newly created InfluxDB instance in the browser.
Heroku will show an error page. An error will be displayed because Heroku's 'web' process type requires an app to listen for incoming requests on the port described by the $PORT environment variable, otherwise it will kill the dyno. InfluxDB's API and admin panel run on ports 8086 and 8083, respectively.
Unfortunately, InfluxDB does not allow those ports to be set from environment variables, only through the config file (/etc/config.toml). A small bash script executed before InfluxDB starts up could set the correct port in the config file before InfluxDB starts up.
Another problem, Heroku only exposes one port per dyno so the API and the admin panel cannot be exposed to the internet at the same time. A smart reverse proxy could work around that issue using Heroku's X-Forwarded-Port request header.
Bottom line, do not use Heroku dynos to run InfluxDB.
* This means the benefits of a standalone Go executable are lost when deploying to Heroku, since it needs to be recompiled for Heroku's stack.
** Creating a slug directly from the InfluxDB executable does not work because there is no built-in way to listen to the right port given by Heroku in the $PORT environment variable.
I like to think anything is possible on a Heroku node when using a custom buildpack, but there are some considerations when hosting with Heroku:
ops, e.g. backup, monitoring (does it entail installing extra services, opening extra ports, etc - Heroku might get in the way here)
performance, considering dyno size
and if you need a larger dyno, cost becomes an issue. You'll get more bang for your buck when you go the IaaS route.
other "features" of a dyno, e.g. disk ephemerality
I highly recommend hosted InfluxDB or spinning up your own on a VPS, all of which you can point your existing Heroku-based apps to. It will then help to get those instances as close together as possible (i.e. same region, or co-located if possible), presuming a need for low latency between DB and app stack.
I accessed the dyno via heroku run bash and made a file foo. However, when I check from my app, it still cannot find foo. So I dug deeper by trying to install nginx on a dyno, turning on autoindex, and I can confirm that the files are different from what's accessed via heroku run bash and from nginx. Why is that? How do I put files to the filesystem where my running process is showing.
When you issue heroku run bash a new dyno is created just for this one-off, and you are given access to it. Any file you create will "disappear" once your log-off, since the Heroku file-system is ephemeral.
That means the file-system is restored to its native state whenever a new dyno is created, or a dyno is rebooted. The "native" state is what's in your slug -- the "compiled" version of your app -- whatever is built by the build-pack after you "git push" to Heroku.
If you want a read-only file available to all your Dynos, either put it in your slug (for example: by including it in git, but also by using a different build-pack), or put it somewhere all your dynos can access (like a shared database, a Redis/Memcache instance, or most logically: S3).
What is the recommended way to upgrade a Heroku Postgres production database to 9.2 with minimal downtime? Is it possible to use a follower, or should we take the pgbackups/snapshots route?
Until logical followers in 9.4, you'll have to dump and restore (for the reasons Craig describes). You can simplify this with pgbackups:transfer. The direct transfer is faster than dump and restore, but know that you won't have a snapshot to keep.
The script below is basically Heroku's Using PG Backups to Upgrade Heroku Postgres Databases
with modification for pgbackups:transfer. (If you have multiple instances, say a staging server, add "-a" or "--remote" to each Heroku line to specify which server.)
# get the pgbackups plugin
heroku plugins:install git://github.com/heroku/heroku-pg-extras.git
# provision new db
heroku addons:add heroku-postgresql:crane --version=9.2
# wait for it to come online, make note of new color
heroku pg:wait
# prevent new data from arriving during dump
heroku ps:scale worker=0 web=0
heroku maintenance:on
# copy over the DB. could take a while.
heroku pgbackups:transfer OLDCOLOR NEWCOLOR
# promote new database as default for DATABASE_URL
heroku pg:promote NEWCOLOR
# start everything back up and test
heroku ps:scale worker=N web=N
heroku maintenance:off
heroku open
# remove old database
heroku addons:remove HEROKU_POSTGRESQL_OLDCOLOR
Note that if you compare your data size between them, the new one may be much smaller because of efficiencies in 9.2. (My 9.2 was about 70% of the 9.1.)
Heroku followers are, AFAIK, just PostgreSQL streaming replica servers. This means you can't use them across versions, you must have binary-compatible databases.
The same techniques should apply as ordinary PostgreSQL, except that you may not be able to use pg_upgrade on Heroku. This requires shell (ssh, etc) access as the postgres user on the system that hosts the database, so I doubt it's possible on Heroku unless they've provided a tool to run pg_upgrade for you. I can't find much information on this.
You will probably have to look at using Slony-I, Bucardo, or another trigger-based replication solution to do the upgrade unless you can find a way to run pg_upgrade on a Heroku database instance. The general idea is that you set up a new 9.2 instance, use Slony to clone data from the 9.1 instance into it, then once they're fully in sync you stop the 9.1 instance, remove the Slony triggers, and switch clients over to the 9.2 instance.
Search for more information on "postgresql low downtime upgrade slony" etc, see how you go.
A bad side of pushing to Heroku is that I must push the code (and the server restarts automatically) before running my db migrations.
This can obviously cause some 500 errors on users navigating the website having the new code without the new tables/attributes: the solution proposed by Heroku is to use the maintenance mode, but I want a way with no downside letting my webapp running everytime!
Is there a way? For example with Capistrano:
I prepare the code to deploy in a new dir
I run (backward) migrations and the old code continue to work perfectly
I swith mongrel instance to the new dir and restart the server
...and I have no downtime!
You could setup a second Heroku app which points to the same DB as your primary production app and use the secondary app to run your DB migrations without interrupting production (assuming the migrations don't break the previous version of your app).
Let's call the Heroku apps PRODUCTION and STAGING.
Your deploy sequence would become something like:
Deploy new code to STAGING
git push heroku staging
Run database migrations on STAGING (to update PROD db)
heroku run -a staging-app rake db:migrate
Deploy new code to PRODUCTION
git push heroku production
The staging app won't cost you anything since you won't need to exceed Heroku's free tier and it would be pretty trivial to setup a rake deploy script to do this for you automatically.
Good luck!
If you're able to live with two versions of the same app live at the same time, Heroku now has a preboot feature.
https://devcenter.heroku.com/articles/preboot
The only method to improve the process somewhat is what this guy suggests. This is still not a hot deploy scenario though:
http://casperfabricius.com/site/2009/09/20/manage-and-rollback-heroku-deployments-capistrano-style/
One thing I would suggest is pushing only your migrations up to Heroku first and running them before you push your codebase. This would entail committing the migrations as standalone commits and manually pushing them each time (which is not ideal). I'm very surprised there is not a better solution to this issue with all of the large apps hosted on Heroku now.
You actually will have some downtime when Heroku restarts your app. They have a new feature called Preboot that starts up new dynos before taking out the old ones: https://devcenter.heroku.com/articles/labs-preboot/
As for database migrations, that article links to this one on how to deal with that issue: http://pedro.herokuapp.com/past/2011/7/13/rails_migrations_with_no_downtime/
I first commit the migrations, run them, then push the rest of the code. Add a single file like so:
git commit -m 'added migration' -- db/migrate/2012-your-migration.rb
Heroku can't deploy by capistrano. You are block by tool released by heroku.
The no downtime system is impossible in all cases. How change your schema with big change without stop your server. If you don't stop it, you can avoid some change and your database can be inconsistent. SO the using of maintenance page is a normal solution.
If you want a little solution to avoid problem is a balancing in two server. One with only read database during your migration. You can switch to this instance during your migration avoiding the maintenance page. After your migration you come back to your master.
Right now I don't see any possibility to do this without downtime. I hate it too.
This console command does it in the smallest amount of time I can think of
git push heroku master &&
heroku maintenance:on &&
sleep 5 &&
heroku run rails db:migrate &&
sleep 3 &&
heroku ps:restart &&
heroku maintenance:off
git push heroku master to push the master branch to heroku
heroku maintenance:on to put on maintenance so no 500s
sleep 5 to let the dynos start up the new code (without it, the migration might fail)
heroku run rails db:migrate to do the actual migration
heroku ps:restart out of experience the restart makes sure the new dynos have the latest schema
heroku maintenance:off turns of the maintenance
You might have to add -a <app name> behind all heroku commands if you have multiple apps.
Just one command will run these in series in terminal on Mac OSX.