I have been using heroku for previous 4-5 years, and most recently got to work on Infra as code and i was wondering which technology/methodology Heroku uses for sniping dynos or workers, I have tried searching online whether its LXC or something else but was not able to find any answer. Thanks
Heroku uses LXC:
Each application on the Heroku platform runs within its own isolated environment and cannot interact with other applications or areas of the system. This restrictive operating environment is designed to prevent security and stability issues. These self-contained environments isolate processes, memory, and the file system using LXC while host-based firewalls restrict applications from establishing local network connections.
Source: https://www.heroku.com/policy/security
Related
I am building a project which will live on a single server that will contain multiple services running side by side. I am using ansible to provision the server to automate setting everything up.
Services running:
Headless CMS
Database
Other nodejs API etc...
If in the future I would need to scale this project up, I would then want to separate the above services out onto their own servers which has led me to creating separate ansible roles for each of the above services.
My Question:
I am having real difficulty in working with pm2 to get my 2 nodejs apps running with each other.
I know that I can have a single ecosystem.config.js file containing multiple apps which would fit my current architecture (everything hosted on a single server). However this would be a pain later down the road if I were to switch one of my ansible roles to its own server.
Is there a way to deploy to production my nodejs related apps using pm2 management but in a way where they have their own configuration files and systemd service which I can define in ansible?
If I have multiple ecosystem.config.js files for each nodejs app, can pm2 mangage these with the default systemd service it offers when running:
pm2 startup
Or should I just write my own separate systemd services which I could then manually install in each ansible role through templates?
I'm really lost here and have spent so much time trying to work out the best approach to take so any help would be great!!
After doing some more research on this matter I came across this super helpful thread on pm2's github. Basically it seems that for automation using systemd services is the way to go and not bother with pm2's startup (which does create a systemd service, but more complex to manage when using automation software such as ansible.)
I strongly recommend you read it if you stumble accross this question!
https://github.com/Unitech/pm2/issues/2914
My Java application is built around an embedded database which stores db data directly to disk. I understand that Heroku by default is built on an ephemeral filesystem and anything stored in it will be removed when dynos restart or just don't stick.
What is the workaround to make such an application available and deployed in Heroku?
I understand that Heroku by default is built on an ephemeral filesystem
This isn't "by default". It's fundamental to Heroku's architecture and cannot be changed. Heroku is designed to be trivially horizontally scalable, and part of that design is that state should exist apart from any one dyno. Dynos are disposable.
What is the workaround to make such an application available and deployed in Heroku?
As far as I know, none exists. Either change how you save your data or choose another host.
(You might be able to mount a shared persistent filesystem on your dynos, but that's awkward and undermines Heroku's architecture. I don't advise it. None of Heroku's offical addons provides a persistent filesystem, and a quick search finds a few blog posts outlining attempts but I don't see any successes.)
Is there anything that I can install on my EC2 instances that make AWS Heroku-like?
e.g:
heroku create app
git push
But for AWS.
Well, now there is! It's called AWS Elastic Beanstalk (still in beta, as of March 2013)
After running the initial setup, further deployments should be as simple as git aws.push
EDIT: Just a nice and broad overview of deployment possibilities at AWS, by Werner Vogels (AWS CTO):
There are few topics I need to touch on before I can answer your question thoroughly; so please, bear with me.
A bit of insight
With respects to your two examples, Heroku utilizes a number of different technologies in order to achieve the level of simplicity it provides as a service platform. One of these technologies include: Heroku's proprietary toolbelt, which offers a set of command-line tools —that allows developers to interface with their applications— and an interact with many of the tools Heroku provides —such as terminal access for a number of different languages. The toolbelt itself relies on two other technologies: Ruby and Git; which come prepackaged with the install.
In a nutshell
Now, when you create a Heroku app you are effectively creating a git repository on the celedon cedar runtime stack (by default); this repository is then added as a remote repo. This allows you to immediately run git push heroku master. There is a lot more happening behind the scenes: for instance, when you push, your commits get intercepted by a git pre-receive hook which runs your app through a slug compiler and prepackages it for distribution across the dyno manifold; yet, I digress. For more information on more advance topics, check-out: https://devcenter.heroku.com/; there is a wealth of information here to read.
The stack
Now, let me explain the cedar stack as this is mainly what your question concerns. The Celedon cedar is one of many; however, this is the current default (for many reasons). This polyglot runtime stack currently provides six web languages (at the time of writing,) running on Ubuntu (11.04 stable, I belive). All of these technologies are operating on top of the AWS EC2 computing environment.
So to finally answer your question: You will need to install a suitable operating system such like: Ubuntu; a set of languages such like: Ruby, Python, Node.js, etc; Git (for deployment) and the rest is up to you.
If you have fixed number of instances make sense instead of usage Elastic Beanstalk use custom git deployment, like described in the article: http://www.jeffhoefs.com/2012/09/setup-git-deploy-for-aws-ec2-ubuntu-instance/.
Main idea to setup GIT repository on EC2 instance. When you want to deploy something just push your changes to remote repository, installed on EC2 instance.
I think this approach have next benefits in comparison with Elastic Beanstalk:
You don't pay for S3 buckets for storing application versions;
You have full control on application deployment steps.
Heroku is fantastic for prototyping ideas and running simple web services, I often use it to run Python web services like Flask and Django and try out ideas. However I've always struggled to understand how you can use the infrastricture to run those amazingly powerful support or utility services every startup needs in its stack. 4 exmaples of services I can't live without and would recommend to any startup.
Jenkins
Statsd
Graphite
Graylog
How would you run these on Heroku? Would it be best just getting dedicated boxes (Rackspace, e.t.c) with these support services installed.
Has anyone one run utility deamons (services) on Heroku?
There are two basic options. The first is to find or create a Heroku addon to accomplish the task. For example, there are many hosted logging solutions you can use instead of Graylog; Rails on Fire or Travis can be used instead of Jenkins. If an appropriate addon doesn't exist, you can effectively make your own by just running the service on an AWS EC2 instance.
The other alternative is to push the service into being a 12factor application so that it can run on Heroku as well. For example, you could stub out whisper's filesystem calls so that they store in a backing service instead. This is often pretty painful and brittle, though, unless you can get your changes accepted by the upstream maintainers.
you could also use another free service in conjunction with it. OpenShift has a lot of Java related build services and tools that can be added.
I am using a mix of heroku, openshift, mongolab and my own web hosting. Throw in dropbox and box for some space...
I just started learning Ruby on rails and I was wondering what Heroku really is? I know that its a cloud that helps us to avoid using servers? When do we actually use it?
Heroku is a cloud platform as a service. That means you do not have to worry about infrastructure; you just focus on your application.
In addition to what Jonny said, there are a few features of Heroku:
Instant Deployment with Git push - build of your application is performed by Heroku using your build scripts
Plenty of Add-on resources (applications, databases etc.)
Processes scaling - independent scaling for each component of your app without affecting functionality and performance
Isolation - each process (aka dyno) is completely isolated from each other
Full Logging and Visibility - easy access to all logging output from every component of your app and each process (dyno)
Heroku provides very well written tutorial which allows you to start in minutes. Also they provide first 750 computation hours free of charge which means you can have one processes (aka Dyno) at no cost. Also performance is very good e.g. simple web application written in node.js can handle around 60 - 70 requests per second.
Heroku competitors are:
OpenShift by Red Hat
Windows Azure
Amazon Web Services
Google App Engine
VMware
HP Cloud Services
Force.com
It's a cloud-based, scalable server solution that allows you to easily manage the deployment of your Rails (or other) applications provided you subscribe to a number of conventions (e.g. Postgres as the database, no writing to the filesystem).
Thus you can easily scale as your application grows by bettering your database and increasing the number of dynos (Rails instances) and workers.
It doesn't help you avoid using servers, you will need some understanding of server management to effectively debug problems with your platform/app combination. However, while it is comparatively expensive (i.e. per instance when compared to renting a slice on Slicehost or something), there is a free account and it's a rough trade off between whether it's more cost effective to pay someone to build your own solution or take the extra expense.
Heroku Basically provides with webspace to upload your app
If you are uploading a Rails app then you can follow this tutorial
https://github.com/mrkushjain/herokuapp
As I see it, it is a scalable administrated web hosting service, ready to grow in any sense so you don't have to worry about it.
It's not useful for a normal PHP web application, because there are plenty of web hosting services with ftp over there for a simple web without scalability needs, but if you need something bigger Heroku or something similar is what you need.
It is exposed as a service via a command line tool so you can write scripts to automate your deployments. Anyway it is pretty similar to other web hosting services with Git enabled, but Heroku makes it simpler.
That's its thing, to make the administration stuff simpler to you, so it saves you time. But I'm not sure, as I'm just starting with it!
A nice introduction of how it works in the official documentation is:
https://devcenter.heroku.com/articles/how-heroku-works
Per DZone: https://dzone.com/articles/heroku-or-amazon-web-services-which-is-best-for-your-startup
Heroku is a Platform as a Service (PaaS) product based on AWS, and is vastly different from Elastic Compute Cloud. It’s very important to differentiate ‘Infrastructure as a Service’ and ‘Platform as a Service’ solutions as we consider deploying and supporting our application using these two solutions.
Heroku is way simpler to use than AWS Elastic Compute Cloud. Perhaps it’s even too simple. But there’s a good reason for this simplicity. The Heroku platform equips us with a ready runtime environment and application servers. Plus, we benefit from seamless integration with various development instruments, a pre-installed operating system, and redundant servers.
Therefore, with Heroku, we don’t need to think about infrastructure management, unlike with AWS EC2. We only need to choose a subscription plan and change our plan when necessary.
That article does a good job explaining the differences between Heroku and AWS but it looks like you can choose other iaas (infrastructure) providers other than AWS. So ultimately Heroku seems to just simplify the process of using a cloud provider but at a cost.