I need monitor apps deployed to Heroku by Prometheus monitoring system.
Problem is that if you have more dynos app, you need to know all IP address of your dynost to be able to pull metrics from all dynos.
For K8s or AWS we are able to get full list of PODs/instances, So you are able to do this.
So question is, do you know, how to get IPs of all dynos from Heroku?
I'm considering exposing the $DYNO environment variable as a label to all metrics so Prometheus can have a consistent view on which dyno it's scraping. Given a short enough scraping interval, all dynos should be able to scraped within reasonable time.
Pushgateway is not a recommended way of monitoring long-standing services.
So question is, do you know, how to get IPs of all dynos from Heroku?
This is not possible on the Heroku platform. The application dynos sit behind a load balancer, you do not have direct access to them.
I need monitor apps deployed to Heroku by Prometheus monitoring system.
Perhaps this could be split into two Prometheus jobs.
Monitor the application directly (at it's load-balanced *.herokuapp.com/metrics endpoint)
and use an exporter that gathers the dynos metrics via push somehow
Consider making use of a Heroku log drain to an exporter, converting the logs into a metrics endpoint.
There is also a private API available on Heroku application stats, not the best idea, but it might work well to gather the basic application stats. Have a look at the network requests in the Heroku dashboard to see how that works.
This would have some similarities with using a pushgateway, as described at https://prometheus.io/docs/practices/pushing/.
Related
Classic approach on GCP is rent a linux host with static monthly payment. It doesn't matter if your application is not running or users aren't consuming it, you will always pay the static monthly payment. I think this is acceptable for production environments but for development and testing not.
This does not happen on Heroku :
If an app has a free web dyno, and that dyno receives no web traffic in a 30-minute period, it will sleep. In addition to the web dyno sleeping, the worker dyno (if present) will also sleep.
Free web dynos do not consume free dyno hours while sleeping.
Question
How stop or delete app on google (gae, cloud run, cloud build, containers) if does not receive web traffic?
If it is possible using just google tools it would be great:
https://cloud.google.com/products
Idea
Developing a basic router with nodejs which works as minimal balancer. If web traffic is not detected for some apps, an instruction to google cloud platform api could stop the app or container. This would also apply to other clouds.
Any help is appreciated.
Update
I cannot find any solution yet. I will try to add that feature here https://github.com/jrichardsz/http-operator or a basic shell script to detect incoming request to a specific port like How to print incoming http request on specific port
GCP is offering several serverless products (like you mentioned) and they offer a pricing where you are only charged for the resources you use (when requests are processed).
In Cloud Run you are only billed when an instance is handling a
request using the autoscaling to know more. See their pricing as well for a better overview.
For Google App Engine the app.yaml configuration file contains several settings you can use to adjust the trade-off between performance and resource load for a specific version of your app. You
also check this link how to manage the auto scaling settings.
You can also check this Google Cloud blog for other strategies in auto scaling your applications.
To answer the Comment below:
This video can help you better understand their differences to be able to see the appropriate service for your use case.
To clarify, there's 2 variations of cloud run, the first is managed by google and the other runs on gke. As long as your classic application (api app) is stateless, you should be able to deploy it as a container and take advantage of being charged based on only the resources you use. Snippets would fall under Cloud function where it only runs functions based on triggers.
You can choose to deploy your Cloud Run app on fully managed infrastructure ("serverless", pay per use, auto-scaling up rapidly and down to 0 depending on traffic) or on a Google Kubernetes Engine cluster.
It is also possible to run Docker containers in Serverless using App Engine (Flexible). App Engine is always fully managed, with auto-scaling. App Engine Flex auto-scales gradually and down to 1. App Engine Second Generation auto-scales up rapidly and down to 0.
In your current use case I would recommend to use Cloud Run, check its limitations first before getting started. See the official documentation here and on Cloud Run How-To Guides
I am using Heroku for one of my hobby projects. I bumped into the following problem if webhook sends a request to Heroku and Java app restarts at this time, the request will be lost. Is there any native Heroku way or plugin that ensures that Heroku will get a request, at least once?
I didn't find anything so far, and the default solution is to take AWS Lambda or Google Cloud Function to ensure scalability and availability. I feel that it adds unnecessary complexity and possibly a maintenance headache.
To Summarize:
Is there any Heroku native mechanism or Heroku plugin that ensures that requests will be delivered?
Assuming your Java application restarts are related to dyno restarts or deploys:
you might want to look into the preboot feature on Heroku.
From their documentation:
Preboot changes the standard dyno start behavior for web dynos. Instead of stopping the existing set of web dynos before starting the new ones, preboot ensures that the new web dynos are started (and receive traffic) before the existing ones are terminated. This can contribute to zero downtime deployments.
#Denis Cornehl's answer helped a lot. My answer is an extension to it.
You don't have a free solution if you have free or hobby tires. You will have to follow Enterprise Integration Patterns.
My solution was to put the queue in front of the application and consume messages when the application is up. Even though there is no free queue, some cloud providers give you free quotes for big loads. I recommend considering them before deploying and maintaining your own queues.
I'm just starting with Heroku, and I'm considering using it to host my app.
The problem is, I'm planning to build my app with a micro services architecture.
As I understand Heroku's pricing, they charge per dyno. When using micro services, there are a bunch of "apps" or "services" working together to provide the functionality of one app.
It seems inevitable to do this without using multiple dynos, and by adding dynos, the price goes up very quickly.
I have three questions:
Is my interpretation of Heroku's pricing correct?
Is there a better way to do a micro services architecture using Heroku?
Or, is there a way to deploy multiple docker containers on a single dyno?
Your Heroku costs will definitely go up as you use more dynos and more apps, but if you are just starting out, the money you spend will be in no way more than the time you would spend setting up something like AWS to perform the same functionality.
You can always save money later, once your app is running and users are loving it. If you goal is to prototype and get it out fast, Heroku is still the best choice.
Did you know about Heroku price calculator?
Saw you run 10*1x dynos, plus some other services, like text messages and such. That will cost you $250-350 this month. That's a lot, sure. But you can get your app running tomorrow, and presumably that is worth a lot more than 250.
I am new to Heroku and I am trying to bootstrap a local development environment. Using Foreman, or another tool, can someone please point me to docs that illustrate sending and consuming a message with a worker. Key being setting up the MQ and the worker consuming the message configured locally. Thanks!
IronMQ (and IronWorker) are both cloud services and currently do not have local install capability. It's fairly easy to interact with the API from your local machine though including pushing messages, getting them, etc.
If you plan on using Push Queues, do keep in mind that in order to "push" back to your localhost you'll need to setup something like localtunner or ngrok. Here is some information on that: http://dev.iron.io/mq/reference/push_queues/#testing_on_localhost
Please feel free to hit us at support#iron.io or live chat: get.iron.io/chat
Chad
I just started learning Ruby on rails and I was wondering what Heroku really is? I know that its a cloud that helps us to avoid using servers? When do we actually use it?
Heroku is a cloud platform as a service. That means you do not have to worry about infrastructure; you just focus on your application.
In addition to what Jonny said, there are a few features of Heroku:
Instant Deployment with Git push - build of your application is performed by Heroku using your build scripts
Plenty of Add-on resources (applications, databases etc.)
Processes scaling - independent scaling for each component of your app without affecting functionality and performance
Isolation - each process (aka dyno) is completely isolated from each other
Full Logging and Visibility - easy access to all logging output from every component of your app and each process (dyno)
Heroku provides very well written tutorial which allows you to start in minutes. Also they provide first 750 computation hours free of charge which means you can have one processes (aka Dyno) at no cost. Also performance is very good e.g. simple web application written in node.js can handle around 60 - 70 requests per second.
Heroku competitors are:
OpenShift by Red Hat
Windows Azure
Amazon Web Services
Google App Engine
VMware
HP Cloud Services
Force.com
It's a cloud-based, scalable server solution that allows you to easily manage the deployment of your Rails (or other) applications provided you subscribe to a number of conventions (e.g. Postgres as the database, no writing to the filesystem).
Thus you can easily scale as your application grows by bettering your database and increasing the number of dynos (Rails instances) and workers.
It doesn't help you avoid using servers, you will need some understanding of server management to effectively debug problems with your platform/app combination. However, while it is comparatively expensive (i.e. per instance when compared to renting a slice on Slicehost or something), there is a free account and it's a rough trade off between whether it's more cost effective to pay someone to build your own solution or take the extra expense.
Heroku Basically provides with webspace to upload your app
If you are uploading a Rails app then you can follow this tutorial
https://github.com/mrkushjain/herokuapp
As I see it, it is a scalable administrated web hosting service, ready to grow in any sense so you don't have to worry about it.
It's not useful for a normal PHP web application, because there are plenty of web hosting services with ftp over there for a simple web without scalability needs, but if you need something bigger Heroku or something similar is what you need.
It is exposed as a service via a command line tool so you can write scripts to automate your deployments. Anyway it is pretty similar to other web hosting services with Git enabled, but Heroku makes it simpler.
That's its thing, to make the administration stuff simpler to you, so it saves you time. But I'm not sure, as I'm just starting with it!
A nice introduction of how it works in the official documentation is:
https://devcenter.heroku.com/articles/how-heroku-works
Per DZone: https://dzone.com/articles/heroku-or-amazon-web-services-which-is-best-for-your-startup
Heroku is a Platform as a Service (PaaS) product based on AWS, and is vastly different from Elastic Compute Cloud. It’s very important to differentiate ‘Infrastructure as a Service’ and ‘Platform as a Service’ solutions as we consider deploying and supporting our application using these two solutions.
Heroku is way simpler to use than AWS Elastic Compute Cloud. Perhaps it’s even too simple. But there’s a good reason for this simplicity. The Heroku platform equips us with a ready runtime environment and application servers. Plus, we benefit from seamless integration with various development instruments, a pre-installed operating system, and redundant servers.
Therefore, with Heroku, we don’t need to think about infrastructure management, unlike with AWS EC2. We only need to choose a subscription plan and change our plan when necessary.
That article does a good job explaining the differences between Heroku and AWS but it looks like you can choose other iaas (infrastructure) providers other than AWS. So ultimately Heroku seems to just simplify the process of using a cloud provider but at a cost.