How to handle log drains in one-off dynos? - heroku

We're aggregating our logs to papertrail using heroku's log drains. Everything works great, except, I'm not sure how to set up logging from one-off dynos, which we use to run scripts.
I thought the drain configuration would apply to one-off dynos, but I'm not seeing the output I'd expect from jobs we run using the heroku scheduler. In an attempt to figure out what's going on, I tried running
# heroku run bash --app myapp
# babel-node
> var logger = require('bunyan/my_configured_logger');
> logger.info('YO');
I'd expect this to result in logs being shipped to papertrail, but no dice. So then, I tried the simpler command line
> logger "YO"
and this didn't work either. So, either my tests are misguided, or drain configuration doesn't apply to one-off dynos. I think the former is more likely.
How can I test log drains (configured for a remote papertrail syslog) are working correctly?

Try
heroku run:detached --app myapp babel-node -- -e 'var logger = require("bunyan/my_configured_logger"); logger.info("YO");'
The key here is to run the dyno in detached mode, so that stdout and stderr go to the Heroku log drain instead of the console. That means you can't run bash interactively, so you have to pass the JavaScript to evaluate on the node command line.

Related

How to check output in terminal in production

i have a project serving on production which use pm2 to handle my nodejs server. How could i to check the bash output? Because i need to debug what's wrong in production, and it work in local machine. Which server is serve from digitalOcean.
Appreciate for your command.
You can view the stdout or stderr (bash output) of a pm2-run project with pm2 logs
Read the documentation for more details
Use can use logger to records all the changes along with various methods such as warning, error or info

Prevent Heroku from starting a web dyno

I'd like to configure a Heroku app to run a scheduled task once per day. My source tree looks like this:
bin/myScript
Procfile
package.json
When I deploy the app, I see the following error:
2017-01-11T04:31:36.660973+00:00 app[web.1]: npm ERR! missing script: start
I believe this is because Heroku tries to spin up a web dyno. I don't have a web dyno, nor do I want one. So I created a Procfile with this line:
heroku ps:scale web=0
To prevent heroku from spinning up a web dyno. That didn't work. What else can I do to prevent my app from crashing upon deployment? Does it matter if the scheduled task is going to be run in a separate one-off Dyno anyway?
You should not have the line "heroku ps:scale web=0" in your Procfile.
Doing so tells heroku to create a process type called "heroku" that attempts to run the following command on any dyno instances instantiated for it: "ps:scale web=0". That would probably generate errors, and at any rate, is not what you intended.
Instead You should run "heroku ps:scale web=0" as a Heroku toolbelt CLI command (or do the equivalent from the Resources tab of the GUI, as you already did).
I think I found a fix: in the "Resources" tab of the GUI for the web, there is a list of dynos with on/off sliders next to them. I switched the web dyno slider to off, and now when I deploy there is no crash. Still, it's unclear to me why the Procfile line was insufficient.

What is a good way to run a task whenever a Heroku app restarts?

Use case is to bust the cache.
What is a good way to run given code (or rake task) whenever a Ruby Heroku app is restarted (or deployed)?
There's no way to do this via the Heroku API far as I know. The Heroku Platform API doesn't support this.
What you can do (if you're fast, however!) is listen for a SIGTERM message in your code (that's what Heroku sends to your application process when it attempts to restart it) -- you can then fire off your script quickly.
Here's more information on SIGTERM on Heroku: https://devcenter.heroku.com/articles/dynos#graceful-shutdown-with-sigterm
If you're using some sort of CI, you can probably configure it there. Heres how to do it with CircleCI:
deployment:
production:
branch: production
commands:
- git push git#heroku.com:foo-bar-123.git $CIRCLE_SHA1:master
- heroku run rake <your task> --app <your app name>
If you're not using a CI you can still whip together a script that first does the git push to Heroku and then executes your cache busting task through heroku run (the app's bin/ folder would be an obvious place to put it).
Note: you can also use heroku run:detached, which will send output to your logs instead of stdout.
You can use "release" feature that allows you to run any command before a new release is deployed. https://devcenter.heroku.com/articles/release-phase
Define the command that should be run in your Procfile.
release: rake db:migrate
From documentation:
The release command is run immediately after a release is created, but before the release is deployed to the app’s dyno formation. That means it will be run after an event that creates a new release.

How to reach a Heroku deployment within Heroku bash or Heroku Scheduler

I'm trying to invoke a HTTP endpoint in my deployed Heroku app with Heroku Scheduler, which basically runs commands on Heroku bash at a fixed rate.
When I run $ heroku run bash, I can read my port with echo $PORT (let's say 5555). However when I try to access my webpage using curl http://localhost:5555, it does not work.
What is the way to reach to the app within Heroku bash without using the app name?
When you run "heroku run bash" you are actually firing up a one-off dyno which - according to Heroku - is a different process (from the dyno running your "web" process type) running on a different runtime instance. Routing does not exist between processes, so the HTTP requests you're issuing will always fail.
It's seems that there is no pre-configured environment variable.
However you can set it:
heroku config:add HEROKU_URL=http://<your app>.herokuapp.com

heroku - how to see all the logs

I have a small app on heroku. Whenever I want to see the logs I go to the command line and do
heroku logs
That only shows me about 100 lines. Is there not a way to see complete logs for our application on heroku?
Update (thanks to dawmail333):
heroku logs -n 1500
or, to tail the logs live
heroku logs -t
Heroku log documentation
If you need more than a few thousand lines you can Use heroku's Syslog Drains
Alternatively (old method):
$ heroku run rails c
File.open('log/production.log', 'r').each_line { |line| puts line }
Logging has greatly improved in heroku!
$ heroku logs -n 500
Better!
$ heroku logs --tail
references: http://devcenter.heroku.com/articles/logging
UPDATED
These are no longer add-ons, but part of the default functionality :)
Heroku treats logs as time-ordered streams of events. Accessing *.log files on the filesystem is not recommended in such an environment for a variety of reasons.
First, if your app has more than one dyno then each log file only represents a partial view into the events of your app. You would have to manually aggregate all the files to get the full view.
Second, the filesystem on Heroku is ephemeral meaning whenever your dyno is restarted or moved (which happens about once a day)the log files are lost. So you only get at most a day's view into that single dyno's logs.
Finally, on the Cedar stack running heroku console or even heroku run bash does not connect you to a currently running dyno. It spawns a new one specifically for the bash command. This is called a one-off process. As such, you won't find the log files for your other dynos that are running the actual http processes on the one spawned for heroku run.
Logging, and visibility in general, is a first-class citizen on Heroku and there are several tools that address these issues. First, to see a real-time stream of application events across all dynos and all layers of the application/stack use the heroku logs -t command to tail output to your terminal.
$ heroku logs -t
2010-09-16T15:13:46-07:00 app[web.1]: Processing PostController#list (for 208.39.138.12 at 2010-09-16 15:13:46) [GET]
2010-09-16T15:13:46-07:00 app[web.1]: Rendering template within layouts/application
2010-09-16T15:13:46-07:00 heroku[router]: GET myapp.heroku.com/posts queue=0 wait=0ms service=1ms bytes=975
2010-09-16T15:13:47-07:00 app[worker.1]: 2 jobs processed at 16.6761 j/s, 0 failed ...
This works great for observing the behavior of your application right now. If you want to store the logs for longer periods of time you can use one of the many logging add-ons that provide log retention, alerting and triggers.
Lastly, if you want to store the log files yourself you can setup your own syslog drain to receive the stream of events from Heroku and post-process/analyze yourself.
Summary: Don't use heroku console or heroku run bash to view static log files. Pipe into Heroku's stream of log events for your app using heroku logs or a logging add-on.
Well, the above answers are very helpful it will help you to view from the command line. Whereas if you want to check from GUI so you have to log into your Heroku account and then select your application and finally click on view logs
Follow on Heroku logging
To view your logs we have:
logs command retrives 100 log lines by default.
heroku logs
show maximum 200 lines, --num (or -n) option.
heroku logs -n 200
Show logs in real time
heroku logs --tail
If you have many apps on heroku
heroku logs --app your_app_name
Also see individual streams/filters.
E.g tail only your application logs
heroku logs --source app -t
Or see only the router logs
heroku logs --ps router
Or chain them together
heroku logs --source app --ps worker
So good..
You can access your log files using Heroku's Command Line
Interface (CLI Usage).
If Heroku's CLI is installed and you know your application name (like https://myapp.herokuapp.com/), then you can run the following command:
heroku logs --tail --app=myapp
You can also access the logs in a real-time stream using:
heroku logs --source app --tail --app=myapp
If the logs tell you something like this:
npm ERR! A complete log of this run can be found in:
npm ERR! /app/.npm/_logs/2017-07-11T08_29_45_291Z-debug.log
Then you can also access them using the bash terminal via Heroku CLI:
heroku run bash --app=myapp
less ./.npm/_logs/2017-07-11T08_29_45_291Z-debug.log
heroku logs -t shows us the live logs.
Might be worth it to add something like the free Papertrail plan to your app. Zero configuration, and you get 7 days worth of logging data up to 10MB/day, and can search back through 2 days of logs.
My solution is to get complete log the first time the application start, like:
heroku logs -n 1500 > log
then add fgrep -vf to keep it up to date, like:
heroku logs -n 1500 > newlog ; fgrep -vf log newlog >> log
for continuous logging, just iterate it using watch for every x minutes (or seconds).
You need to use -t or --tail option and you need to define your heroku app name.
heroku logs -t --app app_name
for WAR files:
I did not use github, instead I uploaded directly, a WAR file ( which I found to be much easier and faster ).
So the following helped me:
heroku logs --app appname
Hope it will help someone.
You need to have some logs draining implemented and should be draining your logs there, to see all of the logs (manage historical logs as well):
First option - Splunk can be used: you can drain all your logs like:
heroku drains:add syslog+tls://splunk-server.com:514 -a app_name
And then login into your splunk server and search for any number of logs. I am using Splunk and this is working perfectly fine for me.
Second option - You can purchase add on to your App, like given below: (I haven't used these options, however these are the available ones).
Timber.io
Sumo Logic
LogEnteries
Log DNA
Papertrail
You can also have a look at below options: If you want to have your
logs in JSON format, as it will help if your are pushing your logs to
external system like Splunk/ELK, it would become easy (performance
wise also) to search in JSON.
https://github.com/goodeggs/heroku-log-normalizer
It is not having Readme.md, but some explanation is given at https://github.com/goodeggs/bites/issues/20
Lastly
And you can always use below command as mentioned by other users already:
The following command will tail the generating logs on heroku
heroku logs -t -a <app_name>
The following comand will show the 1000 number of lines of logs from heroku
heroku logs -n 1000 -a <app_name>
Note only 1500 latest lines of logs are available and rest of them gets deleted from heroku dyno.
To see the detailed log you need to put two lines in the production.rb file:
config.logger = Logger.new(STDOUT)
config.logger.level = Logger::DEBUG
and then by running
heroku logs -t
you can see the detailed logs.
I prefer to do it this way
heroku logs --tail | tee -a herokuLogs
You can leave the script running in background and you can simply filter the logs from the text file the way you want anytime.
heroku logs -t shows us the live logs.
heroku logs -n 1500 for specific number of logs
But still I would recommend to use paper trail add-on which have certain benefits and has free basic plan.
I suggest using an addon, I use Logentries. To use it, run in your command line:
heroku addons:create logentries:le_tryit
(that command creates the addon for a free account but clearly you can upgrade if you want)
Logentries allows you to save up to 5GB of log volume per month. That info is searchable by their command search within the last 7 days and it has real-time alerts.
So to answer your question, by using this addon you ensure that your logs aren't lost anymore when you reach the 1500 lines that Heroku saves by default.
Hope this helps! Have a great day!
You can use
heroku logs -n 1500
But this is not a recommended approach(in other word doesn't show you the real picture)
I would suggest you plug some logging tool. ( sumoLogic, paper trail n all ) as an add-on
They all have a free version( with few limitations, though enough for a small app or dev env, which will provide good insight and tool to analyze logs )
I was running into a situation where in my apps' dashboard, when I went to:
More > View logs
I wouldn't get an output, just hung...
So I did a google and found this:
Heroku CLI plugin to list and create builds for Heroku apps.
I installed it and ran:
heroku builds -a example-app /* Lists 10 most recently created builds for example-app, that's where you get the id for the next step*/
Then enter:
heroku builds:output your-app-id-number -a example-app
And that's it, you should get back what you normally see in the dashboard GUI or locally.
Hope this helps someone like it did me!
If your code is in python,
You can use the heroku3 pip library to access the logs for your Heroku app.
First, you'll need to install the library by running.
pip install heroku3
Then, you can use the following code to access your logs:
import heroku3
# Connect to the Heroku API
conn = heroku3.from_key('YOUR_API_KEY')
# Get the app you want to access logs for
app = conn.app('YOUR_APP_NAME')
# Access the logs
logs = app.logs().stream()
# Print the logs
for line in logs:
print(line)
Make sure to replace YOUR_API_KEY and YOUR_APP_NAME with the appropriate values for your app.
Additionally, you can use the heroku logs --tail
command in your command line to stream the logs for your Heroku app. Make sure you are logged in and you have the access to the app.
For cedar stack see:
https://devcenter.heroku.com/articles/oneoff-admin-ps
you need to run:
heroku run bash ...

Resources