Heroku H15 errors with Elixir/Phoenix Channels - heroku

Once I started using Channels with Phoenix, I started getting a lot of these errors on Heroku:
heroku[router]: at=error code=H15 desc="Idle connection" method=GET
path="/socket/websocket?guardian_token=eyJhbGciOiJIUz[...]&vsn=1.0.0" host=[...]
dyno=web.1 connect=0ms service=87803ms status=503 bytes=
A reliable way to reproduce this is on a laptop:
Open a Phoenix page that connects to WebSockets
Close the lid of your laptop
Wait a minute or so (WiFi to disconnect/OS goes to sleep)
The H15 error will occur.
I've set a 45s timeout on my UserChannel already; I'm running Elixir 1.4.2, Phoenix 1.2.1 and my phoenix.js was just upgraded to the one used in Phoenix 1.3.0-rc0 (link to rev)
I can see the error logged on the front-end, but after that it reconnects happily to the backend:
2017-03-04 23:28:55.351 receive: ok user:2 phx_reply (3)
2017-03-04 23:29:24.269 channel: error user:2
2017-03-04 23:29:28.713 push: user:2 phx_join (4)
2017-03-04 23:29:28.745 receive: ok user:2 phx_reply (4)
Any help or ideas how to solve this is much appreciated.
Edit: I'm familiar with Heroku's H15 error code definition so I'm looking for a fix or a workaround for getting rid of the errors (other than migrating away from Heroku, of course).

Phoenix docs recommend setting the timeout for the websocket transport to < 55 seconds:
defmodule HelloWeb.UserSocket do
use Phoenix.Socket
...
## Transports
transport :websocket, Phoenix.Transports.WebSocket,
timeout: 45_000
...
end
Their explanation is:
This ensures that any idle connections are closed by Phoenix before they reach Heroku’s 55-second timeout window.

Related

Heroku app crashing (H10 error) with psiturk experiment

I'm trying to spin up an experiment on a Heroku app with psiturk. I have previously gotten this to work (~2019), but recently (since 2021) I have not been able to run psiturk on Heroku without the app crashing, even using exactly the same code. 
Here are some of the steps I have taken so far:
The experiment runs locally, and I have confirmed that I can connect to my postgres database from the Heroku app.
I have also confirmed the Heroku is dynamically binding the port, and not using the one set in config.txt.
In the Heroku bash environment, I am able to start psiturk and turn on the server without the app crashing, but the app will crash when loading a debug of sandbox HIT link.
The Heroku app uses python and node buildpacks, and the Procfile points to a python script. I am using psiturk 2.3.11 and python 3.8.8, and I have followed the steps in the psiturk documentation for setting config variables.
Here is the Heroku log:
2021-03-29T16:44:16.409840+00:00 app[api]: Starting process with command `bash` by user danibassettlab#gmail.com
2021-03-29T16:44:39.951733+00:00 heroku[run.7975]: State changed from starting to up
2021-03-29T16:44:40.507250+00:00 heroku[run.7975]: Awaiting client
2021-03-29T16:44:40.583037+00:00 heroku[run.7975]: Starting process with command `bash`
2021-03-29T17:04:33.678083+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/pub/ad?assignmentId=debug98GDW3&hitId=debug6JU43S&workerId=debugG38KJ9&mode=debug" host=jeni-docker-test.herokuapp.com request_id=2bdae5da-5f59-475e-8ca6-9ea9a380d5c0 fwd="70.16.141.206" dyno= connect= service= status=503 bytes= protocol=http
2021-03-29T17:04:33.870529+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/favicon.ico" host=jeni-docker-test.herokuapp.com request_id=828565a0-923b-4283-b9de-17b4cf890459 fwd="70.16.141.206" dyno= connect= service= status=503 bytes= protocol=http
And the config.txt I'm using (with account information removed)
[HIT Configuration]
title = Key Sequence Learning Study
description = Psychology Study in which you will learn to quickly press keys to respond to squares on the screen.
lifetime = 96
us_only = true
approve_requirement = 95
ad_group = default_network
psiturk_keywords = network
organization_name = University of Pennsylvania
browser_exclude_rule = mobile, tablet
allow_repeats = false
server_timeout = 240
[Database Parameters]
database_url = postgres:<my_database_url>
table_name = participants_tmp
[Server Parameters]
host = 0.0.0.0
port = 22362
cutoff_time = 30
logfile = server.log
loglevel = 2
debug = false
login_username = examplename
login_pw = examplepassword
threads = 1
secret_key = 'this is my secret key which is hard to guess, i should change this'
[Task Parameters]
experiment_code_version = 'behavioral_pilot_v6_delay'
num_conds = 1
num_counters = 1
[Shell Parameters]
launch_in_sandbox_mode = true
bonus_message = "Bonus for Bassett Lab Experiment. Thanks for participating!"
use_psiturk_ad_server = false
ad_location = https://jeni-docker-test.herokuapp.com/pub
The crash happens before anything is logged to the psiturk server.log file in the experiment folder.
Any advice on next steps?
I found the answer on the psiturk google group. Psiturk 2 needs the parameter ON_HEROKU to be set (I had set ON_CLOUD insteadt. Running
heroku config:set ON_HEROKU=1
fixed the issue.
Alternatively you could upgrade to psiturk 3
The newer version of SQLAlchemy removed the deprecated dialect name postgres. You have to use postgresql. Among your database parameters, you have:
database_url = postgres:<my_database_url>
To fix this, rename postgres:// in the URL to postgresql://.
Alternatively consider to add sslmode='require' to your database URL.

Heroku Apps forever crashed with streamlit

My first time deploying streamlit on Heroku, and the scripts are on this GitHub. I'm actually deploying from local, so this is just to share my scripts to debug. The following is the error message.
2020-12-31T23:47:18.055589+00:00 heroku[router]:
at=error code=H20 desc="App boot timeout"
method=GET path="/" host=stormy-dawn-49490.herokuapp.com
request_id=b3df4074-d363-41e5-8ecd-d101070679f7
fwd="99.230.168.115"
dyno= connect= service= status=503 bytes= protocol=https
2020-12-31T23:48:01.327576+00:00 heroku[web.1]: Error R10 (Boot timeout) ->
Web process failed to bind to $PORT within 60 seconds of launch
I have four scripts, which is also linked to my GitHub repository the beginning of the question.
setup.sh
mkdir -p ~/.streamlit/
echo “\
[general]\n\
email = \”<dont#spam.us>\”\n\
“ > ~/.streamlit/credentials.toml
echo “\
[server]\n\
headless = true\n\
enableCORS=false\n\
port = $PORT\n\
“ > ~/.streamlit/config.toml
Procfile:
web: sh setup.sh && streamlit run app.py
requirements.txt:
fredapi==0.4.2
numpy==1.18.5
pandas==1.1.5
plotly==4.14.1
python-dateutil==2.8.1
python-slugify==4.0.1
Quandl==3.5.3
scikit-learn==0.23.2
streamlit==0.72.0
yfinance==0.1.55
gunicorn== 19.9.0
app.py: I'd suggest read here
I didn't put much in the app.py as I'm just trying how to deploy on Heroku
Thank you in advance!

Phoenix deployment on Heroku failed to bind $PORT

I have big issues with deploying my application on heroku server. I've done it with help of tutorial from phoenix webpage. But it doesn't work. Below you can see the logs and configuration i used.
heroku logs
2016-02-07T04:28:55.623435+00:00 heroku[web.1]: Error R10 (Boot timeout) -> Web
process failed to bind to $PORT within 60 seconds of launch
2016-02-07T04:28:55.623435+00:00 heroku[web.1]: Stopping process with SIGKILL
2016-02-07T04:28:56.344649+00:00 heroku[web.1]: State changed from starting to c
rashed
2016-02-07T04:28:56.348324+00:00 heroku[web.1]: Process exited with status 137
2016-02-07T09:54:29.124036+00:00 heroku[web.1]: State changed from crashed to st
arting
2016-02-07T09:54:34.703208+00:00 heroku[web.1]: Starting process with command `m
ix run --no-halt`
2016-02-07T09:55:35.386922+00:00 heroku[web.1]: Error R10 (Boot timeout) -> Web
process failed to bind to $PORT within 60 seconds of launch
2016-02-07T09:55:35.386922+00:00 heroku[web.1]: Stopping process with SIGKILL
2016-02-07T09:55:36.330258+00:00 heroku[web.1]: State changed from starting to c
rashed
2016-02-07T09:55:36.323082+00:00 heroku[web.1]: Process exited with status 137
2016-02-07T12:14:35.182306+00:00 heroku[router]: at=error code=H10 desc="App cra
shed" method=GET path="/" host=kpsz.herokuapp.com request_id=7de0ee97-14cf-46f9-
a9a3-cbffa09ac379 fwd="83.7.11.212" dyno= connect= service= status=503 bytes=
2016-02-07T12:14:35.850018+00:00 heroku[router]: at=error code=H10 desc="App cra
shed" method=GET path="/favicon.ico" host=kpsz.herokuapp.com request_id=4e9b1979
-deb6-497b-80b2-655615f43d01 fwd="83.7.11.212" dyno= connect= service= status=50
3 bytes=
Configuration prod.exs
config :kpsz, Kpsz.Endpoint,
http: [port: System.get_env("PORT")],
url: [host: "example.com", port: 80],
cache_static_manifest: "priv/static/manifest.json",
secret_key_base: System.get_env("SECRET_KEY_BASE")
config :kpsz, Kpsz.Repo,
adapter: Ecto.Adapters.Postgres,
url: System.get_env("DATABASE_URL"),
pool_size: 20
I'm not sure if that's the problem but you could try to replace
http: [port: System.get_env("PORT")]
with
http: [port: {:system, "PORT"}]
At least that's what I'm using in my apps. You can also read the docs for more information http://www.phoenixframework.org/docs/heroku
I had a similar problem.
Adding a Procfile with a single line web: mix phoenix.server solved it.
heroku destroy project name
heroku create
and for some weird reason it works now...
Thanks very much for help.
I ran into this issue and, for me, the fix was as follow (copied from the answer on this question).
The fix is twofold:
to use the phoenix buildpack config and export the PORT as well, like so: config_vars_to_export=(DATABASE_URL PORT)
Avoid changing the MIX_ENV without recompiling the app, as that behavior is unsupported, as explained here.
As Recreation solves the problem, I suspect the problem is with loading environment variables.
Once I had a similar problem in my local. My observations
mix phoenix.server (you might see the message compiled so and so files)
Stopped the server
Export/Change the env variable in zshrc/bashrc file and source it
Start the server (you might not see that compilation message, as there are no changes in your codebase and so your new/edited env variables won't get reflected)
If you just add a space in that file will make it compile and take that env variable.
Reason for your problem might be: Setting env variables after the server is running/deployed, as env variables are converted to value during the compilation time itself.
I had a similar problem when deploying the phoenix umbrella project with docker.
I've taken these steps:
Heroku https://hexdocs.pm/phoenix/heroku.html#deploying-to-heroku-using-the-container-stack
Dockerfile https://hexdocs.pm/phoenix/releases.html#containers
I've also looked at other links, but these two was the best.
The only problem with that Dockerfile is that it needs $PORT (and $SECRET_KEY_BASE) on the build step which is not present. So we have to "stub" it somehow in prod.exs. BUT we have to provide real $PORT on a run step (runtime configuration). So you have to be sure that you renamed config/prod.secret.exs to config/releases.exs (the 1 step at https://hexdocs.pm/phoenix/releases.html#runtime-configuration) or just create config/releases.exs which would rewrite $PORT (and maybe some other application environment variables - for example, $SECRET_KEY_BASE).
UPD: as #Daniel mentions - releases is an Elixir 1.9 feature.

Running War file with Jetty Runner in Heroku

If someone could give me some help on heroku I would appreciate.
If I run this command on my local console, it works on -> localhost:8080
java -jar tools/jetty-runner-7.6.3.jar --port 8080 target/scala-2.9.1/myapp.war
So then I create the Procfile with this:
web: java -jar tools/jetty-runner-7.6.3.jar --port 8080 target/scala-2.9.1/myapp.war
Then I test by running:
foreman start
and it also works in -> localhost:8080
But after I deployed the app and make heroku open, it does not work...
[success] Total time: 1 s, completed Jun 22, 2012 11:16:59 AM
-----> Discovering process types
Procfile declares types -> web
-----> Compiled slug size is 70.0MB
-----> Launching... done, v5
http://myapp.herokuapp.com deployed to Heroku
To git#heroku.com:myapp.git
XXXXX..XXXXX master -> master
$heroku logs
2012-06-22T11:14:54+00:00 heroku[slugc]: Slug compilation started
2012-06-22T11:17:57+00:00 heroku[api]: Release v5 created by my_email#my_email.com
2012-06-22T11:17:57+00:00 heroku[api]: Deploy dcdd123 by my_email#my_email.com
2012-06-22T11:18:00+00:00 heroku[slugc]: Slug compilation finished
2012-06-22T11:18:21+00:00 heroku[router]: Error H14 (No web processes running) -> GET myapp.herokuapp.com/ dyno= queue= wait= service= status=503 bytes=
2012-06-22T11:18:30+00:00 heroku[router]: Error H14 (No web processes running) -> GET myapp.herokuapp.com/ dyno= queue= wait= service= status=503 bytes=
Any ideas, suggestions? Thanks,
Regards,
Andre
Andre - We have a Heroku Labs feature that allows you to deploy WAR files directly to Heroku. Check out https://gist.github.com/a8c62301591a11b4c6e9 for exact details on how to go about doing it.
If you are using jetty runner to run your war you do not need to hard code the port number.
Heroku sets the port internally.
So your Procfile should be like:
java -jar tools/jetty-runner-7.6.3.jar --port $PORT target/scala-2.9.1/myapp.war

omniauth and google with sinatra on heroku

I'm trying to use OmniAuth in a sinatra app with google as the login provider. Using the twitter and facebook providers work fine:
require 'omniauth'
require 'openid'
require 'openid/store/filesystem'
use OmniAuth::Builder do
use OmniAuth::Strategies::OpenID, OpenID::Store::Filesystem.new('./tmp'), :name => 'google', :identifier => 'https://www.google.com/accounts/o8/id'
end
get '/sign_in'
redirect '/auth/google'
end
Thoughts? When the browser tries to get '/auth/google', it gets a 503 - Service Unavailable. There is nothing in the heroku logs
Thoughts or suggestions on what this might be?
I just started a bounty on this question - what I'm hoping for is a sample implementation that works on heroku.
I increased the log level on heroku. Here is what I'm getting now:
2011-07-07T16:15:09+00:00 heroku[nginx]: GET /sign_in/?p=google HTTP/1.1 | 207.224.213.179 | 269 | http | 302
2011-07-07T16:15:09+00:00 app[web.1]: I, [2011-07-07T09:15:09.863397 #1] INFO -- OpenID: Error attempting to use stored discovery information: OpenID::TypeURIMismatch
2011-07-07T16:15:09+00:00 app[web.1]: I, [2011-07-07T09:15:09.863480 #1] INFO -- OpenID: Attempting discovery to verify endpoint
2011-07-07T16:15:09+00:00 app[web.1]: I, [2011-07-07T09:15:09.863512 #1] INFO -- OpenID: Performing discovery on https://www.google.com/accounts/o8/id?id=AItOawlAKE9QAQb9mMRyYCCmAkEYIcqfmhrA080
2011-07-07T16:15:09+00:00 app[web.1]: I, [2011-07-07T09:15:09.865033 #1] INFO -- OpenID: WARNING: making https request to https://www.google.com/accounts/o8/id?id=AItOawlAKE9QAQb9mMRyYCCmAkEYIcqfmhrA080 without verifying server certificate; no CA path was specified.
2011-07-07T16:15:09+00:00 app[web.1]: 207.224.213.179 - - [07/Jul/2011 09:15:09] "POST /auth/google/callback HTTP/1.1" 302 - 0.0956
2011-07-07T16:15:09+00:00 heroku[router]: GET myurl.com/auth/google/callback dyno=web.1 queue=0 wait=0ms service=100ms status=302 bytes=0
The page at https://github.com/intridea/omniauth/wiki/OpenID-and-Google-Apps gives two possible solutions:
On Heroku you can change the path to "./tmp" or use Memcached.
It seems like you already tried the Memcached solution - do you have a Memcached instance running and properly setup on Heroku?
You could use google's openid to sign in.
use OmniAuth::Builder do
provider :openid, OpenID::Store::Filesystem.new('/tmp')
use OmniAuth::Strategies::OpenID, OpenID::Store::Filesystem.new('/tmp'), :name => 'google', :identifier => 'https://www.google.com/accounts/o8/id'
end

Resources