This question has been asked a lot, but still, nothing seems to be working. My application runs perfectly in testing, but when run on heroku, I get the error 500 saying "we're sorry , but something went wrong." I've checked the log files for the server, and my error seems to be here somewhere:
2012-10-21T15:29:40+00:00 app[web.1]:
2012-10-21T15:29:40+00:00 app[web.1]: Started GET "/" for 117.192.23.223 at 2012-10-21 15:29:40 +0000
2012-10-21T15:29:40+00:00 app[web.1]: cache: [GET /] miss
2012-10-21T15:29:40+00:00 app[web.1]:
2012-10-21T15:29:40+00:00 heroku[router]: GET whispering-coast-7415.herokuapp.com/ dyno=web.1 queue=0 wait=0ms service=5ms status=301 bytes=115
2012-10-21T15:29:40+00:00 app[web.1]:
2012-10-21T15:29:40+00:00 app[web.1]: Started GET "/movies" for 117.192.23.223 at 2012-10-21 15:29:40 +0000
2012-10-21T15:29:40+00:00 app[web.1]:
2012-10-21T15:29:40+00:00 app[web.1]: Processing by MoviesController#index as HTML
2012-10-21T15:29:40+00:00 app[web.1]: Completed 500 Internal Server Error in 2ms
2012-10-21T15:29:40+00:00 heroku[router]: GET whispering-coast-7415.herokuapp.com/movies dyno=web.1 queue=0 wait=0ms service=17ms status=500 bytes=728
2012-10-21T15:29:40+00:00 app[web.1]: ActiveRecord::StatementInvalid (PG::Error: ERROR: column "movies.id" must appear in the GROUP BY clause or be used in an aggregate function
2012-10-21T15:29:40+00:00 app[web.1]: LINE 1: SELECT "movies".* FROM "movies" GROUP BY rating
2012-10-21T15:29:40+00:00 app[web.1]:
2012-10-21T15:29:40+00:00 app[web.1]: ^
2012-10-21T15:29:40+00:00 app[web.1]:
2012-10-21T15:29:40+00:00 app[web.1]: cache: [GET /movies] miss
2012-10-21T15:29:40+00:00 app[web.1]: : SELECT "movies".* FROM "mo
vies" GROUP BY rating):
2012-10-21T15:29:40+00:00 app[web.1]: app/controllers/movies_controller.rb:11:in `index'
2012-10-21T15:29:40+00:00 app[web.1]: app/models/movie.rb:4:in `get_ratings'
2012-10-21T15:29:40+00:00 app[web.1]:
2012-10-21T15:29:41+00:00 heroku[router]: GET whispering-coast-7415.herokuapp.com/favicon.ico dyno=web.1 queue=0 wait=0ms service=8ms status=200 bytes=0
2012-10-21T15:29:41+00:00 app[web.1]: cache: [GET /favicon.ico] stale, invalid, store
PostgreSQL (on Heroku) requires you to GROUP BY all the columns in the SELECT list so in this case instead of:
SELECT "movies".* FROM "movies" GROUP BY rating
you might want to do something like:
SELECT rating FROM "movies" GROUP BY rating
or in Rails:
Movie.select(:rating).group(:rating)
Related
I have a rails app that is working fine on Heroku, but all the POST requests coming from Rails admin are hanging for 55 seconds before causing a H15 "Idle Connection" error.
I've read that it can come from:
ActionCable not pinging the server during 55 consecutive seconds but I'm not using real time features with Rails Admin and I removed my ActionCable code to test and the problem still occurs
The server tries to return a response to a different IP than the one that initiated the call once you've configured your DNS, but I'm not, I'm not using a custom domain name, I use the my website-herokuapp.com one.
Here's the log of a POST request from my app from the admin:
2021-01-24T19:09:36.245896+00:00 app[web.1]: I, [2021-01-24T19:09:36.245747 #4] INFO -- : source=rack-timeout id=21d5a60a-a82a-4cf7-bbe1-b0254f5823c7 wait=11ms timeout=15000ms state=ready
2021-01-24T19:09:36.246186+00:00 app[web.1]: D, [2021-01-24T19:09:36.246074 #4] DEBUG -- : source=rack-timeout id=21d5a60a-a82a-4cf7-bbe1-b0254f5823c7 wait=11ms timeout=15000ms service=0ms state=active
2021-01-24T19:09:36.246456+00:00 app[web.1]: I, [2021-01-24T19:09:36.246394 #4] INFO -- : [21d5a60a-a82a-4cf7-bbe1-b0254f5823c7] Started PUT "/admin/first_job/252990/edit" for 92.183.112.168 at 2021-01-24 19:09:36 +0000
2021-01-24T19:09:36.248487+00:00 app[web.1]: I, [2021-01-24T19:09:36.248393 #4] INFO -- : [21d5a60a-a82a-4cf7-bbe1-b0254f5823c7] Processing by RailsAdmin::MainController#edit as HTML
2021-01-24T19:09:36.248636+00:00 app[web.1]: I, [2021-01-24T19:09:36.248524 #4] INFO -- : [21d5a60a-a82a-4cf7-bbe1-b0254f5823c7] Parameters: {"utf8"=>"✓", "authenticity_token"=>"9QGOhotnG3BkLq6TbkXuXZgMVxJ+AinueV0oEUaVw5iaQ6DRJukISaCV4x+lcpXog9ExNYf9DB8/u/jufN6+Dg==", "first_job"=>{"company_name"=>"My company"}, "return_to"=>"https://mywebsite-staging.herokuapp.com/admin/first_job", "_save"=>"", "model_name"=>"first_job", "id"=>"252990"}
2021-01-24T19:09:36.285663+00:00 app[web.1]: D, [2021-01-24T19:09:36.285530 #4] DEBUG -- : [21d5a60a-a82a-4cf7-bbe1-b0254f5823c7] User Load (1.5ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 ORDER BY "users"."id" ASC LIMIT $2 [["id", 307509], ["LIMIT", 1]]
2021-01-24T19:09:36.307192+00:00 app[web.1]: D, [2021-01-24T19:09:36.307039 #4] DEBUG -- : [21d5a60a-a82a-4cf7-bbe1-b0254f5823c7] FirstJob Load (2.0ms) SELECT "first_jobs".* FROM "first_jobs" WHERE "first_jobs"."id" = $1 ORDER BY "first_jobs"."id" ASC LIMIT $2 [["id", 252990], ["LIMIT", 1]]
2021-01-24T19:09:36.312261+00:00 app[web.1]: D, [2021-01-24T19:09:36.312087 #4] DEBUG -- : [21d5a60a-a82a-4cf7-bbe1-b0254f5823c7] (1.0ms) BEGIN
2021-01-24T19:09:36.317945+00:00 app[web.1]: D, [2021-01-24T19:09:36.317805 #4] DEBUG -- : [21d5a60a-a82a-4cf7-bbe1-b0254f5823c7] Company Load (1.8ms) SELECT "companies".* FROM "companies" WHERE "companies"."id" = $1 LIMIT $2 [["id", 26218], ["LIMIT", 1]]
2021-01-24T19:09:36.322239+00:00 app[web.1]: D, [2021-01-24T19:09:36.322101 #4] DEBUG -- : [21d5a60a-a82a-4cf7-bbe1-b0254f5823c7] (1.1ms) COMMIT
2021-01-24T19:09:36.323441+00:00 app[web.1]: I, [2021-01-24T19:09:36.323351 #4] INFO -- : [21d5a60a-a82a-4cf7-bbe1-b0254f5823c7] Redirected to https://mywebsite-staging.herokuapp.com/admin/first_job
2021-01-24T19:09:36.324756+00:00 app[web.1]: I, [2021-01-24T19:09:36.324674 #4] INFO -- : Completed 302 Found in 76ms (ActiveRecord: 19.6ms)
2021-01-24T19:09:36.324858+00:00 app[web.1]: I, [2021-01-24T19:09:36.324425 #4] INFO -- : source=rack-timeout id=21d5a60a-a82a-4cf7-bbe1-b0254f5823c7 wait=11ms timeout=15000ms service=79ms state=completed
2021-01-24T19:10:31.518929+00:00 heroku[router]: at=error code=H15 desc="Idle connection" method=POST path="/admin/first_job/252990/edit" host=mywebsite-staging.herokuapp.com request_id=21d5a60a-a82a-4cf7-bbe1-b0254f5823c7 fwd="92.183.112.168" dyno=web.1 connect=0ms service=55282ms status=503 bytes= protocol=https
Oh and of course, everything is working fine on my local.
Do someone has an idea?
I have an SQLAlchemy database running on a flask app that works fine locally, but as soon as I run it on Heroku I get a strange issue (See stack trace below).
I don't have a model called 'user' so I don't know what's going on. I've made sure that I've run db.create_all().
Creating a row in my table works fine, but it's only once I try selecting some rows do I get an issue (__init__.py):
#app.route('/blog')
def blog():
title = 'Blog'
name = get_name()
blog_text = []
for post in sorted(Post.query.filter_by(category=Post.Category.blog),
key=lambda p: p.posted_at, reverse=True):
content = post.content
blog_text.append(content)
content = '\n'.join(blog_text)
return render_template('blog.html', **locals())
And my 'Post' logic in my models.py:
class Post(db.Model):
__tablename__ = 'user'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(200), nullable=False)
posted_at = db.Column(db.DateTime, nullable=False, default=datetime.now())
content = db.Column(db.String(200), nullable=False)
content_type = db.Column(db.String(200), default="md", nullable=False) # md / html
category = db.Column(db.String(200), nullable=False, default="blog") # poetry / blog
class Type:
md = "md"
html = "html"
class Category:
poetry = "poetry"
blog = "blog"
...
Here's the error:
2019-07-12T21:59:09.120100+00:00 app[web.1]: 10.63.22.102 - - [12/Jul/2019:21:59:09 +0000] "GET /blog HTTP/1.1" 500 290 "https://secure-savannah-20745.herokuapp.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
2019-07-12T21:59:09.120443+00:00 heroku[router]: at=info method=GET path="/blog" host=secure-savannah-20745.herokuapp.com request_id=c32b2484-c8ff-4910-a4e2-5089eca7c6d3 fwd="98.0.144.26" dyno=web.1 connect=2ms service=66ms status=500 bytes=455 protocol=https
2019-07-12T21:59:13.632588+00:00 app[web.1]: 10.63.22.102 - - [12/Jul/2019:21:59:13 +0000] "GET /projects HTTP/1.1" 200 2342 "https://secure-savannah-20745.herokuapp.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
2019-07-12T21:59:13.837771+00:00 app[web.1]: 10.63.22.102 - - [12/Jul/2019:21:59:13 +0000] "GET /static/circles.js HTTP/1.1" 200 0 "https://secure-savannah-20745.herokuapp.com/projects" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
2019-07-12T21:59:13.838434+00:00 heroku[router]: at=info method=GET path="/static/circles.js" host=secure-savannah-20745.herokuapp.com request_id=8476c020-fc32-47b3-a4e1-7b6faeb115ca fwd="98.0.144.26" dyno=web.1 connect=1ms service=6ms status=200 bytes=5449 protocol=https
2019-07-12T21:59:13.633394+00:00 heroku[router]: at=info method=GET path="/projects" host=secure-savannah-20745.herokuapp.com request_id=faa3d473-7bda-4e39-810b-e3994eac7390 fwd="98.0.144.26" dyno=web.1 connect=1ms service=1494ms status=200 bytes=2504 protocol=https
2019-07-12T21:59:18.074035+00:00 heroku[router]: at=info method=GET path="/blog" host=secure-savannah-20745.herokuapp.com request_id=e9748e89-dcce-43db-8db8-a16c7fe657ab fwd="98.0.144.26" dyno=web.1 connect=3ms service=12ms status=500 bytes=455 protocol=https
2019-07-12T21:59:18.070372+00:00 app[web.1]: [2019-07-12 21:59:18,069] ERROR in app: Exception on /blog [GET]
2019-07-12T21:59:18.070385+00:00 app[web.1]: Traceback (most recent call last):
2019-07-12T21:59:18.070396+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
2019-07-12T21:59:18.070398+00:00 app[web.1]: cursor, statement, parameters, context
2019-07-12T21:59:18.070403+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 550, in do_execute
2019-07-12T21:59:18.070405+00:00 app[web.1]: cursor.execute(statement, parameters)
2019-07-12T21:59:18.070407+00:00 app[web.1]: sqlite3.OperationalError: no such table: user
2019-07-12T21:59:18.070410+00:00 app[web.1]:
2019-07-12T21:59:18.070412+00:00 app[web.1]: The above exception was the direct cause of the following exception:
2019-07-12T21:59:18.070414+00:00 app[web.1]:
2019-07-12T21:59:18.070416+00:00 app[web.1]: Traceback (most recent call last):
2019-07-12T21:59:18.070418+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app
2019-07-12T21:59:18.070421+00:00 app[web.1]: response = self.full_dispatch_request()
2019-07-12T21:59:18.070423+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request
2019-07-12T21:59:18.070425+00:00 app[web.1]: rv = self.handle_user_exception(e)
2019-07-12T21:59:18.070427+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception
2019-07-12T21:59:18.070430+00:00 app[web.1]: reraise(exc_type, exc_value, tb)
2019-07-12T21:59:18.070432+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
2019-07-12T21:59:18.070434+00:00 app[web.1]: raise value
2019-07-12T21:59:18.070436+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
2019-07-12T21:59:18.070438+00:00 app[web.1]: rv = self.dispatch_request()
2019-07-12T21:59:18.070446+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
2019-07-12T21:59:18.070450+00:00 app[web.1]: return self.view_functions[rule.endpoint](**req.view_args)
2019-07-12T21:59:18.070452+00:00 app[web.1]: File "/app/main.py", line 65, in blog
2019-07-12T21:59:18.070455+00:00 app[web.1]: key=lambda p: p.posted_at, reverse=True):
2019-07-12T21:59:18.070457+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3324, in __iter__
2019-07-12T21:59:18.070459+00:00 app[web.1]: return self._execute_and_instances(context)
2019-07-12T21:59:18.070461+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3349, in _execute_and_instances
2019-07-12T21:59:18.070463+00:00 app[web.1]: result = conn.execute(querycontext.statement, self._params)
2019-07-12T21:59:18.070466+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
2019-07-12T21:59:18.070468+00:00 app[web.1]: return meth(self, multiparams, params)
2019-07-12T21:59:18.070470+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection
2019-07-12T21:59:18.070472+00:00 app[web.1]: return connection._execute_clauseelement(self, multiparams, params)
2019-07-12T21:59:18.070474+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1107, in _execute_clauseelement
2019-07-12T21:59:18.070477+00:00 app[web.1]: distilled_params,
2019-07-12T21:59:18.070479+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
2019-07-12T21:59:18.070481+00:00 app[web.1]: e, statement, parameters, cursor, context
2019-07-12T21:59:18.070484+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
2019-07-12T21:59:18.070486+00:00 app[web.1]: util.raise_from_cause(sqlalchemy_exception, exc_info)
2019-07-12T21:59:18.070488+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 399, in raise_from_cause
2019-07-12T21:59:18.070491+00:00 app[web.1]: reraise(type(exception), exception, tb=exc_tb, cause=cause)
2019-07-12T21:59:18.070493+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 153, in reraise
2019-07-12T21:59:18.070495+00:00 app[web.1]: raise value.with_traceback(tb)
2019-07-12T21:59:18.070498+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
2019-07-12T21:59:18.070500+00:00 app[web.1]: cursor, statement, parameters, context
2019-07-12T21:59:18.070502+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 550, in do_execute
2019-07-12T21:59:18.070504+00:00 app[web.1]: cursor.execute(statement, parameters)
2019-07-12T21:59:18.070507+00:00 app[web.1]: sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: user
2019-07-12T21:59:18.070513+00:00 app[web.1]: [SQL: SELECT user.id AS user_id, user.name AS user_name, user.posted_at AS user_posted_at, user.content AS user_content, user.content_type AS user_content_type, user.category AS user_category
2019-07-12T21:59:18.070515+00:00 app[web.1]: FROM user
2019-07-12T21:59:18.070517+00:00 app[web.1]: WHERE user.category = ?]
2019-07-12T21:59:18.070520+00:00 app[web.1]: [parameters: ('blog',)]
2019-07-12T21:59:18.070661+00:00 app[web.1]: (Background on this error at: http://sqlalche.me/e/e3q8)
2019-07-12T21:59:18.072919+00:00 app[web.1]: 10.63.22.102 - - [12/Jul/2019:21:59:18 +0000] "GET /blog HTTP/1.1" 500 290 "https://secure-savannah-20745.herokuapp.com/projects" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
You missed that your table is named user in your model:
class Post(db.Model):
__tablename__ = 'user'
# ^^^^^^
The reason it doesn't exist is because Heroku doesn't keep your filesystem changes. You are using a SQLite database:
... app[web.1]: sqlite3.OperationalError: no such table: user
# ^^^^^^^
but a sqlite database is stored on the filesystem of your Heroku dyno. Heroku cleans up (cycles) dynos all the time and spins up a new dyno for you for new requests. From their documentation on how Heroku works:
Changes to the filesystem on one dyno are not propagated to other dynos and are not persisted across deploys and dyno restarts. A better and more scalable approach is to use a shared resource such as a database or queue.
Each time a new dyno is spun up to handle your requests, you'll have a new, clean filesystem without your sqlite database, and so an empty database is created again without any tables in it.
Bottom line: You can't use sqlite on Heroku.
Use a Postgres database instead, see the Heroku Postgres documentation. Don't worry, there is a free tier you can use. When you provision the database as an add-on, the DATABASE_URL config var is set for you, which you can read from your app environment.
I usually configure Flask apps that are designed to run on Heroku to read the environment variable but to fall back to a SQLite database when the environment variable is not set:
import os
from pathlib import Path
_project_root = Path(__file__).resolve().parent.parent
_default_sqlite_db = _project_root / "database.db"
SQLALCHEMY_DATABASE_URI = os.environ.get(
"DATABASE_URL", f"sqlite:///{_default_sqlite_db}"
)
This makes it easy to develop the app locally on SQLite, or set up a local Postgres database for integration testing, and then deploy to Heroku for staging or production.
My server on Heroku [seemingly] randomly returns H18 errors.
I use:
Gunicorn (I tried both sync and gevent workers)
Flask
Flask-Restful
If I repeat the same request several times, I occasionally get an H18. I noticed that the most often I get such a response when an exception is raised in my code. I have a handler defined that returns a valid HTTP error response and most of the time it succeeds. Sometimes it fails though - for the reason that I cannot understand.
Here's how the "Bad situation" log looks like:
2018-03-10T17:50:51.875115+00:00 app[web.1]: [2018-03-10 17:50:51 +0000] [11] [DEBUG] PUT /api/v1/views/270
2018-03-10T17:50:51.879035+00:00 app[web.1]: PPMException: Token expired
2018-03-10T17:50:51.879442+00:00 app[web.1]: Traceback (most recent call last):
2018-03-10T17:50:51.879445+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
2018-03-10T17:50:51.879447+00:00 app[web.1]: rv = self.dispatch_request()
2018-03-10T17:50:51.879449+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
2018-03-10T17:50:51.879451+00:00 app[web.1]: return self.view_functions[rule.endpoint](**req.view_args)
2018-03-10T17:50:51.879452+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask_restful/__init__.py", line 480, in wrapper
2018-03-10T17:50:51.879456+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask/views.py", line 84, in view
2018-03-10T17:50:51.879454+00:00 app[web.1]: resp = resource(*args, **kwargs)
2018-03-10T17:50:51.879458+00:00 app[web.1]: return self.dispatch_request(*args, **kwargs)
2018-03-10T17:50:51.879464+00:00 app[web.1]: resp = meth(*args, **kwargs)
2018-03-10T17:50:51.879466+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask_httpauth.py", line 88, in decorated
2018-03-10T17:50:51.879463+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask_restful/__init__.py", line 595, in dispatch_request
2018-03-10T17:50:51.879469+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask_httpauth.py", line 125, in authenticate
2018-03-10T17:50:51.879468+00:00 app[web.1]: if not self.authenticate(auth, password):
2018-03-10T17:50:51.879471+00:00 app[web.1]: return self.verify_password_callback(username, client_password)
2018-03-10T17:50:51.879473+00:00 app[web.1]: File "/app/app/api_security.py", line 31, in verify_password
2018-03-10T17:50:51.879477+00:00 app[web.1]: PPMException: Token expired
2018-03-10T17:50:51.879475+00:00 app[web.1]: raise PPMException("SECURITY_TOKEN_EXPIRED", http_code=401)
2018-03-10T17:50:51.879479+00:00 app[web.1]:
2018-03-10T17:50:51.881229+00:00 app[web.1]: 10.189.169.232 - eyJleHAiOjE1MjA3MDQyNTAsImlhdCI6MTUyMDcwNDI0MCwiYWxnIjoiSFMyNTYifQ.eyJ1c2VyX2lkIjoiMyIsIm9yaWdpbmFsX2lzc3VlX2RhdGUiOiIyMDE4LTAzLTEwVDE3OjQ5OjI0Ljk3NTkwMCJ9.mB_NCtrGeDSq0ATs-Zl-Fg8qYRCx8YyIkjVdaLi_O6c
[10/Mar/2018:17:50:51 +0000] "PUT /api/v1/views/270 HTTP/1.1" 401 91 "http://localhost:3000/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
2018-03-10T17:50:51.881448+00:00 app[web.1]: [2018-03-10 17:50:51 +0000] [11] [DEBUG] Closing connection.
2018-03-10T17:50:51.975825+00:00 heroku[router]: sock=backend at=error code=H18 desc="Server Request Interrupted" method=PUT path="/api/v1/views/270" host=***-dev.herokuapp.com request_id=68b4ddb1-19b4-4753-9b3f-395c80fc777a fwd="81.164.187.197" dyno=web.1 connect=2ms service=106ms status=503 bytes=483 protocol=https
vs. how the same exception looks without H18:
2018-03-10T17:53:22.660154+00:00 app[web.1]: [2018-03-10 17:53:22 +0000] [10] [DEBUG] PUT /api/v1/views/270
2018-03-10T17:53:22.667461+00:00 app[web.1]: PPMException: Token expired
2018-03-10T17:53:22.669037+00:00 app[web.1]: Traceback (most recent call last):
2018-03-10T17:53:22.669041+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
2018-03-10T17:53:22.669042+00:00 app[web.1]: rv = self.dispatch_request()
2018-03-10T17:53:22.669044+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
2018-03-10T17:53:22.669046+00:00 app[web.1]: return self.view_functions[rule.endpoint](**req.view_args)
2018-03-10T17:53:22.669048+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask_restful/__init__.py", line 480, in wrapper
2018-03-10T17:53:22.669050+00:00 app[web.1]: resp = resource(*args, **kwargs)
2018-03-10T17:53:22.669055+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask/views.py", line 84, in view
2018-03-10T17:53:22.669057+00:00 app[web.1]: return self.dispatch_request(*args, **kwargs)
2018-03-10T17:53:22.669059+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask_restful/__init__.py", line 595, in dispatch_request
2018-03-10T17:53:22.669060+00:00 app[web.1]: resp = meth(*args, **kwargs)
2018-03-10T17:53:22.669062+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask_httpauth.py", line 88, in decorated
2018-03-10T17:53:22.669064+00:00 app[web.1]: if not self.authenticate(auth, password):
2018-03-10T17:53:22.669070+00:00 app[web.1]: File "/app/app/api_security.py", line 31, in verify_password
2018-03-10T17:53:22.669075+00:00 app[web.1]:
2018-03-10T17:53:22.669066+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask_httpauth.py", line 125, in authenticate
2018-03-10T17:53:22.669068+00:00 app[web.1]: return self.verify_password_callback(username, client_password)
2018-03-10T17:53:22.669071+00:00 app[web.1]: raise PPMException("SECURITY_TOKEN_EXPIRED", http_code=401)
2018-03-10T17:53:22.669073+00:00 app[web.1]: PPMException: Token expired
2018-03-10T17:53:22.677909+00:00 app[web.1]: 10.31.61.151 - eyJleHAiOjE1MjA3MDQyNTAsImlhdCI6MTUyMDcwNDI0MCwiYWxnIjoiSFMyNTYifQ.eyJ1c2VyX2lkIjoiMyIsIm9yaWdpbmFsX2lzc3VlX2RhdGUiOiIyMDE4LTAzLTEwVDE3OjQ5OjI0Ljk3NTkwMCJ9.mB_NCtrGeDSq0ATs-Zl-Fg8qYRCx8YyIkjVdaLi_O6c [10/Mar/2018:17:53:22 +0000] "PUT /api/v1/views/270 HTTP/1.1" 401 91 "http://localhost:3000/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
***2018-03-10T17:53:22.678478+00:00 heroku[router]: at=info method=PUT path="/api/v1/views/270" host=***-dev.herokuapp.com request_id=9e22f6a9-c980-478f-9ff1-bc8bbd4fe8c7 fwd="81.164.187.197" dyno=web.1 connect=0ms service=19ms status=401 bytes=483 protocol=https
2018-03-10T17:53:22.678989+00:00 app[web.1]: [2018-03-10 17:53:22 +0000] [10] [DEBUG] Closing connection.
The only difference I see is that in the first case I first see Gunicorn saying "Closing connection" and then the heroku router saying H18, while in the second case heroku router reports the response and then Gunicorn is closing the connection. If Gunicorn is closing connection prematurely, is there a way for me to control it from the application?
I think I got this figured out.
The reason seems to be that there were still request data left unread, so I added "request.stream.read()" in my error handling code.
So on my localhost, the custom devise forms work great! No issues whatsoever - when I log in from my custom form on a different page, it logs the user in and redirects to the homepage as expected.
However, when I deploy to Heroku, the app stops behaving - the custom form directs the user to the actual login page without logging them in.
This is the output on my Heroku Logs:
2012-10-18T09:08:08+00:00 app[web.1]: Started POST "/user/sign_in" for 116.15.222.245 at 2012-10-18 09:08:08 +0000
2012-10-18T09:08:08+00:00 app[web.1]: cache: [POST /user/sign_in] invalidate, pass
2012-10-18T09:08:08+00:00 app[web.1]: Processing by UserSessionsController#create as HTML
2012-10-18T09:08:08+00:00 app[web.1]: Parameters: {"utf8"=>"✓", "authenticity_token"=>"4WDTRkX842UVz7nFuGbWbA3z47+rraKXQgl3PRYuWJY=", "user"=>{"email"=>"spree#example.com", "password"=>"[FILTERED]", "remember_me"=>"0"}, "commit"=>"Log In"}
2012-10-18T09:08:08+00:00 app[web.1]: Redirected to https://example.com/user/sign_in
2012-10-18T09:08:08+00:00 app[web.1]: Completed 302 Found in 11ms
2012-10-18T09:08:08+00:00 heroku[router]: POST example.com/user/sign_in dyno=web.1 queue=0 wait=0ms service=16ms status=302 bytes=101
2012-10-18T09:08:08+00:00 heroku[nginx]: 108.162.225.233 - - [18/Oct/2012:09:08:08 +0000] "POST /user/sign_in HTTP/1.1" 302 101 "http://example.com/referral_login" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:15.0) Gecko/20100101 Firefox/15.0.1" example.com
2012-10-18T09:08:10+00:00 app[web.1]:
2012-10-18T09:08:10+00:00 app[web.1]: Started GET "/user/sign_in" for 116.15.222.245 at 2012-10-18 09:08:10 +0000
2012-10-18T09:08:10+00:00 app[web.1]:
2012-10-18T09:08:10+00:00 heroku[router]: GET example.com/user/sign_in dyno=web.1 queue=0 wait=0ms service=16ms status=200 bytes=7835
2012-10-18T09:08:10+00:00 app[web.1]: cache: [GET /user/sign_in] miss
2012-10-18T09:08:10+00:00 app[web.1]: Processing by UserSessionsController#new as HTML
2012-10-18T09:08:10+00:00 app[web.1]: Rendered shared/_error_messages.html.erb (0.1ms)
2012-10-18T09:08:10+00:00 app[web.1]: Rendered shared/_login.html.erb (2.3ms)
2012-10-18T09:08:10+00:00 app[web.1]: Rendered user_sessions/new.html.erb within layouts/spree_application (3.0ms)
2012-10-18T09:08:10+00:00 app[web.1]: Rendered shared/_head.html.erb (1.0ms)
2012-10-18T09:08:10+00:00 app[web.1]: Rendered shared/_store_menu.html.erb (0.7ms)
2012-10-18T09:08:10+00:00 app[web.1]: Rendered shared/_top_menu_bar.html.erb (0.5ms)
2012-10-18T09:08:10+00:00 heroku[nginx]: 108.162.225.233 - - [18/Oct/2012:09:08:10 +0000] "GET /user/sign_in HTTP/1.1" 200 3412 "http://example.com/referral_login" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:15.0) Gecko/20100101 Firefox/15.0.1" example.com
This is my output on my localhost:
Started POST "/user/sign_in" for 127.0.0.1 at 2012-10-18 17:21:36 +0800
Processing by UserSessionsController#create as HTML
Parameters: {"utf8"=>"✓", "authenticity_token"=>"m38Y47hk4UyGYAMnd76y5Qo2nK1Nv//v7s2+XdNHCbU=", "user"=>{"email"=>"spree#example.com", "password"=>"[FILTERED]", "remember_me"=>"0"}, "commit"=>"Log In"}
User Load (0.4ms) SELECT "users".* FROM "users" WHERE "users"."email" = 'spree#example.com' LIMIT 1
(23.6ms) SELECT COUNT(DISTINCT "users"."id") FROM "users" LEFT OUTER JOIN "roles_users" ON "roles_users"."user_id" = "users"."id" LEFT OUTER JOIN "roles" ON "roles"."id" = "roles_users"."role_id" WHERE "roles"."name" = 'admin'
(0.5ms) UPDATE "users" SET "last_sign_in_at" = '2012-10-18 05:56:54.897695', "current_sign_in_at" = '2012-10-18 09:21:39.033308', "sign_in_count" = 75, "updated_at" = '2012-10-18 09:21:39.081684' WHERE "users"."id" = 1
Redirected to http://localhost:3000/
Completed 302 Found in 227ms
Started GET "/" for 127.0.0.1 at 2012-10-18 17:21:39 +0800
Processing by LandingController#index as HTML
User Load (0.2ms) SELECT "users".* FROM "users" WHERE "users"."id" = 1 LIMIT 1
Rendered landing/index.html.erb within layouts/spree_application (1.7ms)
Rendered shared/_head.html.erb (2.0ms)
Rendered shared/_store_menu.html.erb (2.6ms)
Rendered shared/_top_menu_bar.html.erb (1.7ms)
Tracker Load (0.3ms) SELECT "trackers".* FROM "trackers" WHERE "trackers"."active" = 't' AND "trackers"."environment" = 'development' LIMIT 1
Rendered shared/_google_analytics.html.erb (14.1ms)
Completed 200 OK in 183ms (Views: 148.1ms | ActiveRecord: 1.9ms)
Can anyone help to shed some light as to why this might be and how to go about resolving this? I looked through Devise wiki, did a bunch of Google searches and tried a few things without success. Would appreciate any help!
Spree defaults to require SSL in production environments, so if your POST request is not https, it'll redirect to the login page.
We're facing an interesting problem on our staging server on Heroku. Each post/put/delete request freezes for about 10 secondes or more and some of them eventually time out resulting in a Heroku H-12 time out error.
Another interesting detail: the request seems to be processed after the time out because the database records get updated. We do not process a big data, we're just trying to update one field on one record in the DB.
Our local environment (localhost:3000) is working properly. We do not have any blocking action (sending email, processing big queries, ...)
We've tried activating/deactivating caching (dalli), asset compilation, we've switched the instance of our staging server. We've tried different settings in our staging.rb.
Any suggestions on how we could try to troubleshoot this?
2012-10-08T21:32:15+00:00 app[web.1]: Started PUT "/users/151" for 67.180.34.143 at 2012-10-08 21:32:15 +0000
2012-10-08T21:32:15+00:00 app[web.1]: Processing by UsersController#update as HTML
2012-10-08T21:32:15+00:00 app[web.1]: Parameters: {"utf8"=>"✓", "authenticity_token"=>"n4xK0Esi2zgBqSRtDTu3IEhnUAv0//bAkjLnKyk7xFk=", "user"=>{"role"=>"2", "designer_status"=>"1", "invitation_limit"=>"56"}, "commit"=>"Update", "id"=>"151"}
2012-10-08T21:32:45+00:00 heroku[router]: Error H12 (Request timeout) -> POST ziba-exygy-2.herokuapp.com/users/151 dyno=web.1 queue= wait= service=30000ms status=503 bytes=0
2012-10-08T21:32:46+00:00 app[web.1]: Redirected to https://ziba-exygy-2.herokuapp.com/users
2012-10-08T21:32:46+00:00 app[web.1]: Completed 302 Found in 30304ms (ActiveRecord: 53.7ms)
2012-10-08T21:32:46+00:00 app[web.1]: cache: [POST /users/151] invalidate, pass
2012-10-08T21:32:46+00:00 app[web.1]: User Load (3.7ms) SELECT `users`.* FROM `users` WHERE `users`.`id` = 42 LIMIT 1
2012-10-08T21:32:46+00:00 app[web.1]: User Load (12.3ms) SELECT `users`.* FROM `users` WHERE `users`.`id` = 151 LIMIT 1
2012-10-08T21:32:46+00:00 app[web.1]: (2.8ms) BEGIN
2012-10-08T21:32:46+00:00 app[web.1]: User Exists (3.2ms) SELECT 1 FROM `users` WHERE (`users`.`email` = BINARY 'pierre+tttttttttt#blablabla.com' AND `users`.`id` != 151) LIMIT 1
2012-10-08T21:32:46+00:00 app[web.1]: (3.1ms) COMMIT
2012-10-08T21:32:46+00:00 app[web.1]: (2.7ms) BEGIN
2012-10-08T21:32:46+00:00 app[web.1]: CACHE (0.0ms) SELECT 1 FROM `users` WHERE (`users`.`email` = BINARY 'pierre+tttttttttt# blablabla.com' AND `users`.`id` != 151) LIMIT 1
2012-10-08T21:32:46+00:00 app[web.1]: (11.6ms) UPDATE `users` SET `invitation_limit` = 56, `updated_at` = '2012-10-08 21:32:31' WHERE `users`.`id` = 151
2012-10-08T21:32:46+00:00 app[web.1]: (2.9ms) COMMIT
Turned out that our staging server didn't include all the Heroku add-ons the live server has.
One of the add-on missing was Flying Sphinx. Some of our models assume Sphinx was there and would try to use it causing that behavior.