My server on Heroku [seemingly] randomly returns H18 errors.
I use:
Gunicorn (I tried both sync and gevent workers)
Flask
Flask-Restful
If I repeat the same request several times, I occasionally get an H18. I noticed that the most often I get such a response when an exception is raised in my code. I have a handler defined that returns a valid HTTP error response and most of the time it succeeds. Sometimes it fails though - for the reason that I cannot understand.
Here's how the "Bad situation" log looks like:
2018-03-10T17:50:51.875115+00:00 app[web.1]: [2018-03-10 17:50:51 +0000] [11] [DEBUG] PUT /api/v1/views/270
2018-03-10T17:50:51.879035+00:00 app[web.1]: PPMException: Token expired
2018-03-10T17:50:51.879442+00:00 app[web.1]: Traceback (most recent call last):
2018-03-10T17:50:51.879445+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
2018-03-10T17:50:51.879447+00:00 app[web.1]: rv = self.dispatch_request()
2018-03-10T17:50:51.879449+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
2018-03-10T17:50:51.879451+00:00 app[web.1]: return self.view_functions[rule.endpoint](**req.view_args)
2018-03-10T17:50:51.879452+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask_restful/__init__.py", line 480, in wrapper
2018-03-10T17:50:51.879456+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask/views.py", line 84, in view
2018-03-10T17:50:51.879454+00:00 app[web.1]: resp = resource(*args, **kwargs)
2018-03-10T17:50:51.879458+00:00 app[web.1]: return self.dispatch_request(*args, **kwargs)
2018-03-10T17:50:51.879464+00:00 app[web.1]: resp = meth(*args, **kwargs)
2018-03-10T17:50:51.879466+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask_httpauth.py", line 88, in decorated
2018-03-10T17:50:51.879463+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask_restful/__init__.py", line 595, in dispatch_request
2018-03-10T17:50:51.879469+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask_httpauth.py", line 125, in authenticate
2018-03-10T17:50:51.879468+00:00 app[web.1]: if not self.authenticate(auth, password):
2018-03-10T17:50:51.879471+00:00 app[web.1]: return self.verify_password_callback(username, client_password)
2018-03-10T17:50:51.879473+00:00 app[web.1]: File "/app/app/api_security.py", line 31, in verify_password
2018-03-10T17:50:51.879477+00:00 app[web.1]: PPMException: Token expired
2018-03-10T17:50:51.879475+00:00 app[web.1]: raise PPMException("SECURITY_TOKEN_EXPIRED", http_code=401)
2018-03-10T17:50:51.879479+00:00 app[web.1]:
2018-03-10T17:50:51.881229+00:00 app[web.1]: 10.189.169.232 - eyJleHAiOjE1MjA3MDQyNTAsImlhdCI6MTUyMDcwNDI0MCwiYWxnIjoiSFMyNTYifQ.eyJ1c2VyX2lkIjoiMyIsIm9yaWdpbmFsX2lzc3VlX2RhdGUiOiIyMDE4LTAzLTEwVDE3OjQ5OjI0Ljk3NTkwMCJ9.mB_NCtrGeDSq0ATs-Zl-Fg8qYRCx8YyIkjVdaLi_O6c
[10/Mar/2018:17:50:51 +0000] "PUT /api/v1/views/270 HTTP/1.1" 401 91 "http://localhost:3000/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
2018-03-10T17:50:51.881448+00:00 app[web.1]: [2018-03-10 17:50:51 +0000] [11] [DEBUG] Closing connection.
2018-03-10T17:50:51.975825+00:00 heroku[router]: sock=backend at=error code=H18 desc="Server Request Interrupted" method=PUT path="/api/v1/views/270" host=***-dev.herokuapp.com request_id=68b4ddb1-19b4-4753-9b3f-395c80fc777a fwd="81.164.187.197" dyno=web.1 connect=2ms service=106ms status=503 bytes=483 protocol=https
vs. how the same exception looks without H18:
2018-03-10T17:53:22.660154+00:00 app[web.1]: [2018-03-10 17:53:22 +0000] [10] [DEBUG] PUT /api/v1/views/270
2018-03-10T17:53:22.667461+00:00 app[web.1]: PPMException: Token expired
2018-03-10T17:53:22.669037+00:00 app[web.1]: Traceback (most recent call last):
2018-03-10T17:53:22.669041+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
2018-03-10T17:53:22.669042+00:00 app[web.1]: rv = self.dispatch_request()
2018-03-10T17:53:22.669044+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
2018-03-10T17:53:22.669046+00:00 app[web.1]: return self.view_functions[rule.endpoint](**req.view_args)
2018-03-10T17:53:22.669048+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask_restful/__init__.py", line 480, in wrapper
2018-03-10T17:53:22.669050+00:00 app[web.1]: resp = resource(*args, **kwargs)
2018-03-10T17:53:22.669055+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask/views.py", line 84, in view
2018-03-10T17:53:22.669057+00:00 app[web.1]: return self.dispatch_request(*args, **kwargs)
2018-03-10T17:53:22.669059+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask_restful/__init__.py", line 595, in dispatch_request
2018-03-10T17:53:22.669060+00:00 app[web.1]: resp = meth(*args, **kwargs)
2018-03-10T17:53:22.669062+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask_httpauth.py", line 88, in decorated
2018-03-10T17:53:22.669064+00:00 app[web.1]: if not self.authenticate(auth, password):
2018-03-10T17:53:22.669070+00:00 app[web.1]: File "/app/app/api_security.py", line 31, in verify_password
2018-03-10T17:53:22.669075+00:00 app[web.1]:
2018-03-10T17:53:22.669066+00:00 app[web.1]: File "/app/.heroku/python/lib/python2.7/site-packages/flask_httpauth.py", line 125, in authenticate
2018-03-10T17:53:22.669068+00:00 app[web.1]: return self.verify_password_callback(username, client_password)
2018-03-10T17:53:22.669071+00:00 app[web.1]: raise PPMException("SECURITY_TOKEN_EXPIRED", http_code=401)
2018-03-10T17:53:22.669073+00:00 app[web.1]: PPMException: Token expired
2018-03-10T17:53:22.677909+00:00 app[web.1]: 10.31.61.151 - eyJleHAiOjE1MjA3MDQyNTAsImlhdCI6MTUyMDcwNDI0MCwiYWxnIjoiSFMyNTYifQ.eyJ1c2VyX2lkIjoiMyIsIm9yaWdpbmFsX2lzc3VlX2RhdGUiOiIyMDE4LTAzLTEwVDE3OjQ5OjI0Ljk3NTkwMCJ9.mB_NCtrGeDSq0ATs-Zl-Fg8qYRCx8YyIkjVdaLi_O6c [10/Mar/2018:17:53:22 +0000] "PUT /api/v1/views/270 HTTP/1.1" 401 91 "http://localhost:3000/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
***2018-03-10T17:53:22.678478+00:00 heroku[router]: at=info method=PUT path="/api/v1/views/270" host=***-dev.herokuapp.com request_id=9e22f6a9-c980-478f-9ff1-bc8bbd4fe8c7 fwd="81.164.187.197" dyno=web.1 connect=0ms service=19ms status=401 bytes=483 protocol=https
2018-03-10T17:53:22.678989+00:00 app[web.1]: [2018-03-10 17:53:22 +0000] [10] [DEBUG] Closing connection.
The only difference I see is that in the first case I first see Gunicorn saying "Closing connection" and then the heroku router saying H18, while in the second case heroku router reports the response and then Gunicorn is closing the connection. If Gunicorn is closing connection prematurely, is there a way for me to control it from the application?
I think I got this figured out.
The reason seems to be that there were still request data left unread, so I added "request.stream.read()" in my error handling code.
Related
I built an MLflow model and call a prediction on a streamlit dashboard, in it work fine in local.
In Heroku, the app which works fine locally failed to send the request online, what am I missing to such deployment?
Below the error code raised.
Procfile :
web: mlflow sagemaker deploy -m mlflow_model/
web: sh setup.sh && streamlit run app.py
Heroku logs:
2022-01-07T10:14:15.759252+00:00 app[web.1]: Traceback (most recent call last):
2022-01-07T10:14:15.759252+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.9/site-packages/streamlit/script_runner.py", line 354, in _run_script
2022-01-07T10:14:15.759252+00:00 app[web.1]: exec(code, module.__dict__)
2022-01-07T10:14:15.759252+00:00 app[web.1]: File "/app/app.py", line 204, in <module>
2022-01-07T10:14:15.759252+00:00 app[web.1]: main()
2022-01-07T10:14:15.759252+00:00 app[web.1]: File "/app/app.py", line 119, in main
2022-01-07T10:14:15.759253+00:00 app[web.1]: pred = request_prediction(MLFLOW_URI, ml_data)[0]
2022-01-07T10:14:15.759253+00:00 app[web.1]: File "/app/app.py", line 63, in request_prediction
2022-01-07T10:14:15.759253+00:00 app[web.1]: response = requests.request(
2022-01-07T10:14:15.759253+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.9/site-packages/requests/api.py", line 61, in request
2022-01-07T10:14:15.759253+00:00 app[web.1]: return session.request(method=method, url=url, **kwargs)
2022-01-07T10:14:15.759254+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.9/site-packages/requests/sessions.py", line 529, in request
2022-01-07T10:14:15.759254+00:00 app[web.1]: resp = self.send(prep, **send_kwargs)
2022-01-07T10:14:15.759254+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.9/site-packages/requests/sessions.py", line 645, in send
2022-01-07T10:14:15.759254+00:00 app[web.1]: r = adapter.send(request, **kwargs)
2022-01-07T10:14:15.759254+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.9/site-packages/requests/adapters.py", line 519, in send
2022-01-07T10:14:15.759254+00:00 app[web.1]: raise ConnectionError(e, request=request)
2022-01-07T10:14:15.759259+00:00 app[web.1]: requests.exceptions.ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: /invocations (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f2452d23790>: Failed to establish a new connection: [Errno 111] Connection refused'))
We have in place some guidelines to ensure we can help each other more easily:
do not upload screenshots of code or error logs, but copy-paste and format them
add some code and/or configuration which can help understand the context (i.e. Procfile, application startup code). It is great if you can include a Github repository (if your code is public of course) and ALWAYS avoid including passwords and tokens.
Back to your problem: can you post the Heroku logs and the code to run the streamlit app?
I'm basically trying to deploy the example in the CherryPy tutorial to Heroku (https://docs.cherrypy.org/en/latest/tutorials.html#tutorial-8-make-it-smoother-with-ajax). When I run the code locally, everything works fine. After deployment to Heroku, I get several error messages:
2020-04-16T17:42:19.422074+00:00 app[web.1]: Traceback (most recent call last):
2020-04-16T17:42:19.422076+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/cherrypy/_cprequest.py", line 670, in respond
2020-04-16T17:42:19.422076+00:00 app[web.1]: response.body = self.handler()
2020-04-16T17:42:19.422076+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/cherrypy/lib/encoding.py", line 217, in __call__
2020-04-16T17:42:19.422077+00:00 app[web.1]: self.body = self.oldhandler(*args, **kwargs)
2020-04-16T17:42:19.422077+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/cherrypy/_cpdispatch.py", line 61, in __call__
2020-04-16T17:42:19.422078+00:00 app[web.1]: return self.callable(*self.args, **self.kwargs)
2020-04-16T17:42:19.422078+00:00 app[web.1]: File "app.py", line 19, in GET
2020-04-16T17:42:19.422078+00:00 app[web.1]: return cherrypy.session['mystring']
2020-04-16T17:42:19.422079+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/cherrypy/__init__.py", line 246, in __getitem__
2020-04-16T17:42:19.422079+00:00 app[web.1]: return child[key]
2020-04-16T17:42:19.422079+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/cherrypy/lib/sessions.py", line 309, in __getitem__
2020-04-16T17:42:19.422080+00:00 app[web.1]: return self._data[key]
2020-04-16T17:42:19.422080+00:00 app[web.1]: KeyError: 'mystring'
2020-04-17T09:24:31.597650+00:00 app[web.1]: File "app.py", line 45, in <module>
2020-04-17T09:24:31.597765+00:00 app[web.1]: webapp.generator = StringGeneratorWebService()
2020-04-17T09:24:31.597768+00:00 app[web.1]: TypeError: expose_() missing 1 required positional argument: 'func'
2020-04-17T09:24:32.000000+00:00 app[api]: Build succeeded
My Procfile simply contains
web: python app.py
My guess is that the problem has something to do with the little css-trick that avoids refreshing the page in order display the result of a request. But, as you can see, I'm totally new to deployment and this is also my first question on stackoverflow--so my apologies in advance in case I'm missing something obvious here. Any help is much appreciated!
I have an SQLAlchemy database running on a flask app that works fine locally, but as soon as I run it on Heroku I get a strange issue (See stack trace below).
I don't have a model called 'user' so I don't know what's going on. I've made sure that I've run db.create_all().
Creating a row in my table works fine, but it's only once I try selecting some rows do I get an issue (__init__.py):
#app.route('/blog')
def blog():
title = 'Blog'
name = get_name()
blog_text = []
for post in sorted(Post.query.filter_by(category=Post.Category.blog),
key=lambda p: p.posted_at, reverse=True):
content = post.content
blog_text.append(content)
content = '\n'.join(blog_text)
return render_template('blog.html', **locals())
And my 'Post' logic in my models.py:
class Post(db.Model):
__tablename__ = 'user'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(200), nullable=False)
posted_at = db.Column(db.DateTime, nullable=False, default=datetime.now())
content = db.Column(db.String(200), nullable=False)
content_type = db.Column(db.String(200), default="md", nullable=False) # md / html
category = db.Column(db.String(200), nullable=False, default="blog") # poetry / blog
class Type:
md = "md"
html = "html"
class Category:
poetry = "poetry"
blog = "blog"
...
Here's the error:
2019-07-12T21:59:09.120100+00:00 app[web.1]: 10.63.22.102 - - [12/Jul/2019:21:59:09 +0000] "GET /blog HTTP/1.1" 500 290 "https://secure-savannah-20745.herokuapp.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
2019-07-12T21:59:09.120443+00:00 heroku[router]: at=info method=GET path="/blog" host=secure-savannah-20745.herokuapp.com request_id=c32b2484-c8ff-4910-a4e2-5089eca7c6d3 fwd="98.0.144.26" dyno=web.1 connect=2ms service=66ms status=500 bytes=455 protocol=https
2019-07-12T21:59:13.632588+00:00 app[web.1]: 10.63.22.102 - - [12/Jul/2019:21:59:13 +0000] "GET /projects HTTP/1.1" 200 2342 "https://secure-savannah-20745.herokuapp.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
2019-07-12T21:59:13.837771+00:00 app[web.1]: 10.63.22.102 - - [12/Jul/2019:21:59:13 +0000] "GET /static/circles.js HTTP/1.1" 200 0 "https://secure-savannah-20745.herokuapp.com/projects" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
2019-07-12T21:59:13.838434+00:00 heroku[router]: at=info method=GET path="/static/circles.js" host=secure-savannah-20745.herokuapp.com request_id=8476c020-fc32-47b3-a4e1-7b6faeb115ca fwd="98.0.144.26" dyno=web.1 connect=1ms service=6ms status=200 bytes=5449 protocol=https
2019-07-12T21:59:13.633394+00:00 heroku[router]: at=info method=GET path="/projects" host=secure-savannah-20745.herokuapp.com request_id=faa3d473-7bda-4e39-810b-e3994eac7390 fwd="98.0.144.26" dyno=web.1 connect=1ms service=1494ms status=200 bytes=2504 protocol=https
2019-07-12T21:59:18.074035+00:00 heroku[router]: at=info method=GET path="/blog" host=secure-savannah-20745.herokuapp.com request_id=e9748e89-dcce-43db-8db8-a16c7fe657ab fwd="98.0.144.26" dyno=web.1 connect=3ms service=12ms status=500 bytes=455 protocol=https
2019-07-12T21:59:18.070372+00:00 app[web.1]: [2019-07-12 21:59:18,069] ERROR in app: Exception on /blog [GET]
2019-07-12T21:59:18.070385+00:00 app[web.1]: Traceback (most recent call last):
2019-07-12T21:59:18.070396+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
2019-07-12T21:59:18.070398+00:00 app[web.1]: cursor, statement, parameters, context
2019-07-12T21:59:18.070403+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 550, in do_execute
2019-07-12T21:59:18.070405+00:00 app[web.1]: cursor.execute(statement, parameters)
2019-07-12T21:59:18.070407+00:00 app[web.1]: sqlite3.OperationalError: no such table: user
2019-07-12T21:59:18.070410+00:00 app[web.1]:
2019-07-12T21:59:18.070412+00:00 app[web.1]: The above exception was the direct cause of the following exception:
2019-07-12T21:59:18.070414+00:00 app[web.1]:
2019-07-12T21:59:18.070416+00:00 app[web.1]: Traceback (most recent call last):
2019-07-12T21:59:18.070418+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app
2019-07-12T21:59:18.070421+00:00 app[web.1]: response = self.full_dispatch_request()
2019-07-12T21:59:18.070423+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request
2019-07-12T21:59:18.070425+00:00 app[web.1]: rv = self.handle_user_exception(e)
2019-07-12T21:59:18.070427+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception
2019-07-12T21:59:18.070430+00:00 app[web.1]: reraise(exc_type, exc_value, tb)
2019-07-12T21:59:18.070432+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
2019-07-12T21:59:18.070434+00:00 app[web.1]: raise value
2019-07-12T21:59:18.070436+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
2019-07-12T21:59:18.070438+00:00 app[web.1]: rv = self.dispatch_request()
2019-07-12T21:59:18.070446+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
2019-07-12T21:59:18.070450+00:00 app[web.1]: return self.view_functions[rule.endpoint](**req.view_args)
2019-07-12T21:59:18.070452+00:00 app[web.1]: File "/app/main.py", line 65, in blog
2019-07-12T21:59:18.070455+00:00 app[web.1]: key=lambda p: p.posted_at, reverse=True):
2019-07-12T21:59:18.070457+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3324, in __iter__
2019-07-12T21:59:18.070459+00:00 app[web.1]: return self._execute_and_instances(context)
2019-07-12T21:59:18.070461+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3349, in _execute_and_instances
2019-07-12T21:59:18.070463+00:00 app[web.1]: result = conn.execute(querycontext.statement, self._params)
2019-07-12T21:59:18.070466+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
2019-07-12T21:59:18.070468+00:00 app[web.1]: return meth(self, multiparams, params)
2019-07-12T21:59:18.070470+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection
2019-07-12T21:59:18.070472+00:00 app[web.1]: return connection._execute_clauseelement(self, multiparams, params)
2019-07-12T21:59:18.070474+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1107, in _execute_clauseelement
2019-07-12T21:59:18.070477+00:00 app[web.1]: distilled_params,
2019-07-12T21:59:18.070479+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
2019-07-12T21:59:18.070481+00:00 app[web.1]: e, statement, parameters, cursor, context
2019-07-12T21:59:18.070484+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
2019-07-12T21:59:18.070486+00:00 app[web.1]: util.raise_from_cause(sqlalchemy_exception, exc_info)
2019-07-12T21:59:18.070488+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 399, in raise_from_cause
2019-07-12T21:59:18.070491+00:00 app[web.1]: reraise(type(exception), exception, tb=exc_tb, cause=cause)
2019-07-12T21:59:18.070493+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 153, in reraise
2019-07-12T21:59:18.070495+00:00 app[web.1]: raise value.with_traceback(tb)
2019-07-12T21:59:18.070498+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
2019-07-12T21:59:18.070500+00:00 app[web.1]: cursor, statement, parameters, context
2019-07-12T21:59:18.070502+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 550, in do_execute
2019-07-12T21:59:18.070504+00:00 app[web.1]: cursor.execute(statement, parameters)
2019-07-12T21:59:18.070507+00:00 app[web.1]: sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: user
2019-07-12T21:59:18.070513+00:00 app[web.1]: [SQL: SELECT user.id AS user_id, user.name AS user_name, user.posted_at AS user_posted_at, user.content AS user_content, user.content_type AS user_content_type, user.category AS user_category
2019-07-12T21:59:18.070515+00:00 app[web.1]: FROM user
2019-07-12T21:59:18.070517+00:00 app[web.1]: WHERE user.category = ?]
2019-07-12T21:59:18.070520+00:00 app[web.1]: [parameters: ('blog',)]
2019-07-12T21:59:18.070661+00:00 app[web.1]: (Background on this error at: http://sqlalche.me/e/e3q8)
2019-07-12T21:59:18.072919+00:00 app[web.1]: 10.63.22.102 - - [12/Jul/2019:21:59:18 +0000] "GET /blog HTTP/1.1" 500 290 "https://secure-savannah-20745.herokuapp.com/projects" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.100 Safari/537.36"
You missed that your table is named user in your model:
class Post(db.Model):
__tablename__ = 'user'
# ^^^^^^
The reason it doesn't exist is because Heroku doesn't keep your filesystem changes. You are using a SQLite database:
... app[web.1]: sqlite3.OperationalError: no such table: user
# ^^^^^^^
but a sqlite database is stored on the filesystem of your Heroku dyno. Heroku cleans up (cycles) dynos all the time and spins up a new dyno for you for new requests. From their documentation on how Heroku works:
Changes to the filesystem on one dyno are not propagated to other dynos and are not persisted across deploys and dyno restarts. A better and more scalable approach is to use a shared resource such as a database or queue.
Each time a new dyno is spun up to handle your requests, you'll have a new, clean filesystem without your sqlite database, and so an empty database is created again without any tables in it.
Bottom line: You can't use sqlite on Heroku.
Use a Postgres database instead, see the Heroku Postgres documentation. Don't worry, there is a free tier you can use. When you provision the database as an add-on, the DATABASE_URL config var is set for you, which you can read from your app environment.
I usually configure Flask apps that are designed to run on Heroku to read the environment variable but to fall back to a SQLite database when the environment variable is not set:
import os
from pathlib import Path
_project_root = Path(__file__).resolve().parent.parent
_default_sqlite_db = _project_root / "database.db"
SQLALCHEMY_DATABASE_URI = os.environ.get(
"DATABASE_URL", f"sqlite:///{_default_sqlite_db}"
)
This makes it easy to develop the app locally on SQLite, or set up a local Postgres database for integration testing, and then deploy to Heroku for staging or production.
This question has been asked a lot, but still, nothing seems to be working. My application runs perfectly in testing, but when run on heroku, I get the error 500 saying "we're sorry , but something went wrong." I've checked the log files for the server, and my error seems to be here somewhere:
2012-10-21T15:29:40+00:00 app[web.1]:
2012-10-21T15:29:40+00:00 app[web.1]: Started GET "/" for 117.192.23.223 at 2012-10-21 15:29:40 +0000
2012-10-21T15:29:40+00:00 app[web.1]: cache: [GET /] miss
2012-10-21T15:29:40+00:00 app[web.1]:
2012-10-21T15:29:40+00:00 heroku[router]: GET whispering-coast-7415.herokuapp.com/ dyno=web.1 queue=0 wait=0ms service=5ms status=301 bytes=115
2012-10-21T15:29:40+00:00 app[web.1]:
2012-10-21T15:29:40+00:00 app[web.1]: Started GET "/movies" for 117.192.23.223 at 2012-10-21 15:29:40 +0000
2012-10-21T15:29:40+00:00 app[web.1]:
2012-10-21T15:29:40+00:00 app[web.1]: Processing by MoviesController#index as HTML
2012-10-21T15:29:40+00:00 app[web.1]: Completed 500 Internal Server Error in 2ms
2012-10-21T15:29:40+00:00 heroku[router]: GET whispering-coast-7415.herokuapp.com/movies dyno=web.1 queue=0 wait=0ms service=17ms status=500 bytes=728
2012-10-21T15:29:40+00:00 app[web.1]: ActiveRecord::StatementInvalid (PG::Error: ERROR: column "movies.id" must appear in the GROUP BY clause or be used in an aggregate function
2012-10-21T15:29:40+00:00 app[web.1]: LINE 1: SELECT "movies".* FROM "movies" GROUP BY rating
2012-10-21T15:29:40+00:00 app[web.1]:
2012-10-21T15:29:40+00:00 app[web.1]: ^
2012-10-21T15:29:40+00:00 app[web.1]:
2012-10-21T15:29:40+00:00 app[web.1]: cache: [GET /movies] miss
2012-10-21T15:29:40+00:00 app[web.1]: : SELECT "movies".* FROM "mo
vies" GROUP BY rating):
2012-10-21T15:29:40+00:00 app[web.1]: app/controllers/movies_controller.rb:11:in `index'
2012-10-21T15:29:40+00:00 app[web.1]: app/models/movie.rb:4:in `get_ratings'
2012-10-21T15:29:40+00:00 app[web.1]:
2012-10-21T15:29:41+00:00 heroku[router]: GET whispering-coast-7415.herokuapp.com/favicon.ico dyno=web.1 queue=0 wait=0ms service=8ms status=200 bytes=0
2012-10-21T15:29:41+00:00 app[web.1]: cache: [GET /favicon.ico] stale, invalid, store
PostgreSQL (on Heroku) requires you to GROUP BY all the columns in the SELECT list so in this case instead of:
SELECT "movies".* FROM "movies" GROUP BY rating
you might want to do something like:
SELECT rating FROM "movies" GROUP BY rating
or in Rails:
Movie.select(:rating).group(:rating)
So on my localhost, the custom devise forms work great! No issues whatsoever - when I log in from my custom form on a different page, it logs the user in and redirects to the homepage as expected.
However, when I deploy to Heroku, the app stops behaving - the custom form directs the user to the actual login page without logging them in.
This is the output on my Heroku Logs:
2012-10-18T09:08:08+00:00 app[web.1]: Started POST "/user/sign_in" for 116.15.222.245 at 2012-10-18 09:08:08 +0000
2012-10-18T09:08:08+00:00 app[web.1]: cache: [POST /user/sign_in] invalidate, pass
2012-10-18T09:08:08+00:00 app[web.1]: Processing by UserSessionsController#create as HTML
2012-10-18T09:08:08+00:00 app[web.1]: Parameters: {"utf8"=>"✓", "authenticity_token"=>"4WDTRkX842UVz7nFuGbWbA3z47+rraKXQgl3PRYuWJY=", "user"=>{"email"=>"spree#example.com", "password"=>"[FILTERED]", "remember_me"=>"0"}, "commit"=>"Log In"}
2012-10-18T09:08:08+00:00 app[web.1]: Redirected to https://example.com/user/sign_in
2012-10-18T09:08:08+00:00 app[web.1]: Completed 302 Found in 11ms
2012-10-18T09:08:08+00:00 heroku[router]: POST example.com/user/sign_in dyno=web.1 queue=0 wait=0ms service=16ms status=302 bytes=101
2012-10-18T09:08:08+00:00 heroku[nginx]: 108.162.225.233 - - [18/Oct/2012:09:08:08 +0000] "POST /user/sign_in HTTP/1.1" 302 101 "http://example.com/referral_login" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:15.0) Gecko/20100101 Firefox/15.0.1" example.com
2012-10-18T09:08:10+00:00 app[web.1]:
2012-10-18T09:08:10+00:00 app[web.1]: Started GET "/user/sign_in" for 116.15.222.245 at 2012-10-18 09:08:10 +0000
2012-10-18T09:08:10+00:00 app[web.1]:
2012-10-18T09:08:10+00:00 heroku[router]: GET example.com/user/sign_in dyno=web.1 queue=0 wait=0ms service=16ms status=200 bytes=7835
2012-10-18T09:08:10+00:00 app[web.1]: cache: [GET /user/sign_in] miss
2012-10-18T09:08:10+00:00 app[web.1]: Processing by UserSessionsController#new as HTML
2012-10-18T09:08:10+00:00 app[web.1]: Rendered shared/_error_messages.html.erb (0.1ms)
2012-10-18T09:08:10+00:00 app[web.1]: Rendered shared/_login.html.erb (2.3ms)
2012-10-18T09:08:10+00:00 app[web.1]: Rendered user_sessions/new.html.erb within layouts/spree_application (3.0ms)
2012-10-18T09:08:10+00:00 app[web.1]: Rendered shared/_head.html.erb (1.0ms)
2012-10-18T09:08:10+00:00 app[web.1]: Rendered shared/_store_menu.html.erb (0.7ms)
2012-10-18T09:08:10+00:00 app[web.1]: Rendered shared/_top_menu_bar.html.erb (0.5ms)
2012-10-18T09:08:10+00:00 heroku[nginx]: 108.162.225.233 - - [18/Oct/2012:09:08:10 +0000] "GET /user/sign_in HTTP/1.1" 200 3412 "http://example.com/referral_login" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:15.0) Gecko/20100101 Firefox/15.0.1" example.com
This is my output on my localhost:
Started POST "/user/sign_in" for 127.0.0.1 at 2012-10-18 17:21:36 +0800
Processing by UserSessionsController#create as HTML
Parameters: {"utf8"=>"✓", "authenticity_token"=>"m38Y47hk4UyGYAMnd76y5Qo2nK1Nv//v7s2+XdNHCbU=", "user"=>{"email"=>"spree#example.com", "password"=>"[FILTERED]", "remember_me"=>"0"}, "commit"=>"Log In"}
User Load (0.4ms) SELECT "users".* FROM "users" WHERE "users"."email" = 'spree#example.com' LIMIT 1
(23.6ms) SELECT COUNT(DISTINCT "users"."id") FROM "users" LEFT OUTER JOIN "roles_users" ON "roles_users"."user_id" = "users"."id" LEFT OUTER JOIN "roles" ON "roles"."id" = "roles_users"."role_id" WHERE "roles"."name" = 'admin'
(0.5ms) UPDATE "users" SET "last_sign_in_at" = '2012-10-18 05:56:54.897695', "current_sign_in_at" = '2012-10-18 09:21:39.033308', "sign_in_count" = 75, "updated_at" = '2012-10-18 09:21:39.081684' WHERE "users"."id" = 1
Redirected to http://localhost:3000/
Completed 302 Found in 227ms
Started GET "/" for 127.0.0.1 at 2012-10-18 17:21:39 +0800
Processing by LandingController#index as HTML
User Load (0.2ms) SELECT "users".* FROM "users" WHERE "users"."id" = 1 LIMIT 1
Rendered landing/index.html.erb within layouts/spree_application (1.7ms)
Rendered shared/_head.html.erb (2.0ms)
Rendered shared/_store_menu.html.erb (2.6ms)
Rendered shared/_top_menu_bar.html.erb (1.7ms)
Tracker Load (0.3ms) SELECT "trackers".* FROM "trackers" WHERE "trackers"."active" = 't' AND "trackers"."environment" = 'development' LIMIT 1
Rendered shared/_google_analytics.html.erb (14.1ms)
Completed 200 OK in 183ms (Views: 148.1ms | ActiveRecord: 1.9ms)
Can anyone help to shed some light as to why this might be and how to go about resolving this? I looked through Devise wiki, did a bunch of Google searches and tried a few things without success. Would appreciate any help!
Spree defaults to require SSL in production environments, so if your POST request is not https, it'll redirect to the login page.