I just created a new PR App in Heroku with the GrapheneDB add on.
I'm seeing the following in my logs:
2018-05-18T20:11:52.702973+00:00 app[web.1]: neo4j.exceptions.ProtocolError: Connection to ('ip', 24786) closed without handshake response
2018-05-18T20:11:52.704465+00:00 app[web.1]: 2018-05-18 20:11:52,704 INFO ~~ [CONNECT] ('ip', 24786)
2018-05-18T20:11:52.705999+00:00 app[web.1]: 2018-05-18 20:11:52,705 INFO C: [HANDSHAKE] 0x6060B017 [1, 0, 0, 0]
2018-05-18T20:11:52.707922+00:00 app[web.1]: 2018-05-18 20:11:52,707 ERROR S: [CLOSE]
2018-05-18T20:11:52.710745+00:00 app[web.1]: 2018-05-18 20:11:52,708 ERROR Connection to ('ip', 24786) closed without handshake response
So far I have:
Connected to the browser
nslookup from the console shows the same ip as the logs (I can't ping from the heroku instance)
Rotated database credentials from the GrapheneDB dashboard
Removed the add on and re-added it
Am I missing something?
Env:
Python 3.6.5
neo4j-driver 1.5.3
Driver Python Code:
from django.conf import settings
from django.utils.functional import SimpleLazyObject
from neo4j.v1 import GraphDatabase, basic_auth, TRUST_ON_FIRST_USE
def get_driver():
return GraphDatabase.driver(
settings.NEO4J_BOLT_URL,
auth=basic_auth(settings.NEO4J_USERNAME, settings.NEO4J_PASSWORD),
encrypted=False,
trust=TRUST_ON_FIRST_USE,
)
driver = SimpleLazyObject(get_driver)
Just to follow up. It turns out that TRUST_ON_FIRST_USE does not work with GrapheneDB. TRUST_ON_FIRST_USE is also now deprecated.
Removing it allowed the connection to Neo4j work.
Related
Modifying an app based on this example, I am having a problem when deploying my updated Heroku app during initialization of the web.1 dyno.
Everything is working fine in my development envr; when I start the 'app' and 'worker' executables directly, I can access the working app from 127.0.0.1:5000
I already have the postgresql and redistogo addons created as part of the app...
heroku config file
=== app Config Vars
APP_SETTINGS: config.StagingConfig
DATABASE_URL: postgres://[points-to-AWS-EC2].compute-1.amazonaws.com:[port]/[identifier]
REDISTOGO_URL: redis://redistogo:[identifier]#pike.redistogo.com:[port]/
Procfile
web: bash heroku.sh
heroku.sh
#!/bin/bash
gunicorn app:app --daemon
python worker.py
app.py
from collections import Counter
from datetime import datetime
import json
import re
import operator
import os
from flask import Flask, render_template, request, jsonify
from flask_sqlalchemy import SQLAlchemy
from requests import get, exceptions as rx
import nltk
from rq import Queue
from rq.job import Job
from validators import url as vurl
from worker import conn
app = Flask(__name__, template_folder='app-templates/')
app.config.from_object(os.environ['APP_SETTINGS'])
db = SQLAlchemy(app)
q = Queue(connection=conn)
def worker_task(url):
'''
takes website url and ...
worker.py
import os
import redis
from rq import Worker, Queue, Connection
listen = ['default']
redis_url = os.getenv('REDISTOGO_URL', 'redis://localhost:6379')
conn = redis.from_url(redis_url)
if __name__ == '__main__':
# for tracking worker job in a terminal
# assumptions: redis-server is running
try:
with Connection(conn):
worker = Worker(list(map(Queue, listen)))
--->>> worker.work() <<<---
except Exception as e:
print(f"{str(e)}")
During the dyno startup, I get the following errors:
heroku[web.1]: State changed from crashed to starting
heroku[web.1]: Starting process with command bash heroku.sh
app[web.1]: Command # 2 (HSET rq:worker:2bd1d3d3a7f04d2fb7a09b34786689bf
birth 2020-05-23T11:28:04.485474Z
last_heartbeat 2020-05-23T11:28:04.485474Z
queues default pid 11
hostname b428750b-5d7f-4837-b9c2-b2195e234978
version 1.4.1 python_version 3.7.4 (default, Sep 12 2019, 01:19:52)
app[web.1]: [GCC 7.4.0]) of pipeline caused error: wrong number of arguments for 'hset' command
heroku[web.1]: Process exited with status 0
heroku[web.1]: State changed from starting to crashed
Im fairly confident that the error is in the highlighted worker.work() line from the worker.py file, because I get the same error when I run heroku run python worker.py (or directly copying those lines into the heroku run python REPL interpreter)
I had a similar problem, and I fixed it by downgrading my version of rq from 1.4.1 to 1.3.0.
SOLVED was an issue with my package versions...
the original example I was using had the following package version dependencies:
redis = "==3.4.1"
rq = "==1.2.2"
but I was just using the most recently updated packages = "*" for both, and when I originally deployed the package in Heroku, it installed RQ version "1.4.0", but then (apparently) when my new app was deployed, the Heroku installer used the latest 1.4.1 package, which broke during the worker.work() method...
Worker RQ v 1.4.0 output:
{
'connection': Redis<ConnectionPool<Connection<host=pike.redistogo.com,port=10038,db=0>>>,
'hostname': 'e868b6d3-70b4-4e17-98c2-b96aae04d9a8',
'pid': 39,
'job_class': <class 'rq.job.Job'>,
'queue_class': <class 'rq.queue.Queue'>, 'version': '1.4.0',
**'python_version': '3.7.4 (default, Sep 12 2019, 01:19:52) \n[GCC 7.4.0]',**
**'serializer': <module 'pickle' from '/app/.heroku/python/lib/python3.7/pickle.py'>,**
'name': '05c7af34c21f44f0b4374b40525779af',
'queues': [Queue('default')],
'_exc_handlers': [], ... ...
}
Worker RQ v1.4.1 output:
{
'connection': Redis<ConnectionPool<Connection<host=pike.redistogo.com,port=11022,db=0>>>,
'hostname': '19a476fa-b4e6-4e63-b40e-ac8779ae0f9e',
'pid': 39,
'job_class': <class 'rq.job.Job'>,
'queue_class': <class 'rq.queue.Queue'>,
'version': '1.4.1',
**'python_version': '3.7.4 (default, Sep 12 2019, 01:19:52) \n[GCC 7.4.0]',**
**'serializer': <class 'rq.serializers.DefaultSerializer'>,**
'name': 'b63389b3499f4d73be889a33f3777b46',
'queues': [Queue('default')],
'_exc_handlers': [], ... ...
}
You can see a difference in the 'serializer' key between the two, possibly causing the HSET exception that was being throw...
Reverting to RQ v1.2.2 solved the issue for me; I probably could have used the same v1.4.0...
I have a flask app which I've deployed to Heroku, one of the routes is the following
def get_kws():
seed_kw = request.json['firstParam']
audience_max = request.json['secondParam']
interest_mining_service = InterestMiningService(seed_kw, audience_max)
query_result = interest_mining_service.query_keyword().tolist()
if seed_kw in query_result:
print ("yes")
return jsonify(
{
'keyword_data' : interest_mining_service.find_kws().to_json(orient='records'),
'query_results': query_result
}
)
When I test this endpoint locally, I have no issues when sending POST and GET requests to that endpoint. However, when I deploy to Heroku, I get the following error:
File "/app/server/controller.py", line 24, in get_kws
2020-02-08T22:31:05.893850+00:00 app[web.1]: 'query_results': query_result
2020-02-08T22:31:05.893850+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/json.py", line 298, in jsonify
2020-02-08T22:31:05.893851+00:00 app[web.1]: if current_app.config['JSONIFY_PRETTYPRINT_REGULAR'] and not request.is_xhr:
2020-02-08T22:31:05.893851+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/werkzeug/local.py", line 347, in __getattr__
2020-02-08T22:31:05.893852+00:00 app[web.1]: return getattr(self._get_current_object(), name)
2020-02-08T22:31:05.893858+00:00 app[web.1]: AttributeError: 'Request' object has no attribute 'is_xhr'
I've never seen this error Request object has no attribute 'is_xhr' before and it only seems to be happening when I deploy to Heroku. Any guidance on what I should look into?
There also doesn't seem to be an issue with the json key keyword_data - the issue seems limited to query_results which is a list.
The Werkzeug library (dependency from Flask) recently received a major update (0.16.1 --> 1.0.0) and it looks like Flask (<=0.12.4) does not restrict the version of Werkzeug that is fetched.
You have 2 options:
Stick with your current version of Flask and restrict the Werkzeug version that is fetched explicitly in your application's setup.py or requirements.txt by specifying werkzeug<1.0 or werkzeug==0.16.1
Upgrade to a recent version of Flask (>=1.0.0), which is running fine with latest Werkzeug
Or you can just forcefully install the bustard again by calling
pip install Werkzeug==0.16.1
I have faced with this problem too.
Just temporarily fixed by directly checking in request header
request.headers.get("X-Requested-With") == "XMLHttpRequest"
not sure this help ...
I programmed a simple bot that is connected to Google Sheets API. It asks a few questions to the user and stores it in the Sheet I have. When I use bot.polling it works fine, but I was trying the webhook so I could use Heroku to have my bot running 24/7.
I introduced the code:
#server.route('/' + token, methods=['POST'])
def getMessage():
bot.process_new_updates([telebot.types.Update.de_json(request.stream.read().decode("utf-8"))])
return "!", 200
#server.route("/")
def webhook():
bot.remove_webhook()
bot.set_webhook(url = '<Heroku App Link>' + token)
return "!", 200
if __name__ == "__main__":
server.run(host="0.0.0.0", port = int(os.environ.get('PORT', 5000)))
I have the requirements.txt file and the Procfile that currently is like this:
web: python3 bot.py
I tried a bunch of things in the Procfile but this is the only one that didn't gave an error.
When I saw the log this is what I got:
2019-09-24T10:18:54.369238+00:00 heroku[web.1]: State changed from crashed to starting
2019-09-24T10:19:04.416777+00:00 heroku[web.1]: Starting process with command `python3 bot.py`
2019-09-24T10:19:09.868915+00:00 heroku[web.1]: State changed from starting to up
2019-09-24T10:19:09.790773+00:00 app[web.1]: * Serving Flask app "bot" (lazy loading)
2019-09-24T10:19:09.790800+00:00 app[web.1]: * Environment: production
2019-09-24T10:19:09.790853+00:00 app[web.1]: WARNING: This is a development server. Do not use it in a production deployment.
2019-09-24T10:19:09.790940+00:00 app[web.1]: Use a production WSGI server instead.
2019-09-24T10:19:09.790991+00:00 app[web.1]: * Debug mode: off
2019-09-24T10:19:09.799470+00:00 app[web.1]: * Running on http://0.0.0.0:57826/ (Press CTRL+C to quit)
2019-09-24T10:19:15.000000+00:00 app[api]: Build succeeded
Can anyone help me understand why the bot doesn't answer any commands?
I and using jupyter notebook's PySpark kernel, I have successfully selected PySpark kernel but I keep getting the below error
The code failed because of a fatal error:
Error sending http request and maximum retry encountered..
Some things to try:
a) Make sure Spark has enough available resources for Jupyter to create a Spark context.
b) Contact your Jupyter administrator to make sure the Spark magics library is configured correctly.
c) Restart the kernel.
here's the log also
2019-10-10 13:37:43,741 DEBUG SparkMagics Initialized spark magics.
2019-10-10 13:37:43,742 INFO EventsHandler InstanceId: 32a21583-6879-4ad5-88bf-e07af0b09387,EventName: notebookLoaded,Timestamp: 2019-10-10 10:37:43.742475
2019-10-10 13:37:43,744 DEBUG python_jupyter_kernel Loaded magics.
2019-10-10 13:37:43,744 DEBUG python_jupyter_kernel Changed language.
2019-10-10 13:37:44,356 DEBUG python_jupyter_kernel Registered auto viz.
2019-10-10 13:37:45,440 INFO EventsHandler InstanceId: 32a21583-6879-4ad5-88bf-e07af0b09387,EventName: notebookSessionCreationStart,Timestamp: 2019-10-10 10:37:45.440323,SessionGuid: d230b1f3-6bb1-4a66-bde1-7a73a14d7939,LivyKind: pyspark
2019-10-10 13:37:49,591 ERROR ReliableHttpClient Request to 'http://localhost:8998/sessions' failed with 'HTTPConnectionPool(host='localhost', port=8998): Max retries exceeded with url: /sessions (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0000013184159808>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it'))'
2019-10-10 13:37:49,591 INFO EventsHandler InstanceId: 32a21583-6879-4ad5-88bf-e07af0b09387,EventName: notebookSessionCreationEnd,Timestamp: 2019-10-10 10:37:49.591650,SessionGuid: d230b1f3-6bb1-4a66-bde1-7a73a14d7939,LivyKind: pyspark,SessionId: -1,Status: not_started,Success: False,ExceptionType: HttpClientException,ExceptionMessage: Error sending http request and maximum retry encountered.
2019-10-10 13:37:49,591 ERROR SparkMagics Error creating session: Error sending http request and maximum retry encountered.
note that I am trying to configure this on windows.
thanks alot
I faced the same issue, you can solve it by not using a PySpark Kernel (notebook) but a Python 3 kernel (notebook). I used the following code to setup the Spark cluster:
import pyspark # only run after findspark.init()
from pyspark.sql import SparkSession
# May take awhile locally
spark = SparkSession.builder.appName("test").getOrCreate()
spark
If you are trying to connect your Jupyter Notebook to a Spark server through Livy (e.g. AWS Glue Development Endpoint), you have to replace "localhost" with the Spark server IP address in: ~/.sparkmagic/config.json
As mentioned here:
https://aws.amazon.com/blogs/machine-learning/build-amazon-sagemaker-notebooks-backed-by-spark-in-amazon-emr/
Posting below answer as it may help someone facing this issue when using sagemaker notebook with Glue Dev Endpoint.
I received same error message in my PySpark kernel notebook. In my case issue was missing Lifecycle configuration attached to notebook instance which was somehow removed. I delete and recreate dev endpoint every day but it lifecycle config normally remains attached to notebook.
I have a Flask web app that's running perfectly fine on localhost. When I deployed to Heroku, it was a successful build. However, when I opened the web app, there was an error. This was the log message:
2016-11-21T07:17:17.106395+00:00 app[web.1]: File "/app/routes.py", line 2, in <module>
2016-11-21T07:17:17.106395+00:00 app[web.1]: from data import SearchBar, CourseData
2016-11-21T07:17:17.106395+00:00 app[web.1]: File "/app/data.py", line 1, in <module>
2016-11-21T07:17:17.106396+00:00 app[web.1]: from flask_wtf import Form
2016-11-21T07:17:17.106396+00:00 app[web.1]: ImportError: No module named flask_wtf
I have WTForms installed. I'm not sure why it's not running on Heroku, though it's running fine on localhost.