I have a flask app which I've deployed to Heroku, one of the routes is the following
def get_kws():
seed_kw = request.json['firstParam']
audience_max = request.json['secondParam']
interest_mining_service = InterestMiningService(seed_kw, audience_max)
query_result = interest_mining_service.query_keyword().tolist()
if seed_kw in query_result:
print ("yes")
return jsonify(
{
'keyword_data' : interest_mining_service.find_kws().to_json(orient='records'),
'query_results': query_result
}
)
When I test this endpoint locally, I have no issues when sending POST and GET requests to that endpoint. However, when I deploy to Heroku, I get the following error:
File "/app/server/controller.py", line 24, in get_kws
2020-02-08T22:31:05.893850+00:00 app[web.1]: 'query_results': query_result
2020-02-08T22:31:05.893850+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/flask/json.py", line 298, in jsonify
2020-02-08T22:31:05.893851+00:00 app[web.1]: if current_app.config['JSONIFY_PRETTYPRINT_REGULAR'] and not request.is_xhr:
2020-02-08T22:31:05.893851+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.6/site-packages/werkzeug/local.py", line 347, in __getattr__
2020-02-08T22:31:05.893852+00:00 app[web.1]: return getattr(self._get_current_object(), name)
2020-02-08T22:31:05.893858+00:00 app[web.1]: AttributeError: 'Request' object has no attribute 'is_xhr'
I've never seen this error Request object has no attribute 'is_xhr' before and it only seems to be happening when I deploy to Heroku. Any guidance on what I should look into?
There also doesn't seem to be an issue with the json key keyword_data - the issue seems limited to query_results which is a list.
The Werkzeug library (dependency from Flask) recently received a major update (0.16.1 --> 1.0.0) and it looks like Flask (<=0.12.4) does not restrict the version of Werkzeug that is fetched.
You have 2 options:
Stick with your current version of Flask and restrict the Werkzeug version that is fetched explicitly in your application's setup.py or requirements.txt by specifying werkzeug<1.0 or werkzeug==0.16.1
Upgrade to a recent version of Flask (>=1.0.0), which is running fine with latest Werkzeug
Or you can just forcefully install the bustard again by calling
pip install Werkzeug==0.16.1
I have faced with this problem too.
Just temporarily fixed by directly checking in request header
request.headers.get("X-Requested-With") == "XMLHttpRequest"
not sure this help ...
Related
Modifying an app based on this example, I am having a problem when deploying my updated Heroku app during initialization of the web.1 dyno.
Everything is working fine in my development envr; when I start the 'app' and 'worker' executables directly, I can access the working app from 127.0.0.1:5000
I already have the postgresql and redistogo addons created as part of the app...
heroku config file
=== app Config Vars
APP_SETTINGS: config.StagingConfig
DATABASE_URL: postgres://[points-to-AWS-EC2].compute-1.amazonaws.com:[port]/[identifier]
REDISTOGO_URL: redis://redistogo:[identifier]#pike.redistogo.com:[port]/
Procfile
web: bash heroku.sh
heroku.sh
#!/bin/bash
gunicorn app:app --daemon
python worker.py
app.py
from collections import Counter
from datetime import datetime
import json
import re
import operator
import os
from flask import Flask, render_template, request, jsonify
from flask_sqlalchemy import SQLAlchemy
from requests import get, exceptions as rx
import nltk
from rq import Queue
from rq.job import Job
from validators import url as vurl
from worker import conn
app = Flask(__name__, template_folder='app-templates/')
app.config.from_object(os.environ['APP_SETTINGS'])
db = SQLAlchemy(app)
q = Queue(connection=conn)
def worker_task(url):
'''
takes website url and ...
worker.py
import os
import redis
from rq import Worker, Queue, Connection
listen = ['default']
redis_url = os.getenv('REDISTOGO_URL', 'redis://localhost:6379')
conn = redis.from_url(redis_url)
if __name__ == '__main__':
# for tracking worker job in a terminal
# assumptions: redis-server is running
try:
with Connection(conn):
worker = Worker(list(map(Queue, listen)))
--->>> worker.work() <<<---
except Exception as e:
print(f"{str(e)}")
During the dyno startup, I get the following errors:
heroku[web.1]: State changed from crashed to starting
heroku[web.1]: Starting process with command bash heroku.sh
app[web.1]: Command # 2 (HSET rq:worker:2bd1d3d3a7f04d2fb7a09b34786689bf
birth 2020-05-23T11:28:04.485474Z
last_heartbeat 2020-05-23T11:28:04.485474Z
queues default pid 11
hostname b428750b-5d7f-4837-b9c2-b2195e234978
version 1.4.1 python_version 3.7.4 (default, Sep 12 2019, 01:19:52)
app[web.1]: [GCC 7.4.0]) of pipeline caused error: wrong number of arguments for 'hset' command
heroku[web.1]: Process exited with status 0
heroku[web.1]: State changed from starting to crashed
Im fairly confident that the error is in the highlighted worker.work() line from the worker.py file, because I get the same error when I run heroku run python worker.py (or directly copying those lines into the heroku run python REPL interpreter)
I had a similar problem, and I fixed it by downgrading my version of rq from 1.4.1 to 1.3.0.
SOLVED was an issue with my package versions...
the original example I was using had the following package version dependencies:
redis = "==3.4.1"
rq = "==1.2.2"
but I was just using the most recently updated packages = "*" for both, and when I originally deployed the package in Heroku, it installed RQ version "1.4.0", but then (apparently) when my new app was deployed, the Heroku installer used the latest 1.4.1 package, which broke during the worker.work() method...
Worker RQ v 1.4.0 output:
{
'connection': Redis<ConnectionPool<Connection<host=pike.redistogo.com,port=10038,db=0>>>,
'hostname': 'e868b6d3-70b4-4e17-98c2-b96aae04d9a8',
'pid': 39,
'job_class': <class 'rq.job.Job'>,
'queue_class': <class 'rq.queue.Queue'>, 'version': '1.4.0',
**'python_version': '3.7.4 (default, Sep 12 2019, 01:19:52) \n[GCC 7.4.0]',**
**'serializer': <module 'pickle' from '/app/.heroku/python/lib/python3.7/pickle.py'>,**
'name': '05c7af34c21f44f0b4374b40525779af',
'queues': [Queue('default')],
'_exc_handlers': [], ... ...
}
Worker RQ v1.4.1 output:
{
'connection': Redis<ConnectionPool<Connection<host=pike.redistogo.com,port=11022,db=0>>>,
'hostname': '19a476fa-b4e6-4e63-b40e-ac8779ae0f9e',
'pid': 39,
'job_class': <class 'rq.job.Job'>,
'queue_class': <class 'rq.queue.Queue'>,
'version': '1.4.1',
**'python_version': '3.7.4 (default, Sep 12 2019, 01:19:52) \n[GCC 7.4.0]',**
**'serializer': <class 'rq.serializers.DefaultSerializer'>,**
'name': 'b63389b3499f4d73be889a33f3777b46',
'queues': [Queue('default')],
'_exc_handlers': [], ... ...
}
You can see a difference in the 'serializer' key between the two, possibly causing the HSET exception that was being throw...
Reverting to RQ v1.2.2 solved the issue for me; I probably could have used the same v1.4.0...
I have a flask, gunicorn, postgresql project hosted on heroku and it suddenly failed. I can access the logs, but there is no script that I wrote, so I am confused. I haven't added anything between "working" and "not working" so I don't know where I can start.
The log can be found in this pastebin.
The last part is:
2020-02-06T21:09:02.748093+00:00 app[web.1]: from werkzeug.contrib.cache import FileSystemCache
2020-02-06T21:09:02.748100+00:00 app[web.1]: ModuleNotFoundError: No module named 'werkzeug.contrib'
2020-02-06T21:09:02.748789+00:00 app[web.1]: [2020-02-06 21:09:02 +0000] [10] [INFO] Worker exiting (pid: 10)
I tried to add werkzeug to the requirements.txt, but that did not help. Which would have been strange anyway, because it was working fine without the change in the requirement.
If you could help me reducing the requirements.txt, it would be greatly appreciated.
Original requirements.txt:
cs50
Flask
Flask-Session
requests
gunicorn
psycopg2-binary
openpyxl
New, working ones:
astroid==2.3.3
attrs==19.3.0
Authlib==0.13
autopep8==1.5
awscli==1.17.9
backports.shutil-get-terminal-size==1.0.0
backports.shutil-which==3.5.2
beautifulsoup4==4.8.2
botocore==1.14.9
bs4==0.0.1
cairocffi==1.1.0
CairoSVG==2.4.2
certifi==2019.11.28
cffi==1.13.2
chardet==3.0.4
check50==3.0.10
Click==7.0
colorama==0.4.1
compare50==1.1.2
cryptography==2.8
cs50==5.0.3
cssselect2==0.2.2
cycler==0.10.0
defusedxml==0.6.0
docutils==0.15.2
EditorConfig==0.12.2
et-xmlfile==1.0.1
Flask==1.1.1
Flask-Session==0.3.1
help50==3.0.0
html5lib==1.0.1
icdiff==1.9.1
idna==2.8
ikp3db==1.4.1
intervaltree==2.1.0
isort==4.3.21
itsdangerous==1.1.0
jdcal==1.4.1
jellyfish==0.7.2
Jinja2==2.11.1
jmespath==0.9.4
jsbeautifier==1.10.3
kiwisolver==1.1.0
lazy-object-proxy==1.4.3
lib50==2.0.7
logger==1.4
MarkupSafe==1.1.1
matplotlib==3.1.3
mccabe==0.6.1
natsort==7.0.1
nltk==3.4.5
numpy==1.18.1
oauthlib==3.1.0
openpyxl==3.0.3
pandas==1.0.0
pexpect==4.8.0
Pillow==7.0.0
plotly==4.5.0
psycopg2-binary==2.8.4
ptyprocess==0.6.0
pyasn1==0.4.8
pycodestyle==2.5.0
pycparser==2.19
Pygments==2.5.2
pylint==2.4.4
pylint-django==2.0.13
pylint-flask==0.6
pylint-plugin-utils==0.6
pyparsing==2.4.6
PyPDF2==1.26.0
Pyphen==0.9.5
python-dateutil==2.8.1
python-magic==0.4.15
pytz==2019.3
PyYAML==5.2
render50==3.1.3
requests==2.22.0
requests-oauthlib==1.3.0
retrying==1.3.3
rsa==3.4.2
s3cmd==2.0.2
s3transfer==0.3.2
six==1.14.0
sortedcontainers==2.1.0
soupsieve==1.9.5
SQLAlchemy==1.3.13
sqlparse==0.3.0
style50==2.7.4
submit50==3.0.2
termcolor==1.1.0
tinycss2==1.0.2
tqdm==4.42.1
twython==3.7.0
typed-ast==1.4.1
urllib3==1.25.8
virtualenv==16.7.9
WeasyPrint==49
webencodings==0.5.1
Werkzeug==0.16.1
wrapt==1.11.2
gunicorn
Werkzeug released a new version yesterday :
Release history
Apparently werkzeug.contrib has been moved to a separate module
It is recommended to try
./env/bin/pip install werkzeug==0.16.0
Here's another solution that might work for you.
Since that Werkzeug 1.0.0 has removed deprecated code from werkzeug.contrib.
I ran into the same issue when trying to use Proxyfix in werkzeug==1.0.0
After downgrading to werkzeug==0.16.0, I got these warnings:
DeprecationWarning: 'werkzeug.contrib.fixers.ProxyFix' has moved to 'werkzeug.middleware.proxy_fix.ProxyFix'.
This import is deprecated as of version 0.15 and will be removed in 1.0.
DeprecationWarning: 'werkzeug.contrib.cache' is deprecated as of version 0.15 and will be removed in version 1.0. It has moved to pallets.
from werkzeug.contrib.cache import FileSystemCache
To fix the issues:
pip install werkzeug==1.0.0
For ProxyFix:
from werkzeug.middleware.proxy_fix import ProxyFix
For FileSystemCache, you will have to install pallets:
pip install -U cachelib
from cachelib.file import FileSystemCache
I hope this helps and fix your issues 😄
I have been using nipyapi client to manage new Apache NiFi deployments and is working great, but i am getting an issue when trying to ENABLE a Controller Services.
My Setup:
I run NiFi in docker and every time a container starts there is a series of steps such as :
Build NiFi server - OK
Download the temapltes.xml - OK
Upload templates to NiFi - OK
Deploy templates to NiFi Canvas - OK
ENABLE Controller Service - ERROR
import nipyapi
nipyapi.config.nifi_config.host = 'http://localhost:9999/nifi-api'
nipyapi.canvas.get_controller('MariaDB', identifier_type='name', bool_response=False)
#Enable Controler
headers = {'Content-Type': 'application/json'}
url = 'http://localhost:9999/nifi-api/flow/process-groups/'+nipyapi.canvas.get_root_pg_id()+'/controller-services'
r = requests.get(url)
reponse = json.loads(r.text)
controllerId = reponse['controllerServices'][0]['id']
nipyapi.canvas.schedule_controller(controllerId, 'True', refresh=False)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.5/dist-packages/nipyapi/canvas.py", line 1222, in schedule_controller
assert isinstance(controller, nipyapi.nifi.ControllerServiceEntity)
AssertionError
Not sure what i am missing !
PS - i have been trying nifi-toolkit but is not working as well
./cli.sh nifi pg-enable-services --processGroupId 2b8b54ca-016b-1000-0655-c3ec484fd81d -u http://localhost:9999 --verbose
Sometimes it works sometimes does not work !
I would like to stick with one tool eg: toolkit or nipyapi (faster)
Any Help would be great ! thx
Per the error, NiPyAPI expects to be passed the Controller object, not just the ID.
I just created a new PR App in Heroku with the GrapheneDB add on.
I'm seeing the following in my logs:
2018-05-18T20:11:52.702973+00:00 app[web.1]: neo4j.exceptions.ProtocolError: Connection to ('ip', 24786) closed without handshake response
2018-05-18T20:11:52.704465+00:00 app[web.1]: 2018-05-18 20:11:52,704 INFO ~~ [CONNECT] ('ip', 24786)
2018-05-18T20:11:52.705999+00:00 app[web.1]: 2018-05-18 20:11:52,705 INFO C: [HANDSHAKE] 0x6060B017 [1, 0, 0, 0]
2018-05-18T20:11:52.707922+00:00 app[web.1]: 2018-05-18 20:11:52,707 ERROR S: [CLOSE]
2018-05-18T20:11:52.710745+00:00 app[web.1]: 2018-05-18 20:11:52,708 ERROR Connection to ('ip', 24786) closed without handshake response
So far I have:
Connected to the browser
nslookup from the console shows the same ip as the logs (I can't ping from the heroku instance)
Rotated database credentials from the GrapheneDB dashboard
Removed the add on and re-added it
Am I missing something?
Env:
Python 3.6.5
neo4j-driver 1.5.3
Driver Python Code:
from django.conf import settings
from django.utils.functional import SimpleLazyObject
from neo4j.v1 import GraphDatabase, basic_auth, TRUST_ON_FIRST_USE
def get_driver():
return GraphDatabase.driver(
settings.NEO4J_BOLT_URL,
auth=basic_auth(settings.NEO4J_USERNAME, settings.NEO4J_PASSWORD),
encrypted=False,
trust=TRUST_ON_FIRST_USE,
)
driver = SimpleLazyObject(get_driver)
Just to follow up. It turns out that TRUST_ON_FIRST_USE does not work with GrapheneDB. TRUST_ON_FIRST_USE is also now deprecated.
Removing it allowed the connection to Neo4j work.
I'm troubled with redmine.
When I create new issue with Rest API(from Microsoft Excel), Internal Error 500 happened.
Seeing from production.log, "NoMethodError" is happened (see below message)
Started POST "/redmine/issues.xml?key=XXXXXXX" for XX.XX.XX.XXX at 2016-05-30 14:44:10 +0900
Processing by IssuesController#create as XML Parameters: {"issue"=>`
{"project_id"=>"mt001", "subject"=>"TEST_AI", "tracker_id"=>"6",
"category_id"=>"136", "assigned_to_id"=>"4", "fixed_version_id"=>"93",
"priority_id"=>"2", "parent_issue_id"=>nil,
"start_date"=>"2016-05-30", "due_date"=>"2016-06-01",
"description"=>"Redmine TEST AI", "status_id"=>"1"},
"key"=>"XXXXXXX(hidden for security"}
Current user: user1 (id=65)
Completed 500 Internal Server Error in 367ms (ActiveRecord: 19.9ms)
NoMethodError (undefined method `shared_versions' for nil:NilClass):
app/models/issue.rb:823:in `assignable_versions'
app/models/issue.rb:643:in `validate_issue'
app/controllers/issues_controller.rb:141:in `create'
lib/redmine/sudo_mode.rb:63:in `sudo_mode'
So, I tried to create new issew using web browser (IE11).
The ticket was created normally. Here is production.log.
Started POST "/redmine/projects/mt001/issues" for XX.XX.XX.XXX at 2016-05-30 14:58:23 +0900
Processing by IssuesController#create as HTML
Parameters: {"utf8"=>"✓", "authenticity_token"=>"QF1BA5MgwysJsAiFfT/h95DeZ/VgzSGOnryR6xUn4mmWpf7EZ8XFJqpaye/ZLXrKR/wV00dKOz8KbhP94VcUyg==", "form_update_triggered_by"=>"", "issue"=>{"tracker_id"=>"6", "subject"=>"Test2", "description"=>"This is test", "status_id"=>"1", "priority_id"=>"2", "assigned_to_id"=>"4", "category_id"=>"132", "fixed_version_id"=>"127", "parent_issue_id"=>"", "start_date"=>"2016-05-30", "due_date"=>"2016-06-03", "custom_field_values"=>{"12"=>"", "41"=>""}}, "was_default_status"=>"1", "commit"=>"create", "project_id"=>"mt001"}
Current user: user1 (id=65)
Rendered mailer/_issue.text.erb (7.1ms)
Rendered mailer/issue_add.text.erb within layouts/mailer (9.8ms)
Rendered mailer/_issue.html.erb (3.0ms)
Rendered mailer/issue_add.html.erb within layouts/mailer (5.1ms)
Redirected to http://mydepartment.co.jp/redmine/issues/1717
Completed 302 Found in 3697ms (ActiveRecord: 130.3ms)
I already tried to migrate, clear cache.
What should I do next.
Here is my environment on RHEL7.2
Environment:
Redmine version 3.2.0.stable
Ruby version 2.2.4-p230 (2015-12-16) [x86_64-linux]
Rails version 4.2.5
Environment production
Database adapter Mysql2
SCM:
Subversion 1.7.14
Git 1.8.3.1
Filesystem
Redmine plugins:
redmine_export_with_journals 0.0.8
redmine_importer 1.2.2
redmine_local_avatars 1.0.0
redmine_xlsx_format_issue_exporter 0.1.2
sidebar_hide 0.0.7
Here is assinable_versions source-code
# Versions that the issue can be assigned to
def assignable_versions
return #assignable_versions if #assignable_versions
versions = project.shared_versions.open.to_a #Here is line823
if fixed_version
if fixed_version_id_changed?
# nothing to do
elsif project_id_changed?
if project.shared_versions.include?(fixed_version)
versions << fixed_version
end
else
versions << fixed_version
end
end
#assignable_versions = versions.uniq.sort
end
The error is occurring because project is nil and you are trying to call nil.shared_versions. So change the if condition:
if !project.shared_versions.nil?
versions = project.shared_versions.open.to_a
end
to as follows:
versions = project.shared_versions.open.to_a unless project.nil?
This will resolve the 'undefined method' error.
I've solved problem.
There is some difference of REST process.
At least redmine 2.4.4, I can use both project_id = identifier and project_id = project_id for issue create.
But, redmine 3.x, I can only use project_id = project_id for issue create.