I have a Flask app running on nginx + uWSGI.
On my local server (non-nginx), I get a nice stack trace + error reporting for exceptions.
Like this:
$ python run.py
Traceback (most recent call last):
File "run.py", line 1, in <module>
from myappname import app
File "/home/me/myappname/myappname/__init__.py", line 27, in <module>
file_handler.setLevel(logging.debug)
File "/usr/lib/python2.7/logging/__init__.py", line 710, in setLevel
self.level = _checkLevel(level)
File "/usr/lib/python2.7/logging/__init__.py", line 190, in _checkLevel
raise TypeError("Level not an integer or a valid string: %r" % level)
On nginx, there is next to no logging whatsoever (in /var/log/nginx/error.log).
This post suggests adding app.logger.exception('Failed') to my script, which didn't help.
How do I enable this sort of logging for debugging purposes?
Nginx will capture your app's console output, but you must make the app recover from exceptions. Else, you'll only get 500 or 400 errors from Nginx.
Try running the app off Nginx until it seems stable.
Use the logging module to capture app status information to your own log file. This strategy will be useful in the long run.
Related
Hey all,
For context: I had this dev_appserver setup working late last year in 2021, and upon trying to set it up again, I'm getting odd errors.
BadArgumentError: app must not be empty.
I've solved quite a lot of errors up to this point, and this is where I'm at:
JDK 1.11+ installed (for Cloud Datastore Emulator)
Golang 1.15+ installed (for gops & dev_appserver.py - go build)
Gcloud Components:
I run my dev_appserver like this:
export DATASTORE_DATASET=dev8celbux
export DATASTORE_PROJECT_ID=dev8celbux
export DATASTORE_USE_PROJECT_ID_AS_APP_ID=true
dev_appserver.py --enable_console --admin_port=8001 --port=8081 --go_debugging=true --support_datastore_emulator=true --datastore_path=./datastore/local_db.bin setuptables-app.yaml
INFO 2022-09-09 13:26:30,233 devappserver2.py:317] Skipping SDK update check.
INFO 2022-09-09 13:26:30,250 datastore_emulator.py:156] Starting Cloud Datastore emulator at: http://localhost:58946
INFO 2022-09-09 13:26:32,381 datastore_emulator.py:162] Cloud Datastore emulator responded after 2.131000 seconds
INFO 2022-09-09 13:26:32,381 <string>:384] Starting API server at: http://localhost:59078
INFO 2022-09-09 13:26:32,384 <string>:374] Starting gRPC API server at: http://localhost:59079
INFO 2022-09-09 13:26:32,394 instance_factory.py:184] Building with dependencies from go.mod.
INFO 2022-09-09 13:26:32,397 dispatcher.py:280] Starting module "setuptables" running at: http://localhost:8081
INFO 2022-09-09 13:26:32,397 admin_server.py:70] Starting admin server at: http://localhost:8001
WARNING 2022-09-09 13:26:32,398 devappserver2.py:414] No default module found. Ignoring.
2022/09/09 13:26:35 STARTING
INFO 2022-09-09 13:26:37,220 instance.py:294] Instance PID: 9656
This error appears when I try & view the contents within the local datastore at localhost:8001/datastore.
Traceback (most recent call last):
File "C:\Users\user\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\lib\webapp2\webapp2\__init__.py", line 1526, in __call__
rv = self.handle_exception(request, response, e)
File "C:\Users\user\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\lib\webapp2\webapp2\__init__.py", line 1520, in __call__
rv = self.router.dispatch(request, response)
File "C:\Users\user\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\lib\webapp2\webapp2\__init__.py", line 1270, in default_dispatcher
return route.handler_adapter(request, response)
File "C:\Users\user\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\lib\webapp2\webapp2\__init__.py", line 1094, in __call__
return handler.dispatch()
File "C:\Users\user\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\google\appengine\tools\devappserver2\admin\admin_request_handler.py", line 88, in dispatch
super(AdminRequestHandler, self).dispatch()
File "C:\Users\user\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\lib\webapp2\webapp2\__init__.py", line 588, in dispatch
return self.handle_exception(e, self.app.debug)
File "C:\Users\user\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\lib\webapp2\webapp2\__init__.py", line 586, in dispatch
return method(*args, **kwargs)
File "C:\Users\user\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\google\appengine\tools\devappserver2\admin\datastore_viewer.py", line 661, in get
kinds = self._get_kinds(namespace)
File "C:\Users\user\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\google\appengine\tools\devappserver2\admin\datastore_viewer.py", line 597, in _get_kinds
return sorted([x.kind_name for x in q.run()])
File "C:\Users\user\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\google\appengine\ext\db\__init__.py", line 2077, in run
raw_query = self._get_query()
File "C:\Users\user\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\google\appengine\ext\db\__init__.py", line 2482, in _get_query
_app=self._app)
File "C:\Users\user\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\google\appengine\api\datastore.py", line 1371, in __init__
self.__app = datastore_types.ResolveAppId(_app)
File "C:\Users\user\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\google\appengine\api\datastore_types.py", line 238, in ResolveAppId
ValidateString(app, 'app', datastore_errors.BadArgumentError)
File "C:\Users\user\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\google\appengine\api\datastore_types.py", line 186, in ValidateString
raise exception('%s must not be empty.' % name)
BadArgumentError: app must not be empty.
I figured out that it is somewhat related to the APPLICATION_ID environment variable being missing. Upon setting it, I am able to view the Database page. HOWEVER. When getting no errors writing my data to the emulator (line by line debugged to confirm & local_db.bin is created), upon looking at the data, nothing is there. I successfully write 15 entities from the code's point of view. However none appear on the admin page. I think it's due to the manual set of the APPLICATION_ID as I did not do this before. Perhaps should be automatic somehow. Was thinking that this environment variable could maybe do that: export DATASTORE_USE_PROJECT_ID_AS_APP_ID=true but doesn't seem to change anything.
Before calling creation of entities:
After calling creation of entities:
I write the data like this, no doubt this works correctly.
ctx, err := appengine.Namespace(appengine.BackgroundContext(), "celbux101")
...
userKeyOut, err := datastore.Put(ctx, userKey, &F1_4{...})
Also, looked in both the default & the designated namespace (celbux101):
Super stumped. :( Help appreciated!
I really think it may somehow be related to APPLICATION_ID
Yes!
... I managed to come to a solution! As suspected, the data was getting written correctly, as confirmed by the line-by-line debug & the creation of the local_db.bin. The issue is that the dev_appserver's UI is not able to show the database entities due to the incorrect or missing APPLICATION_ID, as deducted.
I figured out that the dev_appserver's UI uses both the APPLICATION_ID & namespace to determine where to look for your entities. Also, the dev_appserver has it's own default APPLICATION_ID.
Solution
The fix is to export this environment variable BEFORE running your dev_appserver.py.
export APPLICATION_ID=dev~None
This magic export allows everything work as expected. You can view the APPLICATION_ID that the UI is trying to use on the top-left of the interface.
EDIT: I just came back to running this on a new computer, and want to add this for future reference:
If you are getting IOError: emulator did not respond within 10s
INSTALL Python27 & add to your path! (alongside your bundled python)
From Google's documentation
export DATASTORE_DATASET=my-project-id
export DATASTORE_EMULATOR_HOST=::1:8432
export DATASTORE_EMULATOR_HOST_PATH=::1:8432/datastore
export DATASTORE_HOST=http://::1:8432
export DATASTORE_PROJECT_ID=my-project-id
This will display your project name (instead of dev~None)
At the moment I'm trying to create a cluster in aws ec2 with Graphlab Create. The code is as follows:
import graphlab as gl
ec2config = gl.deploy.Ec2Config(region='us-west-2', instance_type='m3.large',
aws_access_key_id='secret-acces-key-id',
aws_secret_access_key='secret-access-key')
ec2 = gl.deploy.ec2_cluster.create(name='Test Cluster',
s3_path='s3://test-big-data-2016', ec2_config=ec2config, idle_shutdown_timeout=3600, num_hosts=1)
When the above code is executed I get the following error:
Traceback (most recent call last):
File "test.py", line 59, in
ec2 = gl.deploy.ec2_cluster.create(name='Test Cluster', s3_path='s3://test-big-data-2016', ec2_config=ec2config, idle_shutdown_timeout=36000, num_hosts=1)
File "/Users/remco/anaconda/envs/gl-env/lib/python2.7/site-packages/graphlab/deploy/ec2_cluster.py", line 83, in create
cluster.start()
File "/Users/remco/anaconda/envs/gl-env/lib/python2.7/site-packages/graphlab/deploy/ec2_cluster.py", line 233, in start
self.idle_shutdown_timeout
File "/Users/remco/anaconda/envs/gl-env/lib/python2.7/site-packages/graphlab/deploy/_executionenvironment.py", line 372, in _start_commander_host
raise RuntimeError('Unable to start host(s). Please terminate '
RuntimeError: Unable to start host(s). Please terminate manually from the AWS console.
When I look in EC2 Management Console a new instance is launched and running. But still getting the error in the terminal.
I really don't know what I'm doing wrong here. I followed the exact instructions from: https://turi.com/learn/userguide/deployment/pipeline-example.html
In python, I am using the following:
context = zmq.Context()
socket = context.socket(zmq.PUSH)
socket.bind_to_random_port('tcp://*', min_port=6001, max_port=6004, max_tries=100)
port_selected = socket.???????
How do I know what port is chosen? I will have a look up table in redis for the workers to read.
I am using a push pull model. I need to let workers know what ports to connect to.
I have to do this because I am using the gevent loop in uwsgi and specifying a a plain blind thows and error becuase of a fork. If a use bind_to_random_port then a port is seleced, I just dont know which.
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/gevent-1.0b2-py2.7-linux-x86_64.egg/gevent/greenlet.py",
line 328, in run
result = self._run(*self.args, **self.kwargs)
File "/home/ubuntu/workspace/rtbopsConfig/rtbServers/rtbUwsgiPixelServer/uwsgiPixelServer.py",
line 43, in sendthis
socket.send(push)
File "/usr/local/lib/python2.7/dist-packages/zmq/green/core.py",
line 173, in send
self._wait_write()
File "/usr/local/lib/python2.7/dist-packages/zmq/green/core.py",
line 108, in _wait_write
assert self.__writable.ready(), "Only one greenlet can be waiting
on this event"
AssertionError: Only one greenlet can be waiting on this event
<Greenlet at 0x2d41370: sendthis('2012-07-02 04:05:15')> failed with
AssertionError
port_selected = socket.bind_to_random_port('tcp://*', min_port=6001, max_port=6004, max_tries=100)
I'm starting out to learn about AMQP and RabbitMQ.
To get myself going I have used a CLI tool, rabbitmqadmin, to successfully publish data to a RabbitMQ development install I have created upon my Mac OS X box. So far so good, I can publish messages, and watch them dequeue...
However when I come to try the exact same functionality upon the Heroku / CloudAMQP instance the rabbitmqadmin client seems to fall over.
This is the call:
rabbitmqadmin --host lemur.cloudamqp.com --vhost app4444444_heroku.com --user app4444444_heroku.com --password <withheld> publish routing_key=test payload="hello"
...and this is the output:
Traceback (most recent call last):
File "/usr/local/bin/rabbitmqadmin", line 828, in <module>
main()
File "/usr/local/bin/rabbitmqadmin", line 325, in main
method()
File "/usr/local/bin/rabbitmqadmin", line 428, in invoke_get
result = self.post(uri, json.dumps(upload))
File "/usr/local/bin/rabbitmqadmin", line 354, in post
return self.http("POST", path, body)
File "/usr/local/bin/rabbitmqadmin", line 377, in http
resp = conn.getresponse()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1013, in getresponse
response.begin()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 402, in begin
version, status, reason = self._read_status()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 366, in _read_status
raise BadStatusLine(line)
httplib.BadStatusLine: ''
Any thoughts or ideas gratefully received!
Add --ssl to the command line. CloudAMQP's web ui is https only.
Python 2.6.2 / Django 1.0.2 on Win 2003 Standard IIS 6.
IIs is running two django sites with the following directory structure:
root/http/WEBSITE1
root/http/WEBSITE2
Both sites have their own application pools; each site has it's own pyISAPIe.dll instance.
WEBSITE1 is accessible (serving test django page). However, the browser produces the following for WEBSITE2:
Internal Server Error
An error occurred processing this request.
Request handler failed
Traceback (most recent call last):
File "D:\Python26\lib\site-packages\Http\Isapi.py", line 67, in Request
return RunWSGI(Handler, Base=Base)
File "D:\Python26\lib\site-packages\Http\WSGI.py", line 155, in RunWSGI
Result = Application(Environ, StartResponse)
File "D:\Python26\lib\site-packages\django\core\handlers\wsgi.py", line 241, in call
response = self.get_response(request)
File "D:\Python26\lib\site-packages\django\core\handlers\base.py", line 122, in get_response
return self.handle_uncaught_exception(request, resolver, sys.exc_info())
File "D:\Python26\lib\site-packages\django\core\handlers\base.py", line 166, in handle_uncaught_exception
return callback(request, **param_dict)
File "D:\Python26\lib\site-packages\django\views\defaults.py", line 23, in server_error
t = loader.get_template(template_name) # You need to create a 500.html template.
File "D:\Python26\lib\site-packages\django\template\loader.py", line 81, in get_template
source, origin = find_template_source(template_name)
File "D:\Python26\lib\site-packages\django\template\loader.py", line 74, in find_template_source
raise TemplateDoesNotExist, name
django.template.TemplateDoesNotExist: 500.html
It seems to me that "Isapi.py" located in root/pytho26/Lib/site-packages/Http has everything to do with this because of the "os.environ["DJANGO_SETTINGS_MODULE"] = "WEBSITE1.settings"
How can I modify this file or position it so that both sites run independently.
For anyone interested, the solution lies here
http://groups.google.com/group/pyisapie/browse_thread/thread/b67e861c85e1b26a
Follow option #3 to the letter and you will be able to run multiple IIS sites.