Flask with MongoDB using MongoKit to MongoLabs - heroku

I am a beginner and I have a simple application I have developed locally which uses mongodb with mongoKit as follows:
app = Flask(__name__)
app.config.from_object(__name__)
customerDB = MongoKit(app)
customerDB.register([CustomerModel])
then in views I just use the CustomerDB
I have put everything on heroku cloud but my database connection doesn't work.
I got the link I need to connect by:
heroku config | grep MONGOLAB_URI
but I am not sure how to pull this. I looked at the following post, but I am more confused
How can I use the mongolab add-on to Heroku from python?
Any help would be appreciated.
Thanks!

According to the documentation, Flask-MongoKit supports a set of configuration settings.
MONGODB_DATABASE
MONGODB_HOST
MONGODB_PORT
MONGODB_USERNAME
MONGODB_PASSWORD
The MONGOLAB_URI environment setting needs to be parsed to get each of these. We can use this answer to the question you linked to as a starting point.
import os
from urlparse import urlsplit
from flask import Flask
from flask_mongokit import MongoKit
app = Flask(__name__)
# Get the URL from the Heroku setting.
url = os.environ.get('MONGOLAB_URI', 'mongodb://localhost:27017/some_db_name')
# Parse it.
parsed - urlsplit(url)
# The database name comes from the path, minus the leading /.
app.config['MONGODB_DATABASE'] = parsed.path[1:]
if '#' in parsed.netloc:
# If there are authentication details, split the network locality.
auth, server = parsed.netloc.split('#')
# The username and password are in the first part, separated by a :.
app.config['MONGODB_USERNAME'], app.config['MONGODB_PASSWORD'] = auth.split(':')
else:
# Otherwise the whole thing is the host and port.
server = parsed.netloc
# Split whatever version of netloc we have left to get the host and port.
app.config['MONGODB_HOST'], app.config['MONGODB_PORT'] = server.split(':')
customerDB = MongoKit(app)

Related

No environment configuration found. DefaultAzureCredential()

I am trying to use this python sample to authenticate a client with an Azure Service
# pip install azure-identity
from azure.identity import DefaultAzureCredential
# pip install azure-mgmt-compute
from azure.mgmt.compute import ComputeManagementClient
# pip install azure-mgmt-network
from azure.mgmt.network import NetworkManagementClient
# pip install azure-mgmt-resource
from azure.mgmt.resource import ResourceManagementClient
SUBSCRIPTION_ID = creds_obj['SUBSCRIPTION_ID']
# Create client
# For other authentication approaches, please see: https://pypi.org/project/azure-identity/
resource_client = ResourceManagementClient(
credential=DefaultAzureCredential(),
subscription_id=SUBSCRIPTION_ID
)
network_client = NetworkManagementClient(
credential=DefaultAzureCredential(),
subscription_id=SUBSCRIPTION_ID
)
compute_client = ComputeManagementClient(
credential=DefaultAzureCredential(),
subscription_id=SUBSCRIPTION_ID
)
I keep getting No environment configuration found.
The code sample is directly from the microsoft github: https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/resources/azure-mgmt-resource/azure/mgmt/resource/resources/_resource_management_client.py. Ideally I would like to manage this configuration using environment variables or a config file. Is there any way to do this?
When using Azure Identity client library for Python, DefaultAzureCredential attempts to authenticate via the following mechanisms in this order, stopping when one succeeds:
You could set Environment Variables to fix it.
from azure.identity import DefaultAzureCredential
credential=DefaultAzureCredential()
Or set the properties in config and use ClientSecretCredential to create credential.
from azure.identity import ClientSecretCredential
subscription_id = creds_obj["AZURE_SUBSCRIPTION_ID"]
tenant_id = creds_obj["AZURE_TENANT_ID"]
client_id = creds_obj["AZURE_CLIENT_ID"]
client_secret = creds_obj["AZURE_CLIENT_SECRET"]
credential = ClientSecretCredential(tenant_id=tenant_id, client_id=client_id, client_secret=client_secret)
I was having somewhat similar trouble following this Azure key vault tutorial which brought me here.
The solution I found was overriding the default values in the DefaultAzureCredential() constructor.
https://learn.microsoft.com/en-us/python/api/azure-identity/azure.identity.defaultazurecredential?view=azure-python
For reasons people far smarter than me will be able to explain I found that even though I had credentials from the azure cli it was not using those and instead looking for environment_credentials, which I did not have. So it threw an exception.
Once I set the exclude_environment_credential argument to True it then looked instead for the managed_identity_credentials, again which I did not have.
Eventually when I force excluded all credentials other than those from the cli it worked ok for me
I hope this helps someone. Those with more experience please feel free to edit as you see fit

Websphere deployment - MapModulesToServers

Apologies if this has been asked previously but I have the following code in my Props file.
#
# Properties
#
taskName=MapModulesToServers
row0={module uri server ModuleVersion moduletype moduletypeDisplay} # readonly
row1={"MyApplication" MyApplication.war,WEB-INF/web.xml WebSphere:cell=!{cellName},node=!{nodeName},server=!{webserverName}
+ WebSphere:cell=!{cellName},node=!{nodeName},server=!{httpserverName} 23 moduletype.web "Web Module"}
Now I am trying to get the WAR file mapped to both the WebServer and the HTTPServer, both of which are up and running. At present the process only maps to the Web server and we are forced to map to both manually after deployment, which defeats the aim of auto deployment a little.
Can any tell me what is wrong in the script segment please.
thanks in advance
Kev

How do you use an HTTP/HTTPS proxy with boto3?

On the old boto library is was simple enough to use the proxy, proxy_port, proxy_user and proxy_pass parameters when you open a connection. However, I could not find any equivalent way of programmatically define the proxy parameters on boto3. :(
As of at least version 1.5.79, botocore accepts a proxies argument in the botocore config.
e.g.
import boto3
from botocore.config import Config
boto3.resource('s3', config=Config(proxies={'https': 'foo.bar:3128'}))
boto3 resource
https://boto3.readthedocs.io/en/latest/reference/core/session.html#boto3.session.Session.resource
botocore config
https://botocore.readthedocs.io/en/stable/reference/config.html#botocore.config.Config
If you user proxy server does not have a password
try the following:
import os
os.environ["HTTP_PROXY"] = "http://proxy.com:port"
os.environ["HTTPS_PROXY"] = "https://proxy.com:port"
if you user proxy server has a password
try the following:
import os
os.environ["HTTP_PROXY"] = "http://user:password#proxy.com:port"
os.environ["HTTPS_PROXY"] = "https://user:password#proxy.com:port"
Apart from altering the environment variable, I'll present what I found in the code.
Since boto3 uses botocore, I had a look through the source code:
https://github.com/boto/botocore/blob/66008c874ebfa9ee7530d944d274480347ac3432/botocore/endpoint.py#L265
From this link, we end up at:
def _get_proxies(self, url):
# We could also support getting proxies from a config file,
# but for now proxy support is taken from the environment.
return get_environ_proxies(url)
...which is called by proxies = self._get_proxies(final_endpoint_url) in the EndpointCreator class.
Long story short, if you're using python2 it will use the getproxies method from urllib2 and if you're using python3, it will use urllib3.
get_environ_proxies is expecting a dict containing {'http:' 'url'} (and I'm guessing https too).
You could always patch the code, but that is poor practice.
This is one of the rare occasions when I would recommend monkey-patching, at least until the Boto developers allow connection-specific proxy settings:
import botocore.endpoint
def _get_proxies(self, url):
return {'http': 'http://someproxy:1234/', 'https': 'https://someproxy:1234/'}
botocore.endpoint.EndpointCreator._get_proxies = _get_proxies
import boto3

It's dangerous not recognizing URL Root

I'm using Flask's It's Dangerous to generate encrypted URLs. It's awesome, but I've encountered an issue - I'm running a script daily on Heroku Scheduler. The script lives outside of the app folder, in the directory where the run.py and Procfile are. I create the payload using three variables, the last one being the function assigned to a URL.
payload = reactivate_account_link(candidate.candidate_id, candidate.email, 'reactivate_account')
reactivate_account_link is the following...
def reactivate_account_link(candidate_id, candidate_email, path):
s = get_serializer()
loads = [candidate_id, candidate_email]
payload = s.dumps(loads)
return url_for(path, payload=payload, _external=True)
The problem is that the URL doesn't have the proper root. It creates...
http://localhost/candidates/reactivate_account/WzYsInN1cmFqa2FwQGdtYWlsLmNvbSJd.A484cnO8rRcAqe2M2mNrfoGludo/
as opposed to
http://[--DOMAIN--]/candidates/reactivate_account/WzYsInN1cmFqa2FwQGdtYWlsLmNvbSJd.A484cnO8rRcAqe2M2mNrfoGludo/
This is true for both local and prod. I want to avoid hardcoding the URL.
You need to configure the SERVER_NAME for the app, otherwise you get the default of 'localhost'. Wherever you create and configure your app, add:
SERVER_NAME = 'myapp.heroku.com' # or whatever the external url should be

gevent-socketio + Flask + Gunicorn

Can I use gevent-socketio with Flask, running under Gunicorn, and still enjoy the nice exception printing, debugger, and reload capability that Flask offers? How would my gunicorn worker and WSGI app class look like?
I've got the exact same problem so I solved it by using watchdog.
pip install watchdog
together with this command:
watchmedo shell-command --patterns="*.py*;;*.less;*.css;*.js;*.txt;*.html" --recursive --command='kill -HUP `cat /tmp/gunicorn.pid` && echo "Reloading code" >> /tmp/gunicorn.log' ~/projectfolder
It requires (well, not really, but I point "Reloading code" into the same logfile so It's a nice thing to have) that you daemonize the gunicorn process, which I do like this:
gunicorn_config.py
workers = 2
worker_class = 'socketio.sgunicorn.GeventSocketIOWorker'
bind = '0.0.0.0:5000'
pidfile = '/tmp/gunicorn.pid'
debug = True
loglevel = 'debug'
errorlog = '/tmp/gunicorn.log'
daemon = True
Start the application:
gunicorn run:app -c gunicorn-config.py
View the log:
tail -f /tmp/gunicorn.log
From this point everything should be reloaded with each change in your project.
It's a bit complicated but since gunicorn with a worker (or the built in socketio-server) doesn't have any reloading capabilities I had to do it like this.
It's a different approach compared to the decorator solution in the other answer but I like to keep the actual code clean from development specific solutions. Both accomplish the same thing so I guess you'll just have to pick the solution you like. :)
Oh, as an added bonus you get to use the production server in development which means both environments match each other.
I've been looking into this subject lately. I don't think you can easily use autoreload feature with Flask + gevent-socket.io + Gunicorn. Gunicorn is a production server that does not allow such features natively.
However, I found a nice solution for my development server : user SocketIOServer provided with the library and a Flask snippet for autoreload. Here is the startup script (runserver.py) :
from myapp import app
from gevent import monkey
from socketio.server import SocketIOServer
import werkzeug.serving
# necessary for autoreload (at least)
monkey.patch_all()
PORT = 5000
#werkzeug.serving.run_with_reloader
def runServer():
print 'Listening on %s...' % PORT
ws = SocketIOServer(('0.0.0.0', PORT), app, resource="socket.io", policy_server=False)
ws.serve_forever()
runServer()
This solution is inspired from : http://flask.pocoo.org/snippets/34/
I've made some tweaks to Werkzeug debugger so it now works with socket.io namespaces, see below and enjoy :)
https://github.com/aldanor/SocketIO-Flask-Debug

Resources