Missing GDAL on Heroku - heroku

A Django application running has GeoJSON data stored in one of the models via django-geojson. In the admin interface, this field is editable via django-leaflet. When I run the application locally with ./manage.py runserver, the interface works just fine, the page looks like this.
However, when I deploy to Heroku, which builds just fine, and open the same Location instance in the admin interface, the blue marker isn't anywhere on the map, the map loads normally, and I get this error delivered to me:
Error creating geometry from value '{
"coordinates":[
"-105.2449000",
"40.0474000"
],
"type":"Point"
}' (Initializing geometry from JSON input requires GDAL.)
I'm just recording simple Point data in my models.py
from djgeojson.fields import PointField
from django.db import models
class Location(models.Model):
"""
A model subclass for recording geographic data.
"""
service_id = models.CharField(max_length=255, blank=True, null=True)
name = models.CharField(max_length=255, blank=True, null=True)
geom = PointField() # GeoJSON (remember, coordinates: [long, lat])
def __str__(self):
return self.name
This is my first time using geoJson, Leaflet, and Heroku. Am I missing some dependency, is there's some Heroku configuration I'm neglecting? Here's my requirements.txt.
dj-database-url==0.4.1
Django==1.10
django-filter==0.13.0
django-geojson==2.9.0
django-leaflet==0.18.1
gunicorn==19.6.0
httplib2==0.9.2
jsonfield==1.0.3
oauth2==1.9.0.post1
oauthlib==1.1.2
psycopg2==2.6.2
PyJWT==1.4.1
python-openid==2.2.5
requests==2.10.0
requests-oauthlib==0.6.2
six==1.10.0
wheel==0.24.0
whitenoise==3.2

Heroku is currently running the heroku-18 stack and for python projects the default build pack has the geospatial libraries https://github.com/heroku/heroku-buildpack-python
To use them run heroku config:set BUILD_WITH_GEO_LIBRARIES=1 and add
GDAL_LIBRARY_PATH = os.environ.get('GDAL_LIBRARY_PATH')
GEOS_LIBRARY_PATH = os.environ.get('GEOS_LIBRARY_PATH')
to settings.py.
See also https://github.com/heroku/heroku-buildpack-python/issues/752

Looks like Heroku have dropped support for BUILD_WITH_GEO_LIBRARIES. You now need to add a buildpack to enable support:
# Add the buildpack
heroku buildpacks:add --index 1 https://github.com/heroku/heroku-geo-buildpack.git
# Remove the environment variable
heroku config:unset BUILD_WITH_GEO_LIBRARIES
See Heroku help post

GDAL is not a Python dependency, it is a separate framework that must be installed in your production environment (Heroku).
One way to do this is using a separate buildpack like heroku-geo-buildpack. Heroku has instructions on how to add multiple buildpacks - i.e. here Python and GDAL.
Once installed, the heroku-geo-buildpack suggests the following method for testing GDAL is installed in a Django app. From your Heroku Python console do:
>>> from django.contrib.gis import gdal
>>> gdal.HAS_GDAL
True
You should then be good to go.

Related

How can I call list of ec2 instance based on the app code using tag method

I am trying to get all the instance(server name) ID based on the app. Let's say I have an app in the server. How do I know which apps below to which server. I want my code to find all the instance (server) that belongs to each app. Is there any way to look through the app in the ec2 console and figure out the servers are associated with the app. More of using tag method
import boto3
client = boto3.client('ec2')
my_instance = 'i-xxxxxxxx'
(Disclaimer: I work for AWS Resource Groups)
Seeing your comments that you use tags for all apps, you can use AWS Resource Groups to create a group - the example below assumes you used App:Something as tag, first creates a Resource Group, and then lists all the members of that group.
Using this group, you can for example get automatically a CloudWatch dashboard for those resources, or use this group as a target in RunCommand.
import json
import boto3
RG = boto3.client('resource-groups')
RG.create_group(
Name = 'Something-App-Instances',
Description = 'EC2 Instances for Something App',
ResourceQuery = {
'Type': 'TAG_FILTERS_1_0',
'Query': json.dumps({
'ResourceTypeFilters': ['AWS::EC2::Instance'],
'TagFilters': [{
'Key': 'App',
'Values': ['Something']
}]
})
},
Tags = {
'App': 'Something'
}
)
# List all resources in a group using a paginator
paginator = RG.get_paginator('list_group_resources')
resource_pages = paginator.paginate(GroupName = 'Something-App-Instances')
for page in resource_pages:
for resource in page['ResourceIdentifiers']:
print(resource['ResourceType'] + ': ' + resource['ResourceArn'])
Another option to just get the list without saving it as a group would be to directly use the Resource Groups Tagging API
What you install on an Amazon EC2 instance is totally up to you. You do this by running code on the instance itself. AWS is not involved in the decision of what you install on the instance, nor does it know what you installed on an instance.
Therefore, you will need to keep track of "what apps are installed on what server" yourself.
You might choose to take advantage of Tags on instances to add some metadata, such as the purpose of the server. You could also use AWS Systems Manager to run commands on instances (eg to install software) or even use AWS CodeDeploy to roll-out software to fleets of servers.
However, even with all of these deployment options, AWS cannot track what you have put on each individual server. You will need to do that yourself.
Update: You can use AWS Resource Groups to view/manage resources by tag.
Here's some sample Python code to list tags by instance:
import boto3
ec2_resource = boto3.resource('ec2', region_name='ap-southeast-2')
instances = ec2_resource.instances.all()
for instance in instances:
for tag in instance.tags:
print(instance.instance_id, tag['Key'], tag['Value'])

Terraform Heroku provider: Does it have resource for dyno?

Using terraform, I could create Heroku applications, create and attach add-ons and put the applications in a pipeline. After the infrastructure is created, everything is good except the dynos are not started. I used heroku/nodejs buildpack. Terraform's Heroku provider does not provide any explicit resource type that corresponds to Heroku dyno. Are we supposed to manually push application for deployment on Heroku when the necessary add-ons and pipeline are created with Terraform?
I googled a lot but couldn't figure out what could be the reason for the dynos not getting started after necessary infrastructure is in place.
Please help.
so today I wanted to test heroku with terraform and got the same issue
it looks like you need to push your app to the git_url reference provided by heroku_app
made a working example at https://github.com/nroitero/terraform_heroku
I'm doing as the example below, and it works.
First, defining the heroku app:
resource "heroku_app" "this" {
name = var.HEROKU_APP_NAME
region = var.HEROKU_REGION
space = var.HEROKU_SPACE
internal_routing = var.HEROKU_INTERNAL_ROUTING
Then, indicating where the node application is:
resource "heroku_build" "this" {
app = heroku_app.this.name
#buildpacks = [var.BUILDPACK_URL]
source = {
#url = var.SOURCE_URL
#version = var.SOURCE_VERSION
#testing path instead of source
path = var.SOURCE_PATH
}
}
And to define dynos, I'm using:
resource "heroku_formation" "this" {
app = heroku_app.this.name
type = var.HEROKU_FORMATION_TYPE
quantity = var.HEROKU_FORMATION_QTY
size = var.HEROKU_FORMATION_SIZE
depends_on = [heroku_build.this]
}
For the dyno size parameter (var.HEROKU_FORMATION_SIZE), use the official dyno type "name" as listed on https://devcenter.heroku.com/articles/dyno-types.
For private spaces, names are: private-s, private-m and private-l.

Deploy pure Elixir app on Heroku or other hosting services

I've just made a pure elixir app which generates a random image based on the text user inputs. I do not have any front-end for this and fully rely on terminal to input a text to the app.
I know how to make a Phoenix app using Elixir and deploy it on Heroku or other hosting services. But I wonder if it's possible to deploy a pure Elixir app (not relying on any framework like Phoenix).
Please correct me if I'm wrong for asking this question.
For a CLI application, you can package it as an escript: Mix.Tasks.Escript.Build
This gives you an application with a main/1 entry point.
mix.exs:
defmodule MyApp.Mixfile do
use Mix.Project
def project do
[
app: :my_app,
version: "0.0.1",
escript: escript()
]
end
def escript do
[main_module: MyApp.CLI]
end
end
lib/my_app/cli.ex:
defmodule MyApp.CLI do
def main(_args) do
IO.puts("Hello from MyApp!")
end
end

Google API + proxy + httplib2

I'm currently running a script to pull data from Google Analytics with googleapiclient Python package (that is based on httplib2 client object)
--> My script works perfectly without any proxy.
But I have to put it behind my corporate proxy, so I need to adapt my httplib2.Http() object to embed proxy information.
Following httplib2 doc 1 I tried:
pi = httplib2.proxy_info_from_url('http://user:pwd#someproxy:80')
httplib2.Http(proxy_info=pi).request("http://www.google.com")
But it did not work.
I always get a Time out error, with or without the proxy info (so proxy_info in parameter is not taken into account)
I also downloaded socks in PySocks package (v1.5.6) and tried to "wrapmodule" httplib2 as described in here:
https://github.com/jcgregorio/httplib2/issues/205
socks.setdefaultproxy(socks.PROXY_TYPE_HTTP, "proxyna", port=80, username='p.tisserand', password='Telematics12')
socks.wrapmodule(httplib2)
h = httplib2.Http()
h.request("http://google.com")
But I get an IndexError: (tuple index out of range)
In the meantime,
When I use the requests package, this simple code works perfectly:
os.environ["HTTP_PROXY"] = "http://user:pwd#someproxy:80"
req = requests.get("http://www.google.com")
The problem is that need to fit with googleapiclient requirements and provide a htpplib2.Http() client object.
rather than using Python2, I think you'd better try using httplib2shim
You can have a look at this tutorial on my blog :
https://dinatam.com/fr/python-3-google-api-proxy/
In simple words, just replace this kind of code :
from httplib2 import Http
http_auth = credentials.authorize(Http())
by this one :
import httplib2shim
http_auth = credentials.authorize(httplib2shim.Http())
I decided to recode my web app in Python 2, still using the httplib2 package.
Proxy info are now taken into account. It now works.

Configure "apiKey.properties" file in "shiro.ini" file in Heroku Java Web App

I am trying to build up a Web Application in Heroku with Stormpath User Management addon and Apache Shiro.
When I go through the sample code its mentioned in "shiro.ini" file to provide the "apiKey.properties" file's path in property "stormpathClient.apiKeyFileLocation".
Please suggest how we can configure "apiKey.properties" file's path which contains STORMPATH API KEY ID and SECRET in Heroku application.
In Heroku you can place the Api Key ID and Secret in Environment variables as described here.
So, what you can do:
Create the following class in your application:
package com.stormpath.sample.client;
import java.util.Properties;
public class ApiKeyEnvVariables extends Properties {
public ApiKeyEnvVariables() {
super.put("apiKey.id", System.getenv("STORMPATH_API_KEY_ID"));
super.put("apiKey.secret", System.getenv("STORMPATH_API_KEY_SECRET"));
}
}
Change your shiro.ini to look like this:
apiKeyProps = com.stormpath.sample.client.ApiKeyEnvVariables
#stormpathClient.apiKeyFileLocation = /Users/XXXX/.stormpath/apiKey.properties
stormpathClient.apiKeyProperties = $apiKeyProps
Set STORMPATH_API_KEY_ID and STORMPATH_API_KEY_SECRET environment variables. For example:
heroku config:set STORMPATH_API_KEY_ID=2JQQCIG5E8EKN4DKBT7R151
heroku config:set STORMPATH_API_KEY_ID=1oYULQMkS3dwKkl6wtbNd93IyUrRehCsEJJrIrMwuI0
Now, when your app starts, Stormpath will automatically pick the Api Key Id and Secret from the environment variables.
Hope that helps!

Resources