I'm trying to use stackless python for a heroku app, but they don't support the runtime:
! Requested runtime (stackless-3.3.5) was not found.
This is surprising, since stackless-3.3.5 was recently added to the list of "unofficial" releases that are available on the heroku buildpack.
heroku python versions
Related
I'm trying to use a specific version of pdftotext (version 3.03) on Heroku. I'm using https://github.com/carwow/heroku-buildpack-pdftotext buildpack which by default uses version 0.57 (from what I'm seeing it's including Poppler version 0.57, but pdftotext version is 0.57 as well). I'm assuming, from looking at https://github.com/carwow/heroku-buildpack-pdftotext/blob/master/scripts/build_pdftotext, that this buildpack allows to use specific version of Poppler by providing either POPPLER_VERSION or POPPLER_TARBALL_URL variables. The question is - how can I pass these variables to the build step?
What I've tried so far:
setting POPPLER_VERSION and POPPLER_TARBALL_URL in Config Vars, deploying
per https://devcenter.heroku.com/articles/buildpack-api, (I assume) creating env folder within my application, setting ENV_DIR to env, creating POPPLER_VERSION and POPPLER_TARBALL_URL files containing appropriate values within env folder, deploying
In either case, when running /app/bin/pdftotext version 0.57 is still being reported
In case that anybody runs into this question - the answer is that I'm dumb and I expected Heroku to build binary using provided Makefile. But why should Heroku build it?
As such, I've forked the https://github.com/carwow/heroku-buildpack-pdftotext to https://github.com/eithed/heroku-buildpack-pdftotext, updated the script to use cmake, built it, switched Heroku to use my buildpack - version 20.10.0 is correctly installed.
We use Rails for the majority of our web projects and in a lot of cases we share code we have as dependancies such as our API as Gems. We also have some projects which are similar in nature enough that we've built Engines and then have host applications that use these.
These are all hosted on our own servers and we share the Gems with the projects using Gem In A Box (this is because we can't upload the Gems to anywhere else due to security restrictions). During development we comment out the lines to reference either a specific version on our server or our local version if we are developing within the Gem itself.
So for example:
# version on the server
gem 'our_gem', '1.1.0', source: 'https://gems.domain.net'
# local version
gem 'our_gem', path: '../our_gem'
What happens is that when we do a build on our CI server we then have a script that reads through the Gemfile and finds any gems that have our server as the source and then vendorize them and reference them as the vendored path.
For example:
# this
gem 'our_gem', '1.1.0', source: 'https://gems.domain.net'
# becomes this
gem 'our_gem', path: 'vendor/gems/our_gem-1.1.0'
However the problem we have encountered is that when we are in development and we may have several features or bugs we want to push to our QA team to start testing it means we have to bump the version each time so we can reference it in our Gemfile's as a dependancy. We can't reference the branch due to the Git repository not accessible to our build setup and some other restrictions.
How do teams usually handle in-development dependancies that don't have a version yet? We've thought about having a development version of Gem In A Box and then not passing a version to it in our Gemfile...
# shared dev version of gem on our dev gem server
gem 'our_gem', source: 'https://dev.gems.domain.net'
But the problem here is that we still need to up the version every time in order to push to the Gem Server. Is there a standard way to push a package to a Package Manager whereby the version is always the same (or better yet is version-less but hosted by the manager)?
We also have the same issue with NPM (we run a local version of Vedaccio).
I'd like to deploy an app with a Procfile that runs
web: node myapp.js
Where myapp.js uses phantomjs-node to run headless webkit stuff, returning the results to browser requests. Is this possible?
I also ran into the same problem, the way I fixed it was by using this "Multiple Buildpack" Buildpack. Then in my .buildpacks file I put the following:
http://github.com/heroku/heroku-buildpack-nodejs.git
http://github.com/stomita/heroku-buildpack-phantomjs.git
Finally, you want to add PhantomJS to the path
heroku config:set PATH=$PATH:vendor/phantomjs/bin
I hope this helps.
Heroku Toolbelt now has first class support for multiple buildpacks, so you can get a working Node and PhantomJS setup with the following:
heroku buildpacks:set https://github.com/heroku/heroku-buildpack-nodejs.git
heroku buildpacks:add --index 1 https://github.com/stomita/heroku-buildpack-phantomjs.git
Here's a PhantomJS buildpack: https://github.com/stomita/heroku-buildpack-phantomjs
I also am able to run the x86_64 build of PhamtomJS, just stuck in my app's vendor directory:
http://phantomjs.org/download.html
Checkout my modified version of stomita that includes NodeJS, PhantomJS & CasperJS ready to run.
https://github.com/olragon/heroku-buildpack-nodejs
This is an old thread but for anybody that lands here I have created a working buildpack for Node and Phantom that's a fork of the official Heroku Node buildpack which incorporates the build scripts from Beedesk's custom Phantom buildpack. Here it is https://github.com/datamail/heroku-buildpack-nodejs-phantomjs
I have forked #stomita buildpack to use the official linux build binary from phantomJS.org. I tested it and it is working with heroku.
Hope some security paranoids like me out there will find it useful.
https://github.com/beedesk/heroku-buildpack-phantomjs
The disadvantage of my pack it that, unlike stomita's, it doesn't include fontconfig and freetype. But, PhantomJS should work without them.
I got Numpy and Matplotlib running on Heroku, and I'm trying to install Scipy as well. However, Scipy requires BLAS[1] to install, which is not presented on the Heroku platform. After contacting Heroku support, they suggested me to build BLAS as a static library to deploy, and setup the necessary environment variables.
So, I compiled libblas.a on a 64bit Linux box, and set the following variables as described in [2] :
$ heroku config
BLAS => .heroku/vendor/lib/libfblas.a
LD_LIBRARY_PATH => .heroku/vendor/lib
LIBRARY_PATH => .heroku/vendor/lib
PATH => bin:/usr/local/bin:/usr/bin:/bin
PYTHONUNBUFFERED => true
After adding scipy==0.10.1 in my requirements.txt, the push still fails.
File "scipy/integrate/setup.py", line 10, in configuration
blas_opt = get_info('blas_opt',notfound_action=2)
File "/tmp/build_h5l5y31i49e8/lib/python2.7/site-packages/numpy/distutils/system_info.py", line 311, in get_info
return cl().get_info(notfound_action)
File "/tmp/build_h5l5y31i49e8/lib/python2.7/site-packages/numpy/distutils/system_info.py", line 462, in get_info
raise self.notfounderror(self.notfounderror.__doc__)
numpy.distutils.system_info.BlasNotFoundError:
Blas (http://www.netlib.org/blas/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [blas]) or by setting
the BLAS environment variable.
It seem that pip is not aware of the BLAS environment variable, so I check the environment using heroku run python:
(venv)bash-3.2$ heroku run python
Running python attached to terminal... import up, run.1
Python 2.7.2 (default, Oct 31 2011, 16:22:04)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.system('bash')
~ $ echo $BLAS
.heroku/vendor/lib/libfblas.a
~ $ ls .heroku/vendor/lib/libfblas.a
.heroku/vendor/lib/libfblas.a
~ $
And it seems fine. Now I have no idea how to solve this.
[1] http://www.netlib.org/blas/
[2] http://www.scipy.org/Installing_SciPy/Linux
I managed to get this working on the cedar stack by building numpy and scipy offline as bdists and then modifying the heroku python buildpack to unzip these onto the dyno's vendor/venv areas directly. You can also use the buildpack to set environment variables that persist.
Heroku haven't officially published buildpacks yet - search for 'heroku buildpacks' for more thirdparty/heroku ones and information.
My fork of the python build pack is here:
https://wyn#github.com/wyn/heroku-buildpack-python.git
The changes are in the bin/compile where I source two new steps, a scipy/numpy step and an openopt step. The scripts for the two steps are in bin/steps/npscipy and bin/steps/openopt. I also added some variables to bin/release. Note that I am assuming installation through a setup.py file rather than the requirements.txt approach (c.f. https://devcenter.heroku.com/articles/python-pip#traditional_distributions).
I also download the blas/lapack/atlas/gfortran binaries that were used to build numpy/scipy onto the dyno as there are c extensions that need to link through to them. The reason for building everything offline and downloading is that pip-installing numpy/scipy requires you to have a fortran compiler + associated dev environment and this made my slugs too big.
It seems to work, the slug size is now 35mb and scaling seems fast too. All but one of the numpy tests pass and all of the scipy tests pass.
This is still work in progress for me but I thought I'd share.
In case someone else comes to this like I did...
The excellent answer on this question from #coshx sadly no longer works (at least I could not get it to work). What I did, however was the following:
I forked npsicpy-binaries repository from #coshx and changed all the tar files such that they do not have venv as the root folder inside (my fork is here)
I forked the npsp-helloworld repository from #coshx and made it use a requirements.txt file instead of the setup.py (my fork is here - this means that you can use the whole pip approach).
I forked the heroku-buildpack-python repository from Heroku, took the npscipy file from #coshx and changed it to work with this latest version of the build pack (my fork is here - you can see that there is no venv set up, for example).
After doing those three things I have the npsp-helloworld application working perfectly. You just need to make sure you set the buildpack appropriately when creating the application and you are good to go:
$ heroku create --stack=cedar --buildpack=https://github.com/kmp1/heroku-buildpack-python.git
NOTE: I haven't worked out how to make my own binaries yet, so the three libraries (scipy, numpy and scikit-learn) are not the latest versions so make sure when you install it you do (if anyone can build these I would gladly accept a pull request for them):
pip install scipy==0.11.0
pip install numpy==1.7.0
pip install scikit-learn==0.13.1
By the way - I am really sorry if I have not done things in the correct way, etiquette-wise. I'm still learning git and this whole open source thing.
Another good option is the conda buildpack, which allows you to add any of the free Linux64 packages available through Anaconda/Miniconda to a Heroku app. Some of the most popular packages include numpy, scipy, scikit-learn, statsmodels, pandas, and cvxopt. While the buildpack makes it fairly simple to add packages to an app, the downsides are that the buildback takes up a lot of space and that you have to wait on Anaconda to update the libraries in the repository.
If you are starting a new Python app on Heroku, you can add the conda buildpack using the command:
$ heroku create YOUR_APP_NAME --buildpack https://github.com/kennethreitz/conda-buildpack.git
If you have already setup a Python app on Heroku, you can add the conda buildpack to the existing app using the command:
$ heroku config:add BUILDPACK_URL=https://github.com/kennethreitz/conda-buildpack.git
Or, if you need to specify the app by name:
$ heroku config:add BUILDPACK_URL=https://github.com/kennethreitz/conda-buildpack.git --app YOUR_APP_NAME
To use the buildpack, you will need to include two text files in the app directory, requirements.txt and conda-requirements.txt. Just as with the standard Python buildpack, the requirements.txt file lists packages that should be installed using pip. Packages that should be installed using conda are listed in the conda-requirements.txt file. Some of the most useful scientific packages include numpy, scipy, scikit-learn, statsmodels, pandas, and cvxopt. The full list of available conda packages can be found at repo.continuum.io.
For example:
$ cat requirements.txt
gunicorn==0.14.2
requests==0.11.1
$ cat conda-requirements.txt
scipy
numpy
cvxopt
That’s it! You can now add Anaconda packages to a Python app on Heroku.
The slug compiler is not aware of your environment variables, which is why it fails during push, and not once running.
The only real option you have is to look at the user_env_compile addon that's currently in labs beta.
http://devcenter.heroku.com/articles/labs-user-env-compile
For those who wish to use Python 3.4 in production, I've built numpy 1.8.1, scipy 0.14.0, and scikit-learn 0.15-git (0.14 doesn't work with the others for some reason) as binaries on Ubuntu 10.04 LTS 64-bit, which works on the Heroku cedar stack. Here is the git repo.
My heroku buildpack is quite similar to that of kmp. Note that the bin/steps/npscipy file links to my repository of binaries above.
I'm putting this here in case someone stumbled on this like I do. Since I am using python 3.4, thenovices buildpack doesn't work out for me.
I tried to look at conda buildpack, but it seems like a little bit of an overkill for my application which only requires scipy and numpy. What finally works for me is this excellent guide, using a multi buildpacks approach. The scipy installation was indeed a bit long though. Hope this helps! :)
So, I got a small site started in node.js (my first one) using Express. Pretty happy with it, until I tried to deploy to Heroku and found that I had 0.4.9 installed and they only support 0.4.7.
Is uninstalling 0.4.9 and installing 0.4.7 my only option, or is there a way to do a side-by-side on the two?
You can override the version of node.js and npm by customizing the Heroku build pack:
http://blog.superpat.com/2011/11/15/running-your-own-node-js-version-on-heroku/
Actually...you do not have to remove anything.
Just ensure you are using features of node compliant with node 0.4.7 and when you make your package.json which specifies your dependencies has the correct version number or range specified.
I had a similar issue where one of our developers made is packacge and set the dependency to node 0.4.8 however it didn't require this it was just what version he was using at the time, we ended up updating his package.json to list node 0.4.7 instead and then my package which depended on his deployed to heroku just fine.
It seems Heroku only supports 0.4.7 at the moment and even suggests to develop strictly on that version.
If you have to use heroku then you have to uninstall 0.4.9, install 0.4.7.
If you don't have to use heroku. You can always setup a VPS yourself, and you will have the freedom to install whatever version that pleases you. :D