I'm trying to install the texlive-full package on heroku through making a custom buildpack. I'm working on django application, so I'm currently using version of heroku-buildpack-python (https://github.com/heroku/heroku-buildpack-python).
According to this tutorial:
https://devcenter.heroku.com/articles/buildpack-binaries
I found a binary package of texlive on http://packages.ubuntu.com/lucid/texlive-binaries (Links in the right column - Download Source Package) and I added these lines of code in python default heroku buildpack in the section #Build time in order to extract and install texlive-full on heroku.
# Build Time
# Switch to the repo's context.
cd $BUILD_DIR
TEXLIVE_BINARY="http://archive.ubuntu.com/ubuntu/pool/main/t/texlive-base/texlive-base_2012.20120611-5.debian.tar.gz"
TEXLIVE_VENDOR="vendor/texlive"
# vendor awesome-vm
mkdir -p $1/$TEXLIVE_VENDOR
curl $TEXLIVE_BINARY -o - | tar -xz -C $1/$TEXLIVE_VENDOR -f -
After pushing the django application to heroku I can see that slug is 58.0 MB big (before it was just 10.0 MB), so it might have added texlive-full binary package to it (which is about 44 MB). However the latex equation on the site is still not showing and the same error appears; (That appears when texlive is not installed)
ValueError at / latex returned code 32512 for formula:
Is there some easier way to install texlive-full on heroku? Or what is the correct notation for installing texlive-full in buildpack or any other kind of debian package?
Based on this buildpack I built my own version: https://github.com/syphar/heroku-buildpack-tex
It installs a small version of TeX-Live 2013 in your slug, and you can extend it by adding your own packages (collections or single packages from CTAN).
Since your (compressed) slug-size is limited to 300 MB on Heroku you can't texlive-full inside your application.
This buildpack worked for me:
https://github.com/holiture/heroku-buildpack-tex
That error code tends to be returned because of problems with your path (https://tex.stackexchange.com/questions/21692/latex-compilation-failure-on-mac-os-x-from-python-script) Follow the last step of the tutorial here https://devcenter.heroku.com/articles/buildpack-binaries and add the directory that your build pack works on to your path.
Related
Hi we are seeing wrong go version is downloaded when i push my binary file to cf
Steps followed to push code
run go build command
GOOS="linux" GOARCH=amd64 go build ${LDFLAGS} -o localdeploy/some-app main.go
cd localdeploy
cf push -f manifest.yml
Note: localdeploy folder contains manifest.yml and some-app binary file
Go.mod file
go 1.16
require (
github.com/cloudfoundry-community/go-cfenv v1.18.0
github.com/gin-gonic/gin v1.8.1
github.com/google/uuid v1.3.0
github.com/rs/zerolog v1.28.0
github.com/stretchr/testify v1.8.0
)
manifest file.yml
applications:
- name: some-app-1000-snapshot
command: ./some-app
stack: cflinuxfs3
buildpacks:
- https://github.com/cloudfoundry/binary-buildpack.git
Then i see following logs downloading go 1.15.5 instead of go 1.16
Below are the logs getting when pushed to cf --- application is working but why it is downloading 1.15 instead of 1.16 which is present in my mod file
Staging app and tracing logs...
-----> Download go 1.15.5
-----> Running go build supply
/tmp/buildpackdownloads/d612ac0e3047b21e80ecfeae72c39f81 ~
~
-----> Binary Buildpack version 1.0.46
-----> Download go 1.15.5
-----> Running go build finalize
/tmp/buildpackdownloads/d612ac0e3047b21e80ecfeae72c39f81 ~```
The binary buildpack doesn't install anything for your application. It is effectively a no-op buildpack.
Since you have compiled your application locally, the version of Go that is used for your binary is the version installed locally on your computer. You control that version based on what you have installed locally.
The output of the binary buildpack is confusing here, because you do see it downloading an older Go version. The reason this happens is because the buildpack itself is written in Go and you have the buildpack definition in your manifest.yml pointing to the source code of the buildpack. Thus to run the buildpack, it has to compile itself first. It needs Go to do that, so it downloads Go, builds itself, then runs itself. That's what you're seeing where it says Download go 1.15.5.
Most (all?) CloudFoundry installations are going to have the binary buildpack by default, so you don't need to reference the source. Run cf buildpacks and get the name of the binary buildpack from the list. It'll be something like binary-buildpack or binary_buildpack. Edit your manifest.yml and replace https://github.com/cloudfoundry/binary-buildpack.git with that value.
Now when you push, it'll use the existing buildpack which is already compiled and you shouldn't see those messages about Go being downloaded.
I am completely unfamiliar with GitHub.
I am trying to install this Decimal Clock widget from GitHub (https://github.com/XXIIVV/clock) on my Mac.
I have downloaded and git cloned the files. I have tried using npm install and npm run build to no avail.
When I ran npm install (as instructed in the link), it returns: "up to date, audited 3 packages in 2s, found 0 vulnerabilities"
When I run npm run build_osx (as instructed in the link), I get this error: Missing script: "build_osx"
The files in the folder are:
CNAME, PREVIEW.jpg, index.html, pascal (folder), LICENSE.md, README.md, index.js, web (folder)
Inside the pascal folder is "neralie.pas"
I have been trying to install it for days. Perhaps step-by-step instructions would be helpful.
I have added this pdftk buildpack to my free app on Heroku: https://github.com/fxtentacle/heroku-pdftk-buildpack.git.
I am unable to deploy my application to Heroku:
Preparing runtime environment...
-----> Checking for additional extensions to install...
-----> heroku-pdftk-buildpack app detected
cp: cannot stat 'binaries-heroku-18/*': No such file or directory
! Push rejected, failed to compile heroku-pdftk-buildpack app.
! Push failed
Also, I have added heroku/php and set the index of this buildpack to 1. When I go to my app's temporary URL, Laravel runs, but for some reasons, pdftk doesn't seem to run. Has anybody faced the same issue?
That buildpack hasn't been updated for the heroku-18 stack. Its compile script tries to copy precompiled binaries from binaries-$STACK/ into /app/bin/, but only contains binary directories for cedar-14 and heroku-16.
You could roll your app back to heroku-16, which will be supported until April, 2021, but of course this will also change other package versions:
heroku apps:stacks:set heroku-16 --app myapp
This would be very similar to going from Ubuntu 18.04 to Ubuntu 16.04, but it's likely to be your simplest solution.
Alternatively, you could fork the pdftk buildpack you found and update it for Ubuntu 18.04, but that isn't likely to be straightforward.
pdftk was removed from Ubuntu's official repositories because it depends on a deprecated library called gcj. The buildpack you're trying to use includes libgcj for cedar-14 and heroku-16 in their respective binary directories. You'll have to include that as well if you want to update the buildpack.
Also, I have added heroku/php and set the index of this buildpack to 1. When I go to my app's temporary URL, Laravel runs, but for some reasons, pdftk doesn't seem to run. Has anybody faced the same issue?
The error you're seeing is preventing your application from being deployed. You're not seeing a new version of the application with pdftk, but whatever was last deployed successfully. You'll have to resolve the build issue before you can use pdftk.
I followed this article (http://derekbarber.ca/blog/2014/11/20/using-pdftk-with-rails-on-heroku/) and got PDFTK working on Heroku-18, though on a Rails app.
Code excerpted below, HT #derek-barber.
mkdir -p [my_project]/vendor/pdftk/lib [my_project]vendor/pdftk/bin
cd /tmp
git clone https://github.com/millie/pdftk-source.git
cd pdftk-source
tar xzvf pdftk.tar.gz
mv bin/pdftk [my_project]/vendor/pdftk/bin/
mv lib/libgcj.so.12 [my_project]/vendor/pdftk/lib/
cd [my_project]
git add -f vendor/pdftk/
git commit -m "Add pdftk dependencies"
git push heroku master
heroku config:set LD_LIBRARY_PATH=/app/.heroku/vendor/lib:/app/vendor/pdftk/lib
heroku config:set PATH=/app/.heroku/python/bin:/usr/local/bin:/usr/bin:/bin:/app/vendor/p
Once this PR is merged: https://github.com/fxtentacle/heroku-pdftk-buildpack/pull/9 then the buildpack should work with heroku-18 stack.
Maybe leave a comment to the repo owner asking them to merge?
For now, you could use the forked & updated version: https://github.com/Aesthetikx/heroku-pdftk-buildpack which works with heroku-18 stack.
If you are using an app.json file, then point the pdftk buildpack to the forked version:
"buildpacks": [
...,
{
"url": "https://github.com/fxtentacle/heroku-pdftk-buildpack.git"
}
]
I'm trying to put together a custom buildpack with NodeJS and the CouchBase module/libraries
I've gotten as far as using Vulcan to build libcouchbase and libvbucket and getting the buildpack to retrieve and unpack the tgz files for both.
Everything looks ok there, but I receive errors when npm tries to install the couchbase module:
I get a bunch of errors, but this line:
"../src/couchbase_impl.h:52:36: warning: libcouchbase/couchbase.h: No such file or directory"
leads me to think that it can't find the libcouchbase libraries (which is possible since they aren't in the usual place).
I've tried to add the correct path using CPPFLAGS="-I/app/vendor/couchbase/include/libcouchbase" in both the Config Vars and just exporting that as part of the compile phase, but still no luck.
Here is the gist with the Heroku deploy output and the compile/release buildpack files:
https://gist.github.com/ahamidi/5620503
Any help would be greatly appreciated.
Thanks,
Ali
[Update 1]
I've made some progress and I can now get the slug to compile when deploying to Heroku.
The key was figuring out the ENV Variables that CouchNode looks for when adding custom directories to include.
In this case, the Env Variables were EXTRA_CPPFLAGS and EXTRA_LDFLAGS.
So I updated the compile file to include the following:
export EXTRA_CPPFLAGS="-I$BUILD_DIR/vendor/couchbase/include"
export EXTRA_LDFLAGS="-L$BUILD_DIR/vendor/couchbase/lib -Wl,-rpath,$BUILD_DIR/vendor/couchbase/lib"
The slug compiles and the app is deployed, but I now get a different error in the logs:
Error: libcouchbase.so.2: cannot open shared object file: No such file or directory
So it looks like Node can't see the libcouchbase libraries directory.
For anyone who is curious or experiencing a similar issue, here's what worked for me.
In order to get the couchbase npm module to install I had to tell it where to find the libcouchbase libraries (in compile file):
export EXTRA_CPPFLAGS="-I$BUILD_DIR/vendor/couchbase/include"
export EXTRA_LDFLAGS="-L$BUILD_DIR/vendor/couchbase/lib -Wl,-rpath,$BUILD_DIR/vendor/couchbase/lib"
Then in order to require couchbase in my app I had to set the following Env Variable:
LD_LIBRARY_PATH="/app/vendor/couchbase/lib:$LD_LIBRARY_PATH"
With the command:
heroku config:add LD_LIBRARY_PATH="/app/vendor/couchbase/lib:$LD_LIBRARY_PATH"
You should set CPPFLAGS="-I/app/vendor/couchbase/include" LDFLAGS="-L/app/vendor/couchbase/include -lcouchbase"
from your script it seems like you just unpacking libcouchbase without any further work. you should also build it and install. typical magic spell for node.js client will be ./configure --disable-plugins --disable-examples && make && sudo make install. I'm not sure if sudo part needed on heroku, probably just make install
I got Numpy and Matplotlib running on Heroku, and I'm trying to install Scipy as well. However, Scipy requires BLAS[1] to install, which is not presented on the Heroku platform. After contacting Heroku support, they suggested me to build BLAS as a static library to deploy, and setup the necessary environment variables.
So, I compiled libblas.a on a 64bit Linux box, and set the following variables as described in [2] :
$ heroku config
BLAS => .heroku/vendor/lib/libfblas.a
LD_LIBRARY_PATH => .heroku/vendor/lib
LIBRARY_PATH => .heroku/vendor/lib
PATH => bin:/usr/local/bin:/usr/bin:/bin
PYTHONUNBUFFERED => true
After adding scipy==0.10.1 in my requirements.txt, the push still fails.
File "scipy/integrate/setup.py", line 10, in configuration
blas_opt = get_info('blas_opt',notfound_action=2)
File "/tmp/build_h5l5y31i49e8/lib/python2.7/site-packages/numpy/distutils/system_info.py", line 311, in get_info
return cl().get_info(notfound_action)
File "/tmp/build_h5l5y31i49e8/lib/python2.7/site-packages/numpy/distutils/system_info.py", line 462, in get_info
raise self.notfounderror(self.notfounderror.__doc__)
numpy.distutils.system_info.BlasNotFoundError:
Blas (http://www.netlib.org/blas/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [blas]) or by setting
the BLAS environment variable.
It seem that pip is not aware of the BLAS environment variable, so I check the environment using heroku run python:
(venv)bash-3.2$ heroku run python
Running python attached to terminal... import up, run.1
Python 2.7.2 (default, Oct 31 2011, 16:22:04)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.system('bash')
~ $ echo $BLAS
.heroku/vendor/lib/libfblas.a
~ $ ls .heroku/vendor/lib/libfblas.a
.heroku/vendor/lib/libfblas.a
~ $
And it seems fine. Now I have no idea how to solve this.
[1] http://www.netlib.org/blas/
[2] http://www.scipy.org/Installing_SciPy/Linux
I managed to get this working on the cedar stack by building numpy and scipy offline as bdists and then modifying the heroku python buildpack to unzip these onto the dyno's vendor/venv areas directly. You can also use the buildpack to set environment variables that persist.
Heroku haven't officially published buildpacks yet - search for 'heroku buildpacks' for more thirdparty/heroku ones and information.
My fork of the python build pack is here:
https://wyn#github.com/wyn/heroku-buildpack-python.git
The changes are in the bin/compile where I source two new steps, a scipy/numpy step and an openopt step. The scripts for the two steps are in bin/steps/npscipy and bin/steps/openopt. I also added some variables to bin/release. Note that I am assuming installation through a setup.py file rather than the requirements.txt approach (c.f. https://devcenter.heroku.com/articles/python-pip#traditional_distributions).
I also download the blas/lapack/atlas/gfortran binaries that were used to build numpy/scipy onto the dyno as there are c extensions that need to link through to them. The reason for building everything offline and downloading is that pip-installing numpy/scipy requires you to have a fortran compiler + associated dev environment and this made my slugs too big.
It seems to work, the slug size is now 35mb and scaling seems fast too. All but one of the numpy tests pass and all of the scipy tests pass.
This is still work in progress for me but I thought I'd share.
In case someone else comes to this like I did...
The excellent answer on this question from #coshx sadly no longer works (at least I could not get it to work). What I did, however was the following:
I forked npsicpy-binaries repository from #coshx and changed all the tar files such that they do not have venv as the root folder inside (my fork is here)
I forked the npsp-helloworld repository from #coshx and made it use a requirements.txt file instead of the setup.py (my fork is here - this means that you can use the whole pip approach).
I forked the heroku-buildpack-python repository from Heroku, took the npscipy file from #coshx and changed it to work with this latest version of the build pack (my fork is here - you can see that there is no venv set up, for example).
After doing those three things I have the npsp-helloworld application working perfectly. You just need to make sure you set the buildpack appropriately when creating the application and you are good to go:
$ heroku create --stack=cedar --buildpack=https://github.com/kmp1/heroku-buildpack-python.git
NOTE: I haven't worked out how to make my own binaries yet, so the three libraries (scipy, numpy and scikit-learn) are not the latest versions so make sure when you install it you do (if anyone can build these I would gladly accept a pull request for them):
pip install scipy==0.11.0
pip install numpy==1.7.0
pip install scikit-learn==0.13.1
By the way - I am really sorry if I have not done things in the correct way, etiquette-wise. I'm still learning git and this whole open source thing.
Another good option is the conda buildpack, which allows you to add any of the free Linux64 packages available through Anaconda/Miniconda to a Heroku app. Some of the most popular packages include numpy, scipy, scikit-learn, statsmodels, pandas, and cvxopt. While the buildpack makes it fairly simple to add packages to an app, the downsides are that the buildback takes up a lot of space and that you have to wait on Anaconda to update the libraries in the repository.
If you are starting a new Python app on Heroku, you can add the conda buildpack using the command:
$ heroku create YOUR_APP_NAME --buildpack https://github.com/kennethreitz/conda-buildpack.git
If you have already setup a Python app on Heroku, you can add the conda buildpack to the existing app using the command:
$ heroku config:add BUILDPACK_URL=https://github.com/kennethreitz/conda-buildpack.git
Or, if you need to specify the app by name:
$ heroku config:add BUILDPACK_URL=https://github.com/kennethreitz/conda-buildpack.git --app YOUR_APP_NAME
To use the buildpack, you will need to include two text files in the app directory, requirements.txt and conda-requirements.txt. Just as with the standard Python buildpack, the requirements.txt file lists packages that should be installed using pip. Packages that should be installed using conda are listed in the conda-requirements.txt file. Some of the most useful scientific packages include numpy, scipy, scikit-learn, statsmodels, pandas, and cvxopt. The full list of available conda packages can be found at repo.continuum.io.
For example:
$ cat requirements.txt
gunicorn==0.14.2
requests==0.11.1
$ cat conda-requirements.txt
scipy
numpy
cvxopt
That’s it! You can now add Anaconda packages to a Python app on Heroku.
The slug compiler is not aware of your environment variables, which is why it fails during push, and not once running.
The only real option you have is to look at the user_env_compile addon that's currently in labs beta.
http://devcenter.heroku.com/articles/labs-user-env-compile
For those who wish to use Python 3.4 in production, I've built numpy 1.8.1, scipy 0.14.0, and scikit-learn 0.15-git (0.14 doesn't work with the others for some reason) as binaries on Ubuntu 10.04 LTS 64-bit, which works on the Heroku cedar stack. Here is the git repo.
My heroku buildpack is quite similar to that of kmp. Note that the bin/steps/npscipy file links to my repository of binaries above.
I'm putting this here in case someone stumbled on this like I do. Since I am using python 3.4, thenovices buildpack doesn't work out for me.
I tried to look at conda buildpack, but it seems like a little bit of an overkill for my application which only requires scipy and numpy. What finally works for me is this excellent guide, using a multi buildpacks approach. The scipy installation was indeed a bit long though. Hope this helps! :)