I have a ruby sinatra app, and in one place I need to call out to a python script -- using ruby's system() -- and that python script depends on a python package.
Python3 is pre-installed on heroku (I checked by looking at one of my existing ruby apps), but my question is: How can I install both ruby gems for my main app and the python package needed for the python script?
Obviously installing the gems is just done using the Gemfile, and if I was writing a pure python application I would use requirements.txt, but I'm unclear how to proceed in this case where I need both.
You can use multiple buildpacks in your application. Briefly:
Add a requirements.txt file (or Pipfile and Pipfile.lock) to your root directory containing your Python dependencies
Set your Ruby buildpack explicitly:
heroku buildpacks:set heroku/ruby
Add the official Python buildpack before the Ruby buildpack (your main buildpack should be last):
heroku buildpacks:add --index 1 heroku/python
Now, redeploy your application.
Related
I’m updating the environment on Heroku and one of the buildpack we use is based on Ruby, which is no longer available by default in the new heroku-22 environment/stack (nor required by our PHP app).
From the docs:
[...] end users should add the Ruby buildpack prior to the buildpack in question (they will also need to ensure minimal Gemfile / Gemfile.lock files exist, so that the Ruby buildpack passes detection).
However I have no clue what those files should include as I have zero experience with Ruby. What would be a valid set of minimal Gemfiles to trigger Ruby installation on Heroku?
I suggest you don't use that buildpack at all. It's ancient, and third-party buildpacks are always a bit questionable, even if it's just because they often stop getting updated.
Here's what it claims to do:
This is a Heroku buildpack for vendoring just the mysql binary from the mysql-client-core deb package.
If you just need a mysql binary, you can use the apt buildpack to install it without worrying about Ruby or anything like that.
Add it as your first buildpack:
heroku buildpacks:add --index 1 heroku-community/apt
Create an Aptfile in the root directory of your project that lists the Ubuntu packages you wish to install, e.g.
mysql-client-core-8.0
Note that the buildpack does not do dependency resolution. If any packages you list have their own dependencies you may have to list them explicitly.
Commit, and redeploy.
You should see the Ubuntu packages you listed get installed before your main buildpack runs.
In any case, if you really want to make your application compatible with the Ruby buildpack you should be able to simply include an empty Gemfile in the root of your project:
The Heroku Ruby Support will be applied to applications only when the application has a Gemfile in the root directory. Even if an application has no gem dependencies it should include an empty Gemfile to document that your app has no gem dependencies.
A Gemfile.lock is not required.
Note that you'll need to manually add the buildpacks you require. I believe you'll want Ruby first, then the MySQL buildpack in your question, then whatever language your application is written in, which appears to be PHP:
heroku buildpacks:set heroku/php # Main buildpack; we insert others before it below
heroku buildpacks:add --index 1 heroku/ruby
heroku buildpacks:add --index 2 https://github.com/thoughtbot/heroku-buildpack-mysql.git
heroku buildpacks
# => Should print the buildpacks in the expected order
Edited
It turns out an empty Gemfile is not enough and a Gemfile.lock is actually required as well.
-----> Building on the Heroku-22 stack
-----> Using buildpacks:
1. heroku/python
2. heroku/ruby
3. https://github.com/thoughtbot/heroku-buildpack-mysql
4. heroku/php
5. heroku/nodejs
-----> Python app detected
-----> Using Python version specified in runtime.txt
-----> Stack has changed from heroku-20 to heroku-22, clearing cache
-----> No change in requirements detected, installing from cache
-----> Installing python-3.10.8
-----> Installing pip 22.2.2, setuptools 63.4.3 and wheel 0.37.1
-----> Installing SQLite3
-----> Installing requirements with pip
Collecting supervisor
Downloading supervisor-4.2.4-py2.py3-none-any.whl (749 kB)
Installing collected packages: supervisor
Successfully installed supervisor-4.2.4
-----> Ruby app detected
grep: /tmp/build_bebc9aa2/Gemfile.lock: No such file or directory
-----> Compiling Ruby/NoLockfile
!
! Gemfile.lock required. Please check it in.
!
! Push rejected, failed to compile Ruby app.
! Push failed
An empty Gemfile will trigger the Ruby installation, however the heroku/ruby package itself does require a lock file.
The buildpack will detect your app as Ruby if it has a Gemfile and Gemfile.lock files in the root directory.
After some trial and error I figure out the following files work:
# Gemfile
source 'https://rubygems.org'
# Gemfile.lock
GEM
remote: https://rubygems.org/
specs:
PLATFORMS
ruby
DEPENDENCIES
BUNDLED WITH
1.17.3
We are using a the distribution gem for calculating a gamma distribution and because we specifically require the gamma quantile method, we also require gsl
https://github.com/sciruby/distribution
https://github.com/SciRuby/rb-gsl
This works fine locally but we are trying to push this to heroku and running into immediate problems.
From the looks of things, we require a gsl buildpack which are only supported up to version 1.16 of gsl. There doesn't appear to be a gsl buildpack for the current version 2.1. Installing GSL 1.16 locally appears to be really problematic and we haven't been successful doing this either.
Looking into this further, it looks like we would need to create a binary for the curret version which would need to be hosted on AWS. Seems a bit extreme for what is basically two methods.
Either, we need help building a buildpack or need an alternative to these two specific methods.
Distribution::Gamma.quantile
Distribution::Gamma::Ruby_.cdf
Any help is greatly appreciated
The installation instructions for rb-gsl say (slightly modified for clarity):
Ruby/GSL may be installed as a Ruby Gem by simply running
gem install gsl
Note that the GSL libraries must already be installed before Ruby/GSL can be installed:
Debian/Ubuntu: libgsl0-dev
Rather than a custom buildpack, I suspect you can get away with simply depending on the gsl gem and adding the apt buildpack for the Ubuntu dependency:
Add the apt buildpack early in your list of buildpacks, e.g. by running
heroku buildpacks:add --index 1 heroku-community/apt
Create an Aptfile in the root directory of your repository and list the Ubuntu packages you require inside it, e.g.
libgsl0-dev
Commit the Aptfile and redeploy
Note that the apt buildpack does not do dependency resolution. If libgsl0-dev requires other packages, you'll need to explicitly list them as well.
In Python I can install a package from source in "editable" mode using pip install -e. Then I can carry on editing the code, and any changes will be automatically picked by other Python scripts that import library
Is there a comparable workflow for developing Ruby gems? What is the "Ruby way" of using libs as they are being developed rather than, for example, compiling and installing a gem every time I make a change to the source?
There are two common approaches one might use with bundler:
one executes bundle install --path vendor/bundle and does not run bundle update unless everything is tested.
one tells a bundler to use a local version of the gem:
in Gemfile (this is not supported in mymaingem.gemspec due to rubygems maintainence issues): gem 'mycutegem', :git => 'git://github.com/myname/mycutegem', :branch => 'master';
in command line: bundle config local.mycutegem /path_to_local_git/mycutegem.
The first approach will download everything into subfolder of your current project (here it’d be vendor/bundle.) Feel free to modify everything there, it’ll be reflected.
The second approach is likely better. You are to clone the gem from github and instruct bundle to use your local clone of the respective git repository. This approach provides you with an ability to publish the changes to your main gem into the repository. As soon as dependent repo is published as well, the up-to-date version will be retrieven by your gem subscribers, assuming they have not instructed their bundlers to use their locals.
Hope this helps.
Let's assume you have your gem code residing in a folder (say my_project/mygem/lib). You have some Ruby code in my_project that you want using the mygem code.
What I'd do is add mygem/lib to the $LOAD_PATH global variable in the beginning of said files. Kinda like this:
$LOAD_PATH << File.expand_path('lib', './mygem') # Resolve global paths
require 'a_file' # Would require "mygem/lib/a_file.rb"
I am not sure if this is exactly what you want to achieve, but from the description I infer that you want to have a local copy of some gem and reference that gem in your project.
If this is the case, you can (usually) achieve it in two steps:
Clone the gem from VCS (in most cases: git), e.g.
git clone url-of-the-gem-repo
Reference the local copy using Bundler :path, e.g.
gem "some-gem", :path => "/path/to/local/copy"
If the gem is stored at github, an even better way is to first fork it at github and then clone your own copy. Then, if you provide any improvements to the code in the local repo, you can easily share it with the world using a pull request.
I realize this is a 5 year old question, but I found all of these answers unsatisfying. Since I use ruby to develop CLI tools, using bundler for testing is not ideal. I need to be able to execute my test commands anywhere, and get the actual equivalent of pip install --editable.
Here's my solution.
Use RVM to setup a new gemset for development
rvm gemset use tools-dev --create
Install your gems
gem install mygem.gem
Locate the installation directory
gem list tool -d
In the output, it will say installed at ${rvm_gemset_path}. Copy that path. It will also be an environment variable GEM_HOME, but I include this step for completeness and clarity.
Delete the installed copy
rm -Rf ${rvm_gemset_path}/gems/mygem-${version}
Create a symlink to the git repository working directory.
ln -s ${PWD} ${rvm_gemset_path}/gems/mygem-${version}
This bash script will do it in a more automated fashion assuming that you are in the git directory and you've set GEM_NAME as your gem's name.
rvm gemset use --create ${GEM_NAME}-dev
gem build ${GEM_NAME}.gemspec
gem install ${GEM_NAME}*.gem
gem_install_path=$(ruby -e "puts Gem::Specification.find_by_name('${GEM_NAME}').full_gem_path")
mv $gem_install_path "${gem_install_path}.bak"
ln -s $PWD $gem_install_path
I have a Node.js app that uses Compass, a Ruby gem.
When I push to Heroku it will detect a Node.js app and run npm install. Now, it detects the Gemfile first which it a ruby project and no longer runs npm install. Is it possible to tell Heroku this is a Node.js app which requires the Gemfile for running bundle exec install compass?
You need to use a custom buildpack that supports both Ruby and Node. Take a look at third-party buildpacks and multi buildpacks.
I'm cloning this repo:
https://github.com/freeformz/sinatra-jruby-heroku.git
to try and use JRuby/Sinatra on Heroku's Cedar stack. I follow the included instructions and everything runs great locally with a 'foreman start'. I then git push to Heroku and it initially loads up fine but when I try to access the site I get an error in the logs:
jruby: No such file or directory -- trinidad (LoadError)
So it seems jruby can't find the "/app/.gems/bin/trinidad" file. I initially thought it wasn't there because .gems/ is in the .gitignore file, but I'm pretty sure Heroku creates that server side on a git push.
$APPDIR/.gems is added to the PATH so Heroku should be able to see the trinidad script. I've also tried to change the Procfile around to play with the path like:
web: script/jruby -S bin/trinidad -p $PORT
But no dice. Has anyone had any success deploying anything JRuby to Heroku cedar?
Thanks
As of Bundler 1.2 you are now able to specify the Ruby implementation and version in your Gemfile. The nice thing about this is that Heroku will understand these settings and prepare the your Heroku application for your environment.
Take this Gemfile for example:
source "https://rubygems.org"
ruby "1.9.3"
gem "rails"
gem "puma"
What's cool about this is that by default Celadon Cedar uses Ruby 1.9.2. However, when you specify ruby "1.9.3" in the Gemfile it'll actually compile Ruby 1.9.3 for your Heroku environment.
Now, if you want to add a different Ruby implementation to your Heroku environment, you can do so like this:
source "https://rubygems.org"
ruby "1.9.3", :engine => "jruby", :engine_version => "1.7.0.preview1"
gem "rails"
gem "puma"
Now it'll install and use JRuby 1.7.0.preview1 in Ruby 1.9 mode for your Heroku application upon deployment. It'll also even define the proper JVM options in the Heroku environment variables.
Best of all is that this comes with the official Heroku buildpack, so there is no need to switch to a 3rd party buildpack to get the JRuby/JVM going on Heroku. Although I haven't gotten it to work yet, this should also work with Rubinius, but I believe it's currently bugged. Either that, or I'm doing it wrong.
This is in my opinion an awesome and scalable feature. Just define the Ruby implementation/version/mode you're using in your Gemfile along with your other dependencies and Heroku will ensure the environment is prepared.
Now, with all this in place, Heroku should create binstubs (through Bundler) in APP_ROOT/bin so what you can do is for example this:
web: bin/trinidad -p $PORT -e $RACK_ENV --threaded
Just don't use bundle exec since JRuby doesn't play nice with that. Always use the binstubs provided by Bundler which are always located in APP_ROOT/bin on Heroku.
I believe the details about including gems on this blog entry might be helpful to you:
http://chris.chowie.net/2011/08/28/Sinatra-with-JRuby-on-Heroku/
I suspect that your gems are not in /app/.gems but rather in /app/vendor/bundle
You can find out by running this command:
heroku run ls /app/.gem
heroku run ls /app/vendor/bundle
Either way, you should probably add the GEM_HOME/bin to the path, and not the GEM_HOME as you state.
I've got a blog post on running Jruby and Trinidad on Heroku here:
http://deployingjruby.blogspot.com/2012/03/deploying-with-trinidad-on-heroku.html
And an example app here:
https://github.com/jkutner/jruby-trinidad-heroku
Some of the other material you may find is a little out of date.