Install AWS CLI on heroku - heroku

I would like to use the AWS CLI in an Heroku Ruby project (mainly to use it with a thin wrapper from the ruby application).
Is there any standard way to install additional software like this into an existing application with a Gemfile?

Here are the steps that worked for me:
1) use buildpack-multi to install buildpacks both for ruby and python:
heroku config:add BUILDPACK_URL=https://github.com/ddollar/heroku-buildpack-multi.git
echo "https://github.com/heroku/heroku-buildpack-ruby" >> .buildpacks
echo "https://github.com/heroku/heroku-buildpack-python" >> .buildpacks
echo "web: bundle exec rails server -p $PORT" > Procfile
2) add a requirements.txt file to the root of the project, containing the desired pip package:
echo "awscli" >> requirements.txt
3) deploy to Heroku
git add .buildpacks requirements.txt Procfile
git commit -a -m "use buildpacks for ruby and python, install aws cli"
git push heroku
This works just fine and allows me to use my aws scripts from my ruby app.
As was pointed out to me, using fog is probably the better solution in the long term.

You can use Docker to vendor things for Heroku apps.
Add a Dockerfile
FROM ubuntu:14.04
COPY . /app
Then build an image and run a container:
$ docker build .
$ docker run -it $image_id bash
# apt-get update && apt-get install jq
Now you can copy the data out from another terminal:
$ docker cp $container_id:/usr/bin/jq .
The aws-cli tool is tricker because it needs a whole Python environment.

You should add the Heroku buildpack for the AWS CLI via running:
$ heroku buildpacks:add heroku-community/awscli
More details can be found on herokus page, or in the buildpack's git repo.

Related

Keep docker container running in case of error

I have following docker file
FROM ruby:latest
# throw errors if Gemfile has been modified since Gemfile.lock
RUN bundle config --global frozen 1
WORKDIR /usr/src/app/
COPY Gemfile Gemfile.lock ./
RUN bundle install
ADD . /usr/src/app/
EXPOSE 3333
CMD ["ruby", "/usr/src/app/helloworld.rb"]
When I run image
docker run -t -d hello-world-ruby
Sinatra throws exception (which is OK), and container exits.
How can I keep it running so I can ssh into container and debug what's happening?
A trick you can use is to start the application with a script.
The following will work:
FROM ruby:latest
# throw errors if Gemfile has been modified since Gemfile.lock
RUN bundle config --global frozen 1
WORKDIR /usr/src/app/
COPY Gemfile Gemfile.lock ./
RUN bundle install
ADD . /usr/src/app/
EXPOSE 3333
CMD [/usr/src/app/startup.sh]
and in startup.sh, do:
#!/bin/bash
ruby /usr/src/app/helloworld.rb &
sleep infinity
# or if you're on a version of linux that doesn't support bash and sleep infinity
while true; do sleep 86400; done
chmod 755 and you should be set.
For debugging puproposes it is not needed to keep the container with a failing command alive, with loops or otherwise.
To debug such issues just spawn a new container from the image with entrypoint/command as bash.
docker run -it --name helloruby hello-world-ruby bash
OR
docker run -it --name helloruby --entrypoint bash hello-world-ruby
This will give you shell inside the container where you can run/debug the ruby app
ruby /usr/src/app/helloworld.rb

Error when running a ruby script on AWS Lambda with a gem from a github repository (using Docker)

I try to build a simple ruby script that checks the balance on the Kraken crypto exchange using the custom jonatack/kraken_ruby_client gem from github which is not available on ruby gems.
AWS Lambda throws the following error:
"errorMessage": "https://github.com/jonatack/kraken_ruby_client.git (at master#cda6c28) is not yet checked out. Run `bundle install` first.",
"errorType": "Init<Bundler::GitError>",
"stackTrace": ["/var/runtime/gems/bundler-2.1.4/lib/bundler/source/git.rb:204:in `rescue in load_spec_files'",
...
What I'm doing:
1. Gemfile
I've added the gem to the Gemfile as usual:
gem 'kraken_ruby_client', git: 'https://github.com/jonatack/kraken_ruby_client.git'
Everything works perfectly on my local machine.
2. Custom Dockerfile
Since the gem requires curb I created a Dockerfile, based on the AWS Lambda image for Ruby 2.7, to install libcurl:
FROM lambci/lambda:build-ruby2.7
RUN yum install -y curl-devel
RUN gem install curb
3. Deployment
I then "deploy" the script:
docker run -v `pwd`:`pwd` -w `pwd` -i -t kraken-app:latest bundle install --deployment
zip build.zip * -r -x .git/\* \*.sh specs/\* tests/\* \*.zip config/application.yml
aws lambda update-function-code --function-name kraken-app --zip-file fileb://build.zip --region=eu-west-1 --publish --profile XXXX
And it fails with the above error.
What I've tried with multiple StackOverflow answers:
1. Do what the error says - bundle install
I "bundle installed" and "bundled" as much as I could
2. Paths
All gems from Rubygems are installed into vendor/bundle/ruby/2.7.0/gems/, while the gem from github is installed into /vendor/bundle/ruby/2.7.0/bundler/gems/
So I've tried to add the path to $LOADPATH:
load_paths = Dir["./vendor/bundle/ruby/2.7.0/bundler/gems/**/lib"]
$LOAD_PATH.unshift(*load_paths)
As that didn't help, I even manually moved the folder from /bundler/gems to /gems before deploy, but no avail.
3. Local install in Docker image
I amended the Dockerfile to install the gem "locally" in the image by adding:
RUN gem install specific_install
RUN gem specific_install https://github.com/jonatack/kraken_ruby_client.git
I can run the script just fine in Docker in bash:
docker run -it --rm -v "$PWD":/var/task kraken-app:latest bash
Everything works in the Docker container on my local machine but it doesn't work when deployed to Lambda for some reason? Other Lambda functions work perfectly, I just can't get this custom gem to work...
Do you have any idea what else to try?
Thanks in advance!

How do I run bulma on Heroku?

I can't run bulma on Heroku and npm is not a Heroku command.
What should I type to do npm install bulma on Heroku? Do I need to use yarn?
brew install yarn, yarn add bulma doesn't work on Heroku either.
Here is the error I'm seeing:
Sass::SyntaxError: File to import not found or unreadable: bulma/sass/utilities/_all.sass.
Don't install Bulma yourself, and certainly don't try to install yarn via Homebrew.
Heroku's ephemeral filesystem will prevent it from working properly, interactive commands run on temporary one-off dynos that only exist as long as your session runs, and brew is mostly a macOS package manager (though a Linux version does technically exist).
Heroku will build your application for you as long as you tell it how.
Make sure to include a package.json and package.lock or yarn.lock that includes the JavaScript libraries you need
Tell Heroku which buildpacks to use for your application:
$ heroku buildpacks:set heroku/ruby
$ heroku buildpacks:add --index 1 heroku/nodejs
Make sure the Ruby buildpack is last:
$ heroku buildpacks
=== your-app Buildpack
1. heroku/nodejs
2. heroku/ruby
Deploy your code
Heroku will run the Node.js buildpack first and install JavaScript packages from your lock file, then run the Ruby buildpack and install your gems.

Installing bundler within Jenkins Docker image

Is it possible to install Ruby within a docker image, specifically Jenkins?
I can see from the docs that you can attach to a container or use docker exec -i -t 4e2bf4128e3e bash. This will log me in as jenkins#4e2bf4128e3e.
But if I try and install anything
apt-get install ruby 2.0.0 # Yes will install rvm, this is just an example
I get
E: Could not open lock file /var/lib/dpkg/lock - open (13: Permission denied)
E: Unable to lock the administration directory (/var/lib/dpkg/), are you root?
And when I try
sudo apt-get install ruby 2.0.0
Then I get sudo command not found.
The problem you have is that, as you can see here, the jenkins docker images executes commands as the jenkins user which is forbidden to use apt.
On https://hub.docker.com/_/jenkins/ you have some documentation, namely the "Installing more tools" section which advise you to do this:
FROM jenkins
# if we want to install via apt
USER root
RUN apt-get update && apt-get install -y ruby make more-thing-here
USER jenkins # drop back to the regular jenkins user - good practice
You could create your own image, that layers those two images
Dockerfile
FROM jenkins
FROM ruby
...
Now you have a docker container of your own that has both ruby AND jenkins.

how do you run heroku client commands on travis-ci?

what can i add to my .travis.yml to run heroku client commands?
e.g. before deploy:
$ heroku maintenance:on
$ heroku pgbackups:capture --expire
and after deploy:
$ heroku maintenance:on
i've tried adding those commands to before_deploy in .travis.yml, but it doesn't have access to the heroku cli tool.
bonus points if i can do this on the app level, e.g. do backups on the production branch, but not staging.
I just installed Heroku toolbelt in a build container. That's how it looks in my build:
sudo: required
before_script:
# this install.sh script requires sudo
- wget -qO- https://toolbelt.heroku.com/install.sh | sh
script:
- /usr/local/heroku/bin/heroku restart --app my-heroku-app-name # requires HEROKU_API_KEY env variable
Note: installing Heroku requires sudo privilege (sudo: required) and if you have to access your app - $HEROKU_API_KEY environment variable must be set.
You might be able to everything you need with the Heroku deployment support available on Travis CI.
If you need to do more, you need to install Heroku toolbelt, and figure out how to do what you want.

Resources