Bitbucket pipelines not using cache for bundle install - ruby

I have tried every variation of naming my cache and nothing seems to work
My cache is being downloaded properly as I can see in the build setup.
Cache "bundler": Downloading
Cache "bundler": Downloaded 25.2 MiB in 1 seconds
Cache "bundler": Extracting
Cache "bundler": Extracted in 0 seconds
however when I go to bundle install I always get
Fetching gem metadata from https://rubygems.org/.......
Installing rake 12.3.3
Installing concurrent-ruby 1.1.5
Installing i18n 0.9.5
Installing minitest 5.11.3
Installing thread_safe 0.3.6
Installing tzinfo 1.2.5
Installing activesupport 5.2.3
Installing builder 3.2.3
..........
Here is my bitbucket-pipelines.yml
pipelines:
default:
- step:
name: Run Tests
image: ruby:2.6.4
caches:
- bundler
script:
- apt-get update && apt-get install -y build-essential libpq-dev nodejs python-dev
- bundle install --path ./vendor
- cp config/database.yml.example config/database.yml
- RAILS_ENV=test bundle exec rake db:setup
- RAILS_ENV=test bundle exec rake db:schema:load
- RAILS_ENV=test bundle exec rspec
services:
- postgres
- redis
definitions:
caches:
bundler: ./vendor
services:
redis:
image: redis
postgres:
image: postgres
environment:
POSTGRES_DB: 'xxxxx'
POSTGRES_USER: 'xxxxx'
POSTGRES_PASSWORD: 'xxxxx'

I hope you are solved you problem, the BitBucket Caches docs are out of date. So a solved my problem with this commands:
- gem install bundler -v 2.1.4
- bundle config set path 'vendor/bundle'
- bundle install
- bundle binstubs --all
My be, is needed remove tzinfo-data for linux deploy.
You can understand more reading the bundle binstubs docs.

Related

Using a Docker image to run bundle install ignores bundle config setting

I'm hoping to use a Docker image (ruby:3.0) build an image without (eventually) having to have Ruby installed locally.
For testing purposes, I have Ruby 2.7.0 installed in Windows 10 WSL2 environment:
$ ruby -v
ruby 2.7.0p0 (2019-12-25 revision 647ee6f091) [x86_64-linux-gnu]
I have a Gemfile:
source 'https://rubygems.org'
gem 'sinatra'
group :development, :test do
gem 'thin'
gem 'sqlite3'
end
And bundler set to install Gems in the project's directory:
$ cat .bundle/config
---
BUNDLE_PATH: "vendor/bundle"
If I run bundle install locally:
$ bundle install
Fetching gem metadata from https://rubygems.org/...
Resolving dependencies...
Using bundler 2.2.17
Fetching rack 2.2.3
Fetching tilt 2.0.10
Fetching daemons 1.4.1
Fetching sqlite3 1.4.2
Fetching eventmachine 1.2.7
Fetching ruby2_keywords 0.0.5
Installing rack 2.2.3
Installing tilt 2.0.10
Installing sqlite3 1.4.2 with native extensions
Installing ruby2_keywords 0.0.5
Installing daemons 1.4.1
Installing eventmachine 1.2.7 with native extensions
Fetching mustermann 1.1.1
Installing mustermann 1.1.1
Fetching rack-protection 2.1.0
Installing rack-protection 2.1.0
Fetching sinatra 2.1.0
Installing sinatra 2.1.0
Fetching thin 1.8.1
Installing thin 1.8.1 with native extensions
Bundle complete! 3 Gemfile dependencies, 11 gems now installed.
Bundled gems are installed into `./vendor/bundle`
the Gems are installed to the ./vendor/bundle directory:
confirmed:
$ bundle info thin
* thin (1.8.1)
Summary: A thin and fast web server
Homepage: https://github.com/macournoyer/thin
Source Code: https://github.com/macournoyer/thin
Changelog: https://github.com/macournoyer/thin/blob/master/CHANGELOG
Path: /home/craig/ruby/dev/vendor/bundle/ruby/2.7.0/gems/thin-1.8.1
Next, I use the ruby:3.0 image to bundle the Gems in the project's directory.
I removed the Gemfile.lock and ./vendor/bundle directory, then ran bundle install using the image:
$ docker run --rm -v "$(pwd)":/src -w /src ruby:3.0 bundle install
Fetching gem metadata from https://rubygems.org/...
Resolving dependencies...
Using bundler 2.2.32
Fetching sqlite3 1.4.2
Fetching eventmachine 1.2.7
Fetching rack 2.2.3
Fetching tilt 2.0.10
Fetching ruby2_keywords 0.0.5
Fetching daemons 1.4.1
Installing ruby2_keywords 0.0.5
Installing tilt 2.0.10
Installing daemons 1.4.1
Installing sqlite3 1.4.2 with native extensions
Installing rack 2.2.3
Fetching mustermann 1.1.1
Installing eventmachine 1.2.7 with native extensions
Installing mustermann 1.1.1
Fetching rack-protection 2.1.0
Installing rack-protection 2.1.0
Fetching sinatra 2.1.0
Installing sinatra 2.1.0
Fetching thin 1.8.1
Installing thin 1.8.1 with native extensions
Bundle complete! 3 Gemfile dependencies, 11 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
This process seemed to install the Gems, but not to ./vendor/bundle. Trying to identify the location generates an error:
$ bundle info thin
Could not find daemons-1.4.1 in any of the sources
It would appear that .bundle/config is being ignored. I'm guessing that the Gems are actually being installed into the container that is created to run bundle install.
Is there a way to get this to use .bundle/config?
this happens due to few things:
gem installation directory, which is configured in bundler's configuration that is packed with the image
BUNDLE_APP_CONFIG environment variable defined in the docker image
looking at bundler documentation, we have to note few things:
list of available keys, BUNDLE_PATH is not there, but path is...
The location to install the specified gems to.
executing config
Executing bundle config set --local <name> <value> will set that configuration in the directory for the local application. The configuration will be stored in <project_root>/.bundle/config. If BUNDLE_APP_CONFIG is set, the configuration will be stored in $BUNDLE_APP_CONFIG/config
Bundler loads configuration settings in this order
Local config (<project_root>/.bundle/config ...)
let's give it try...
$ docker run --rm -it -v $PWD:/src --workdir /src --entrypoint /bin/sh ruby:alpine
/src # ls -l
total 0
/src # echo $BUNDLE_APP_CONFIG
/usr/local/bundle
/src # export BUNDLE_APP_CONFIG=$PWD/.bundler
$ bundle config set --local path 'vendor/bundle'
/src # ls -la
total 4
drwxr-xr-x 3 root root 96 Dec 16 21:10 .
drwxr-xr-x 1 root root 4096 Dec 16 21:10 ..
drwxr-xr-x 3 root root 96 Dec 16 21:10 .bundler
/src # bundle config
Settings are listed in order of priority. The top value will be used.
app_config
Set via BUNDLE_APP_CONFIG: "/src/.bundler"
path
Set for your local app (/src/.bundler/config): "vendor/bundle"
silence_root_warning
Set via BUNDLE_SILENCE_ROOT_WARNING: true
/src # bundle init
Writing new Gemfile to /src/Gemfile
/src # echo 'gem "sinatra"' >> Gemfile
/src # bundle install --quiet
/src # ls -l
total 8
-rw-r--r-- 1 root root 161 Dec 16 21:12 Gemfile
-rw-r--r-- 1 root root 398 Dec 16 21:13 Gemfile.lock
drwxr-xr-x 3 root root 96 Dec 16 21:18 vendor
/src # du -sh vendor/bundle/
3.1M vendor/bundle/
/src # bundle exec rackup --version
Rack 1.3 (Release: 2.2.3)
/src # exit
$ ls -ax1
.
..
.bundler
Gemfile
Gemfile.lock
vendor
as you can see:
bundler's configuration is stored in your local directory
bundler installed the gems into your local directory

Docker Ruby Bundler not installing

My Dockerfile looks like below:
FROM ruby:2.5-alpine
RUN apk update add --no-cache build-base nodejs postgresql-dev
RUN mkdir /my-app
WORKDIR /my-app
COPY Gemfile Gemfile.lock ./
RUN bundle install --binstubs
COPY . .
CMD puma -C config/puma.rb
Now when I try to create an image, it gives following error:
You must use Bundler 2 or greater with this lockfile.
The command '/bin/sh -c bundle install --binstubs' returned a non-zero code: 20
The bundler version Gemfile.lock using is 2.1.4.
I did tried adding following line to Dockerfile:
RUN gem update --system
RUN gem install bundler -v 2.1.4
In that case, I get following error:
ERROR: Could not find a valid gem 'bundler' (= 2.1.4), here is why:
Unable to download data from https://rubygems.org/ - no such name (https://rubygems.org/specs.4.8.gz)
I am using Ubuntu 18.04 system. Anyone had faced this issue before? Any help appreciated.
Thank you in advance.
FROM ruby:2.5-alpine
RUN apk update add --no-cache build-base nodejs postgresql-dev
RUN mkdir /my-app
WORKDIR /my-app
RUN gem update --system
RUN gem install bundler -v 2.1.4
COPY Gemfile Gemfile.lock ./
RUN bundle install --binstubs
COPY . .
CMD puma -C config/puma.rb
So, installing bundler was fine but due IPV6 of rubygems.org is still broken and create issue while fetching inside docker. So, i ran the command using host network.
docker image build --network=host -t my-app .
And this issue was resolved.
Thanks for taking time.
I need to create an answer since I can not post this all into a comment. I realize that this is not a solution for you.
If I use this setup, It runs without an error. Can you please look where it differs from your setup and tests? (please also add this to the question)
dockerfile
FROM ruby:2.5-alpine
RUN apk update add --no-cache build-base nodejs postgresql-dev
RUN mkdir /my-app
WORKDIR /my-app
RUN gem update --system && gem install bundler && bundle --version
# installs bundler 2.1.4
COPY Gemfile Gemfile.lock ./
RUN bundle install
COPY . .
CMD /bin/sh
Gemfile
ruby '2.5.8'
source 'https://rubygems.org'
gem 'flay'
Gemfile.lock
GEM
remote: https://rubygems.org/
specs:
erubis (2.7.0)
flay (2.12.0)
erubis (~> 2.7.0)
path_expander (~> 1.0)
ruby_parser (~> 3.0)
sexp_processor (~> 4.0)
path_expander (1.0.3)
ruby_parser (3.13.1)
sexp_processor (~> 4.9)
sexp_processor (4.12.0)
PLATFORMS
ruby
DEPENDENCIES
flay
RUBY VERSION
ruby 2.5.8p224
BUNDLED WITH
2.1.4
Build and started with:
docker build . --no-cache -t foobar
docker run --it foobar

Cannot upgrade past CocoaPods 0.39.0

I am trying to upgrade to CocoaPods 1.2.1 but cannot uninstall CocoaPods 0.39.0.
I have reviewed numerous posts about this issue. Evidently 0.39.0 caused a lot of problems. I do not have enough reputation points to attach all the links I have reviewed but there are several StackOverFlow questions and GitHub Issues on the topic as v.0.39.0 evidently had some problems. Here are two of the many posts I reviewed:
STACKOVERFLOW QUESTIONS:
Cocoa Pods need to completely re-install
COCOAPODS BLOG POST ABOUT SHARDING and v.0.39.0:
http://blog.cocoapods.org/Sharding/
Here is my 'gem env' output:
RubyGems Environment:
- RUBYGEMS VERSION: 2.6.12
- RUBY VERSION: 2.3.1 (2016-04-26 patchlevel 112) [x86_64-darwin15]
- INSTALLATION DIRECTORY:
/Users/johndoe/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0
- USER INSTALLATION DIRECTORY: /Users/johndoe/.gem/ruby/2.3.0
- RUBY EXECUTABLE: /Users/johndoe/.rbenv/versions/2.3.1/bin/ruby
- EXECUTABLE DIRECTORY: /Users/johndoe/.rbenv/versions/2.3.1/bin
- SPEC CACHE DIRECTORY: /Users/johndoe/.gem/specs
- SYSTEM CONFIGURATION DIRECTORY:
/Users/johndoe/.rbenv/versions/2.3.1/etc
- RUBYGEMS PLATFORMS:
- ruby
- x86_64-darwin-15
- GEM PATHS:
- /Users/johndoe/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0
- /Users/johndoe/.gem/ruby/2.3.0
- GEM CONFIGURATION:
- :update_sources => true
- :verbose => true
- :backtrace => false
- :bulk_threshold => 1000
- REMOTE SOURCES:
- https://rubygems.org/
- SHELL PATH:
- /Users/johndoe/.rbenv/versions/2.3.1/bin
- /usr/local/Cellar/rbenv/1.1.1/libexec
- /usr/local/bin
- /usr/local/sbin
- /Users/johndoe/.rbenv/shims
- /Users/johndoe/.rbenv/shims
- /usr/local/bin
- /usr/bin
- /bin
- /usr/sbin
- /sbin
- /Applications/Postgres.app/Contents/Versions/latest/bin
- /Users/johndoe/.rbenv/versions/2.3.1/bin
And here is my 'gem list --local | grep cocoapods' output:
cocoapods (1.2.1)
cocoapods-core (1.2.1, 1.2.0, 0.39.0)
cocoapods-deintegrate (1.0.1)
cocoapods-downloader (1.1.3, 0.9.3)
cocoapods-plugins (1.0.0, 0.4.2)
cocoapods-search (1.0.0, 0.1.0)
cocoapods-stats (1.0.0, 0.6.2)
cocoapods-trunk (1.2.0, 0.6.4)
cocoapods-try (1.1.0, 0.5.1)
I have tried the following solutions:
1) "sudo gem uninstall cocoapods" then "sudo gem install cocoapods" and closed and reopened the terminal after each step
2) I modified my .bash_profile to make sure my echo $PATH includes the - EXECUTABLE DIRECTORY: from my gem env
3) Tried "gem pristine --all"
4) Made sure my home brew was up to date
5) Reinstalled rbenv
6) Tried various combinations of:
$ sudo rm -fr ~/Library/Caches/CocoaPods/
$ sudo rm -fr ~/.cocoapods/repos/master/
$ sudo rm -fr Pods/
and then uninstalled and reinstalled cocoapods…
7) Manually applied “sudo gem uninstall” to each element in the output of my “gem list --local | grep cocoapods” and then …
rm -rf ~/.cocoapods/repos/master
sudo gem install cocoapods
8) Tried
$ sudo gem update cocoapods
9) I tried:
sudo gem install cocoapods:1.2.1
but when I try
pod _1.2.1_ version
or
pod —version
it says 0.39.0
10) The output of “which pod” is:
/usr/local/bin/pod
11) I cannot recall if the first time I tried “sudo gem uninstall cocoapods” it gave me the option of specifically uninstalling 0.39.0. I chose “All Versions” nevertheless
12) I tried “sudo chown -R "$(whoami)" /usr/local/Cellar/rbenv" to make sure it wasn’t a permissions issue
13) I am using Mac OS Sierra 10.12.5
Any help would be GREATLY appreciated! Help!?!
The solution to my problem was:
sudo gem install -n /usr/local/bin cocoapods
This was the only way I could get rid of 0.39.0 and install the latest version of CocoaPods...
I found this solution and an explanation for why this was a problem at the following link:
https://github.com/CocoaPods/CocoaPods/issues/3736
Hooray!

public_suffix not getting installed while running bundle install jekyll github-pages

I have a blog of mine in jekyll and I want to publish it on github-pages.
Taking reference from here:
https://help.github.com/articles/using-jekyll-with-pages.
but I am getting this error while running bundle install
Gem::InstallError: public_suffix requires Ruby version >= 2.0.
An error occurred while installing public_suffix (1.5.1), and Bundler cannot continue.
Make sure that `gem install public_suffix -v '1.5.1'` succeeds before bundling.
below is the details of my gem env
gem env :
RubyGems Environment:
- RUBYGEMS VERSION: 2.2.3
- RUBY VERSION: 2.1.6 (2015-04-13 patchlevel 336) [x86_64-linux-gnu]
- INSTALLATION DIRECTORY: /var/lib/gems/2.1.0
- RUBY EXECUTABLE: /usr/bin/ruby2.1
- EXECUTABLE DIRECTORY: /usr/local/bin
- SPEC CACHE DIRECTORY: /home/ashwin/.gem/specs
- RUBYGEMS PLATFORMS:
- ruby
- x86_64-linux
- GEM PATHS:
- /var/lib/gems/2.1.0
- /home/ashwin/.gem/ruby/2.1.0
- /usr/share/rubygems-integration/2.1.0
- /usr/share/rubygems-integration/2.1
- /usr/share/rubygems-integration/all
- GEM CONFIGURATION:
- :update_sources => true
- :verbose => true
- :backtrace => false
- :bulk_threshold => 1000
- REMOTE SOURCES:
- https://rubygems.org/
- SHELL PATH:
- /usr/local/heroku/bin
- /usr/local/sbin
- /usr/local/bin
- /usr/sbin
- /usr/bin
- /sbin
- /bin
- /usr/games
- /usr/local/games
Just like you my ruby version was correct, but the fix for me was reinstalling bundler.
sudo gem install bundler
Try installing without sudo.
I've had the same issue after upgrading Ruby to 2.2.3 with rbenv (which is installed without admin rights).
After having upgraded, I ran sudo gem install github-pages, which resulted in the above error. Running gem install github-pages worked just fine.
Then, trying to have Jekyll serve my page, I ran bundle exec jekyll serve, which told that there were some bundles missing. bundle install fixed that too, and bundle exec jekyll serve worked fine.
Make sure that gem install public_suffix -v '1.5.1' succeeds before bundling.
I had the same issue and this command solves it.
sudo gem install github-pages -v 33
Then run jekyll.
bundle exec jekyll serve
See here
In the past i've also had issues with installing github pages or Jekyll dependencies because there is a space somewhere in the full path to your project folder and apparrently some of the dependencies cant handle spaces in filenames.
Most recently this seems to have been happening to me with public_suffix version 4.0.5 where a space in my username causes it to break
This has also happened with http_parser

Install Bundler gem using Ansible

I am trying to install Bundler on my VPS using Ansible.
I already have rbenv set up and the global ruby is 2.1.0.
If I SSH as root into the server and run gem install bundler, it installs perfectly.
I have tried the following three ways of using Ansible to install the Bundler gem and all three produce no errors, but when I SSH in and run gem list, Bundler is nowhere to be seen.
Attempt 1:
---
- name: Install Bundler
shell: gem install bundler
Attempt 2:
---
- name: Install Bundler
shell: gem install bundler
Attempt 3:
---
- name: Install Bundler
gem: name=bundler
state=latest
I have also tried the last attempt with user_install=yes and also with user_install=no and neither make any difference.
Any ideas how I can get it to install Bundler correctly via Ansible?
I've been working on this for a little while now and I have 1 ruby version installed: 2.1.0 and ahve found that the shims directory for rbenv does not contain a shim for bundle.
Should a shim for bundle be in there? I'm just getting confused as to why capistrano cannot find the bundle command as it's listed when I run sudo gem list but NOT when I run gem list?
root#weepingangel:/usr/local/rbenv/shims# echo $PATH
/usr/local/rbenv/shims:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
root#weepingangel:/usr/local/rbenv/shims# gem environment
RubyGems Environment:
- RUBYGEMS VERSION: 2.2.0
- RUBY VERSION: 2.1.0 (2013-12-25 patchlevel 0) [x86_64-linux]
- INSTALLATION DIRECTORY: /usr/local/rbenv/versions/2.1.0/lib/ruby/gems/2.1.0
- RUBY EXECUTABLE: /usr/local/rbenv/versions/2.1.0/bin/ruby
- EXECUTABLE DIRECTORY: /usr/local/rbenv/versions/2.1.0/bin
- SPEC CACHE DIRECTORY: /root/.gem/specs
- RUBYGEMS PLATFORMS:
- ruby
- x86_64-linux
- GEM PATHS:
- /usr/local/rbenv/versions/2.1.0/lib/ruby/gems/2.1.0
- /root/.gem/ruby/2.1.0
- GEM CONFIGURATION:
- :update_sources => true
- :verbose => true
- :backtrace => false
- :bulk_threshold => 1000
- :sources => ["http://gems.rubyforge.org", "http://gems.github.com"]
- "gem" => "--no-ri --no-rdoc"
- REMOTE SOURCES:
- http://gems.rubyforge.org
- http://gems.github.com
- SHELL PATH:
- /usr/local/rbenv/versions/2.1.0/bin
- /usr/local/rbenv/libexec
- /usr/local/rbenv/shims
- /usr/local/sbin
- /usr/local/bin
- /usr/sbin
- /usr/bin
- /sbin
- /bin
- /usr/games
Any ideas?
So, I think the two main problems I have:
Why is bundler only visible when I run sudo gem list?
My deploy is saying:
INFO [18d5838c] Running /usr/bin/env bundle install --binstubs
/var/rails_apps/neiltonge/shared/bin --path
/var/rails_apps/neiltonge/shared/bundle --without development test
--deployment --quiet on 188.226.159.96 DEBUG [18d5838c] Command: cd /var/rails_apps/neiltonge/releases/20140301205432 && ( PATH=$PATH
/usr/bin/env bundle install --binstubs
/var/rails_apps/neiltonge/shared/bin --path
/var/rails_apps/neiltonge/shared/bundle --without development test
--deployment --quiet ) DEBUG [18d5838c] /usr/bin/env: bundle: No such file or directory
and this is my $PATH:
/usr/local/rbenv/shims:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
Why can't bundle be located?
The problem is that, when running gem install bundler via ansible, you're not initializing rbenv properly, since rbenv init is run in .bashrc or .bash_profile. So the gem command used is the system one, not the one installed as a rbenv shim. So whenever you install a gem, it is installed system-wide, not in your rbenv environment.
To have rbenv initialized properly, you must execute bash itself and explicitely state that it's a login shell, so it reads it's initialization files :
ansible your_host -m command -a 'bash -lc "gem install bundler"' -u your_rbenv_user
Leave the -u your_rbenv_user part if you really want to do this as root.
If the above command works, you can easily turn it into a playbook action :
- name: Install Bundler
become_user: your_rbenv_user
command: bash -lc "gem install bundler"
It's cumbersome, but it's the only way I found so far.
Since Ansible 1.3 following native solution is possible:
- name: requirements for installing gems
apt:
name: {{ item }}
with_items:
- ruby
- ruby-dev
- make
- name: install gem with proper $PATH
gem:
name: xyz
user_install: no
Mention the user_install parameter! Additionally some dependecies installed by the bundler could need following further package dependencies:
zlib1g-dev
I've met the similar environment issue when I tried to run commands as another user. As mentioned in this feature request you have two options to execute your command in login shell (that will load user environment). For example i'ld like to install bundler as rails user:
- name: Install Bundler
shell: gem install bundler
sudo_user: rails -i
or
- name: Install Bundler
command: sudo -iu rails gem install bundler
This Worked for me:
- name: rubygems | install bundler
shell: gem install bundler
- name: rbenv | rehash
shell: RBENV_ROOT={{ rbenv_root }} rbenv rehash
Sometimes after installing bundler, with rbenv on the system, you need to update your $PATH by runing rbenv rehash. I just tried the same thing with ansible, and it worked. Bundler is available in my $PATH after rehash.
The cleanest and quickest way to install bundler using Ansible is this:
Simply install rbenv by using the role https://github.com/zzet/ansible-rbenv-role and by configuring its plugins like so (obviously, there are more parameters to configure than just the plugins):
rbenv_plugins:
- { name: 'ruby-build',
repo: 'https://github.com/rbenv/ruby-build.git',
version: master }
- { name: 'rbenv-default-gems',
repo: 'https://github.com/rbenv/rbenv-default-gems.git',
version: master }
The included plugin rbenv-default-gems will add bundler by default and into the right directory during the installation process of the version of ruby you will have spcecified.
Then make sure bundler is in PATH.
That's it.
I got it working like this:
- name: Install jekyll and bundler
become_user: bob
gem:
name: "{{ item }}"
environment:
GEM_HOME: /home/bob/gems
PATH: $PATH:/bin/:/usr/bin/:/home/bob/gems/bin
with_items:
- jekyll
- bundler
Replace bob with your local user.
And then use the same principle with the bundler module
- name: Install Gems
become_user: bob
bundler:
gemfile: /home/bob/Gemfile
state: present
environment:
GEM_HOME: /home/bob/gems
PATH: $PATH:/bin/:/usr/bin/:/home/bob/gems/bin

Resources