CircleCi Build is failing due to the wrong Ruby version - ruby

We originally were using Ruby 2.2.3 for our application, however, now I'm updating the gems due to vulnerabilities and as a result, I have to update the ruby version that the application is running on. Locally, this was fairly straight forward (using rbenv), however, when I push it to GitHub, it fails the CircleCi build because it's pulling the ruby version from an old docker image that's using the old ruby version.
So far, I've pulled the old docker image down using docker pull, ran the image using docker run -it, opened it in VSCode using the Remote Extension, and downloaded rbenv, updated the ruby version, committed the changes with docker commit, pushed it to GitHub with docker push, and tested it but it's still failing the CircleCi build with an error saying that the updated gems need to use the updated version of ruby.
config.yml:
version: 2.1
jobs:
build:
docker:
- image: 073991710092.dkr.ecr.us-east-1.amazonaws.com/signage-dev:latest-v2
environment:
RAILS_LOG_TO_STDOUT: 'true'
RAILS_ENV: test
APP_ENV: development
DB_USERNAME: *******
DB_PASSWORD: *******
DB_HOST: mysql
CACHE_HOST: redis
TZ: "America/New_York"
- image: mysql:5.6.37
name: mysql
environment:
MYSQL_ROOT_PASSWORD: *******
MYSQL_USER: *******
MYSQL_PASSWORD: *******
MYSQL_DATABASE: *******
- image: redis:2.8.6
name: redis
command: redis-server
steps:
- checkout
- run:
name: setup
command: |
mkdir app/log
touch app/log/oink.log
chmod a+rw app/log/oink.log
touch app/rspec.xml
mkdir -p app/tmp/cache/assets/test/sprockets/v3.0
- run:
name: setup database
command: (cd ./app && gem install bundler && bundle install && bundle exec rake --trace db:setup)
- run:
name: run tests
command: |
cd ./app
mkdir /tmp/test-results
TEST_FILES="$(circleci tests glob "spec/**/*_spec.rb" | circleci tests split --split-by=timings)"
bundle exec rspec --format progress \
--format RspecJunitFormatter \
--out /tmp/test-results/rspec.xml \
--format progress \
$TEST_FILES
- store_test_results:
path: /tmp/test-results
- store_artifacts:
path: /tmp/test-results
destination: test-results
Here's what happens when I open the image in a container and check the ruby version:
docker run -it 073991710092.dkr.ecr.us-east-1.amazonaws.com/signage-dev:latest-v2 bash
root#836591d2e6bf:/# ls
Gemfile bin dev home lib media opt root sbin sys usr
Gemfile.lock boot etc index.html lib64 mnt proc run srv tmp var
root#836591d2e6bf:/# ruby -v
ruby 2.6.0p0 (2018-12-25 revision 66547) [x86_64-linux]
Error message in CircleCi (Setup Database):
#!/bin/bash -eo pipefail
(cd ./app && gem install bundler && bundle install && bundle exec rake --trace db:setup)
ERROR: Error installing bundler:
bundler requires Ruby version >= 2.3.0.
Exited with code exit status 1
CircleCI received exit code 1

Related

Can not trigger build with indication of "Not Run" in CircleCI

I am planning to migrate my existing CI to CircleCI 2 in favor of the docker. Once I tried to trigger a build it's shown something like this:
I also checked the steps as indicated in the below alert here https://circleci.com/sunset1-0/
Service alert: Your project references CircleCI 1.0 or it has no
configuration. CircleCI 1.0 and projects without configuration files
are no longer supported. You must update your project to use CircleCI
2.0 configuration to continue. Learn more.
Is there anything I missed out?
Below is my .circleci/config.yml
version: 2 # use CircleCI 2.0
jobs: # a collection of steps
build: # runs not using Workflows must have a `build` job as entry point
parallelism: 3 # run three instances of this job in parallel
docker: # run the steps with Docker
- image: circleci/ruby:2.4.2-jessie-node # ...with this image as the primary container; this is where all `steps` will run
environment: # environment variables for primary container
BUNDLE_JOBS: 3
BUNDLE_RETRY: 3
BUNDLE_PATH: vendor/bundle
RAILS_ENV: test
- image: mysql:5.7 # database image
environment: # environment variables for database
MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
steps: # a collection of executable commands
- checkout # special step to check out source code to working directory
- run:
name: setup
command: |
curl -sL https://deb.nodesource.com/setup_10.x | bash \
&& curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - \
&& echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
- run:
name: Dependencies
command: |
apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get install -y \
build-essential mysql-client nodejs yarn && \
apt-get clean && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
# Which version of bundler?
- run:
name: Which bundler?
command: bundle -v
# Restore bundle cache
# Read about caching dependencies: https://circleci.com/docs/2.0/caching/
- restore_cache:
keys:
- rails-demo-bundle-v2-{{ checksum "Gemfile.lock" }}
- rails-demo-bundle-v2-
- run: # Install Ruby dependencies
name: Bundle Install
command: bundle check --path vendor/bundle || bundle install --deployment
# Store bundle cache for Ruby dependencies
- save_cache:
key: rails-demo-bundle-v2-{{ checksum "Gemfile.lock" }}
paths:
- vendor/bundle
# Only necessary if app uses webpacker or yarn in some other way
- restore_cache:
keys:
- rails-demo-yarn-{{ checksum "yarn.lock" }}
- rails-demo-yarn-
- run:
name: Yarn Install
command: yarn install --cache-folder ~/.cache/yarn
# Store yarn / webpacker cache
- save_cache:
key: rails-demo-yarn-{{ checksum "yarn.lock" }}
paths:
- ~/.cache/yarn
- run:
name: Wait for DB
command: dockerize -wait tcp://localhost:5432 -timeout 1m
- run:
name: Database setup
command: bin/rails db:schema:load --trace
- run:
name: Run rspec in parallel
command: |
bundle exec rspec --profile 10 \
--format RspecJunitFormatter \
--out test_results/rspec.xml \
--format progress \
$(circleci tests glob "spec/**/*_spec.rb" | circleci tests split --split-by=timings)
# Save test results for timing analysis
- store_test_results: # Upload test results for display in Test Summary: https://circleci.com/docs/2.0/collect-test-data/
path: test_results
# See https://circleci.com/docs/2.0/deployment-integrations/ for example deploy configs
In order for the build work, your config.yml file must live inside a .circleci folder at the root of your project directory.
You post a very good description. Thanks for the code. The version config looks fine. However, you wrote that your config currently lives at .circle/config.yml (Notice the missing ci at end of circle)
Move the config.yml to .circleci/config.yml and the build should proceed further. (right now the build is failing because it has no configuration.
E

Postgres installation on Docker Ruby Alpine

I'm trying to complete the setup for my Rails application, using Docker. For development I use docker-compose.yml, and for production just the Dockerfile, as I want to deploy using Dokku (which has no support for compose).
In development everything is running fine, I'm getting my containers (app, webpacker and postgres), but in production mode my postgres container is missing... and also Dokku can't deploy correctly. I tried many ways, but nothing seems to work :(
Dockerfile
FROM ruby:2.5.5-alpine
ARG PRECOMPILEASSETS
ENV NODE_OPTIONS "--max_old_space_size=4096"
ENV SECRET_KEY_BASE=foo
RUN apk add --update --no-cache \
build-base \
git \
postgresql-dev \
postgresql-client \
imagemagick \
nodejs-current \
yarn \
python2 \
tzdata \
file
RUN gem install bundler
# Install gems
RUN mkdir /gems
WORKDIR /gems
COPY Gemfile .
COPY Gemfile.lock .
RUN bundle install -j4 --retry 3 \
# Remove unneeded files (cached *.gem, *.o, *.c)
&& rm -rf /usr/local/bundle/cache/*.gem \
&& find /usr/local/bundle/gems/ -name "*.c" -delete \
&& find /usr/local/bundle/gems/ -name "*.o" -delete
RUN yarn install
ARG INSTALL_PATH=/beweeg
ENV INSTALL_PATH $INSTALL_PATH
WORKDIR $INSTALL_PATH
COPY . .
# Precompile assets (or not)
RUN docker/potential_asset_precompile.sh $PRECOMPILEASSETS
# Expose Puma port
EXPOSE 3000
CMD ["docker/startup.sh"]
docker-compose.yml
version: '3.0'
services:
db:
image: postgres:11-alpine
ports:
- 5433:5432
environment:
POSTGRES_PASSWORD: postgres
webpacker:
image: beweeg_development
command: bin/webpack-dev-server
volumes:
- .:/beweeg:cached
ports:
- 3035:3035
app:
image: beweeg_development
build:
context: .
args:
- PRECOMPILEASSETS=NO
links:
- db
- webpacker
ports:
- 3000:3000
volumes:
- .:/beweeg:cached
As you'll see using docker-compose makes no problem, but if I want to build and run as production (without compose), then I cannot get pg installed correctly.
Previously I was using a Ruby slim image, and then using the following command:
RUN apt-get update && apt-get install -y curl gnupg
RUN echo "deb http://apt.postgresql.org/pub/repos/apt/ stretch-pgdg main" > /etc/apt/sources.list.d/pgdg.list
RUN curl -q https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
was working nicely... but in order to reduce my images sizes, I'd prefer to stay now with Alpine.
I'm quite a noob, so please forgive me if the answer seems obvious... And thanks in advance for your help!

Bundler locks in docker + can't install gems (Solved- docker CMD vs ENTRYPOINT)

Update: I've narrowed the [or a?] problem down to the line - groupy-gemcache:/usr/local/bundle in my services: app: volumes dictionary. If I remove it, the container runs fine [i think] but presumably i lose my local gem cacheing.
tldr: After running docker-compose build, things seem ok, but I cannot run any gem or bundle inside my running docker container if I add something to my gemfile. for example, after docker-compose build && docker-compose run app bash:
root#2ea58aff612e:/src# bundle check
The Gemfile's dependencies are satisfied
root#2ea58aff612e:/src# echo 'gem "hello-world"' >> Gemfile
root#2ea58aff612e:/src# bundle
Could not find gem 'hello-world' in any of the gem sources listed in your Gemfile.
Run `bundle install` to install missing gems.
root#2ea58aff612e:/src# bundle install
Could not find gem 'hello-world' in any of the gem sources listed in your Gemfile.
Run `bundle install` to install missing gems.
root#2ea58aff612e:/src# gem
Could not find gem 'hello-world' in any of the gem sources listed in your Gemfile.
Run `bundle install` to install missing gems.
root#2ea58aff612e:/src# gem env
Could not find gem 'hello-world' in any of the gem sources listed in your Gemfile.
Run `bundle install` to install missing gems.
I'm still pretty novice with docker, but i've been trying to configure a perfect dockerfile that can build, cache gems and update while still committing its Gemfile.lock to git [maybe this isn't actually perfect and I'm open to suggestions there]. In my use case, I'm using a docker compose file with images for a rails app and sidekiq worker as well as postgres and redis images- very simliar to the setups described here and here.
My Dockerfile [some things commented out that I cobbled from the tutorials above:
FROM ruby:2.3
ENTRYPOINT ["bundle", "exec"]
ARG bundle_path
# throw errors if Gemfile has been modified since Gemfile.lock
# RUN bundle config --global frozen 1
ENV INSTALL_PATH /src
RUN mkdir -p $INSTALL_PATH
WORKDIR $INSTALL_PATH
# Install dependencies:
# - build-essential: To ensure certain gems can be compiled
# - nodejs: Compile assets
# - npm: Install node modules
# - libpq-dev: Communicate with postgres through the postgres gem
# - postgresql-client-9.4: In case you want to talk directly to postgres
RUN apt-get update && apt-get install -qq -y build-essential nodejs npm libpq-dev postgresql-client-9.4 --fix-missing --no-install-recommends && \
rm -rf /var/lib/apt/lists/*
COPY app/Gemfile app/Gemfile.lock ./
# Bundle and then save the updated Gemfile.lock in our volume since it will be clobbered in the next step
RUN echo $bundle_path && bundle install --path=$bundle_path && \
cp Gemfile.lock $bundle_path
# ./app contains the rails app on host
COPY app .
# Unclobber the updated gemfile.lock
RUN mv $bundle_path/Gemfile.lock ./
CMD ["./script/start.sh"]
docker-compose.yml:
version: '2'
volumes:
groupy-redis:
groupy-postgres:
groupy-gemcache:
services:
app:
build:
args:
bundle_path: /usr/local/bundle
context: .
dockerfile: Dockerfile
command: "./script/start.sh"
links:
- postgres
- redis
volumes:
- ./app:/src
- groupy-gemcache:/usr/local/bundle
ports:
- '3000:3000'
env_file:
- .docker.env
stdin_open: true
tty: true
Wow, I figured it out. The problem was coming from my ENTRYPOINT dockerfile command, as explained at the end of this article on CMD vs ENTRYPOINT.
I believe the result of ENTRYPOINT ["bundle", "exec"] with CMD ["./script/start.sh"] OR docker-compose run app bash was to run a command like bundle exec 'bash'. I confirmed this by removing the entrypoint from the dockerfile, entering the shell as above and manually running bundle exec 'bash' and sure enough i landed in a subshell where I couldn't run any bundle or gem commands- had to exit twice to leave.

Bundler can't see gems installed in dockerized environment

I have a pretty basic "app" which i'm trying to dockerize using some examples found on the internet but, for some reason, it seems that bundler can't see the gems installed in my env.
Gemfile
source 'https://rubygems.org'
gem 'streamio-ffmpeg'
Dockerfile
FROM dpsxp/ffmpeg-with-ruby:2.2.1
MAINTAINER mike
RUN apt-get update && apt-get install -y \
build-essential \
cmake
ENV APP_HOME=/app
RUN mkdir -p $APP_HOME
WORKDIR $APP_HOME
COPY Gemfile* ./
ENV BUNDLE_GEMFILE=$APP_HOME/Gemfile \
BUNDLE_PATH=/bundle
RUN gem install bundler
RUN bundle install --jobs 20 --retry 5 --path=/bundle
COPY . ./
CMD ["rake"]
docker-compose.yml
version: '2'
services:
app:
build:
context: .
volumes:
- '.:/app'
depends_on:
- bundle
bundle:
image: busybox
volumes:
- /bundle
And now, when i'm running $ docker-compose run app gem list, the streamio-ffmpeg gem is not on the list for some reason. Also, when i'm trying to require it inside my ruby app, i'm getting LoadError: cannot load such file
Build logs: https://gist.github.com/mbajur/569b4f221cadb8a85ecc5dae8a595401
When you deal with gems by bundler, all related commands should be run with bundle exec, so in your case, it should work with below command:
docker-compose run app bundle exec gem list
Bundler provides a consistent environment for Ruby projects by tracking and installing the exact gems and versions that are needed.
The command you run directly with gem list is to list the system gems, of course they are not in list.

How to access a installed gem inside travis

I have created a rubygem which can installed by:
* gem install youtube_dlhelper
* cd .rvm/gems/ruby-2.2.1/gems/youtube_dlhelper-0.1.9.5
* rake setup
I set up my .travis.yml:
os:
-linux
dist: trusty
language: ruby
rvm:
- 2.2.1
- 2.2.2
env:
global:
- JRUBY_OPTS=-Xcext.enabled=true
- CI=true
- TRAVIS=true
- LANG=en_US.UTF-8
- LC_ALL=en_US.UTF-8
before_install:
- sudo apt-get update
- sudo apt-get install ffmpeg
before_script:
- cd $GEM_HOME/gems/youtube_dlhelper-0.1.9.5
- rake setup
after_success:
- CI=true TRAVIS=true coveralls --verbose
notifications:
email:
- samannsml#directbox.com
But travis don't find the directory. Also i tried:
before_script:
- cd $GEM_HOME/gems
- ls
But it looks like the directory is empty.
Maybe anyone has a solution for this problem?
I have seen that the ci uses the git code directly. So it is enough to use
before_script:
- rake setup
So it runs :-)

Resources