Configure Dockerfile to Run Cron Tasks using Whenver and Rake - ruby

I want to create a container using Docker which is will be responsible for starting recurrent rake tasks based on the Whenever gem's configuration. I have a plain ruby project (without rails/sinatra) with the following structure:
Gemfile:
source 'https://rubygems.org'
gem 'rake', '~> 12.3', '>= 12.3.1'
gem 'whenever', '~> 0.9.7', require: false
group :development, :test do
gem 'byebug', '~> 10.0', '>= 10.0.2'
end
group :test do
gem 'rspec', '~> 3.5'
end
config/schedule.rb: (whenever's configuration)
ENV.each { |k, v| env(k, v) }
every 1.minutes do
rake 'hello:start'
end
lib/tasks/hello.rb: (rake configuration)
namespace :hello do
desc 'This is a sample'
task :start do
puts 'start something!'
end
end
Dockerfile:
FROM ruby:2.5.3-alpine3.8
RUN echo "http://dl-cdn.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories && \
apk update && apk upgrade && \
apk add build-base bash dcron && \
apk upgrade --available && \
rm -rf /var/cache/apk/* && \
mkdir /usr/app
WORKDIR /usr/app
COPY Gemfile* /usr/app/
RUN bundle install
COPY . /usr/app
RUN bundle exec whenever --update-crontab
CMD ['sh', '-c', 'crond && gulp']
I've used the following resources to get at the this point
How to run a cron job inside a docker container
https://github.com/renskiy/cron-docker-image/blob/master/alpine/Dockerfile
https://stackoverflow.com/a/43622984/5171758 <- very close to I want but no success
If I call my rake task using command line, I get the result I want.
$ rake 'hello:start'
start something!
However, I can't figure out how to make it work using Docker. The container is build but no log is written, no output is shown, nothing happens. Can someone help me showing what I'm doing wrong?
building commands
docker build -t gsc:0.0.1 .
docker container run -a stdin -a stdout -i --net host -t gsc:0.0.1 /bin/bash
Thanks all. Cheers

This is the solution to the problem I listed above. I had some issues at the Dockerfile and schedule.rb. This is what I had to change to make it work correctly.
Dockerfile
wrong echo call
wrong bundle command
change ENTRYPOINT instead of CMD
FROM ruby:2.5.3-alpine3.8
RUN apk add --no-cache --repository http://dl-cdn.alpinelinux.org/alpine/edge/main && \
apk update && apk upgrade && \
apk add build-base bash dcron && \
apk upgrade --available && \
rm -rf /var/cache/apk/* && \
mkdir /usr/app
WORKDIR /usr/app
COPY Gemfile* /usr/app/
RUN bundle install
COPY . /usr/app
RUN bundle exec whenever -c && bundle exec whenever --update-crontab && touch ./log/cron.log
ENTRYPOINT crond && tail -f ./log/cron.log
config/schedule.rb
no need to ENV.each
every 1.minutes do
rake 'hello:start'
end
UPDATE
I've created a GitHub repository and a Docker Hub repository to share with the community this progress.

Related

How to run cronjobs and rackup

My problem is when run cron and rackup service for ruby sinatra in docker.
file cronjobs
* * * * * cd /app && rake parser >> cron.log 2>&1
file Dockerfile
RUN apk update && apk upgrade
RUN apk add --update build-base \
mariadb-dev bash dcron
RUN gem install bundler
WORKDIR /app
COPY Gemfile .
RUN bundle install && bundle clean
COPY . /app
COPY cronjobs /etc/crontabs/root
EXPOSE 80
CMD crond -f && rackup --host 0.0.0.0 -p 80
When run docker only one service is functional
Docker container is running while main process inside it is running. So if you want to run two services inside docker container, one of them has to be run in a background mode.
So, CMD layer should be the following:
CMD ( crond -f & ) && rackup --host 0.0.0.0 -p 80

How to install ruby and bundler with dockerfile?

New to ruby and bundler here.
I am installing them in a docker image with this docker file:
FROM alpine:3.5
# Install Ruby, Ruby Bundler and other ruby dependencies
RUN apk add --update \
ruby ruby-bigdecimal ruby-bundler \
ca-certificates libressl \
libressl-dev build-base ruby-dev \
ruby-rdoc ruby-io-console ruby-irb; \
\
&& bundle config build.nokogiri --use-system-libraries; \
&& bundle config git.allow_insecure true; \
\
&& gem install json foreman --no-rdoc --no-ri; \
&& gem cleanup; \
&& rm -rf /usr/lib/ruby/gems/*/cache/*; \
&& apk del libressl-dev build-base ruby-dev; \
&& rm -rf /var/cache/apk/* /tmp;
CMD ["bundle"]
When I run do a docker run I get:
Don't run Bundler as root. Bundler can ask for sudo if it is needed,
and installing your bundle as root will break this application for all
non-root users on this machine.
Could not locate Gemfile or .bundle/ directory
How do I resolve this? I just want to install ruby and ruby-bundle and be done with this ...
There are pre built ruby images (e.g Alpine 3.11 Ruby 2.7) that include bundler. It's easier to start with them as they generally use the current "best practices" to build.
Notice that they set the BUNDLE_SILENCE_ROOT_WARNING environment variable with ENV directive in the image build to remove that root warning.
You normally wouldn't run bundler as the CMD for a container either, you might run bundler during a RUN image build step though.
Running containers as non-root users is not a bad idea in any case. Use the USER directive to change that.
FROM ruby:2.7
WORKDIR /app
ADD . /app/
RUN set -uex; \
bundle install; \
adduser -D rubyapp; \
mkdir -p /app/data; \
chown rubyapp /app/data
USER rubyapp
CMD [ "ruby", "whatever.rb" ]

Bundler locks in docker + can't install gems (Solved- docker CMD vs ENTRYPOINT)

Update: I've narrowed the [or a?] problem down to the line - groupy-gemcache:/usr/local/bundle in my services: app: volumes dictionary. If I remove it, the container runs fine [i think] but presumably i lose my local gem cacheing.
tldr: After running docker-compose build, things seem ok, but I cannot run any gem or bundle inside my running docker container if I add something to my gemfile. for example, after docker-compose build && docker-compose run app bash:
root#2ea58aff612e:/src# bundle check
The Gemfile's dependencies are satisfied
root#2ea58aff612e:/src# echo 'gem "hello-world"' >> Gemfile
root#2ea58aff612e:/src# bundle
Could not find gem 'hello-world' in any of the gem sources listed in your Gemfile.
Run `bundle install` to install missing gems.
root#2ea58aff612e:/src# bundle install
Could not find gem 'hello-world' in any of the gem sources listed in your Gemfile.
Run `bundle install` to install missing gems.
root#2ea58aff612e:/src# gem
Could not find gem 'hello-world' in any of the gem sources listed in your Gemfile.
Run `bundle install` to install missing gems.
root#2ea58aff612e:/src# gem env
Could not find gem 'hello-world' in any of the gem sources listed in your Gemfile.
Run `bundle install` to install missing gems.
I'm still pretty novice with docker, but i've been trying to configure a perfect dockerfile that can build, cache gems and update while still committing its Gemfile.lock to git [maybe this isn't actually perfect and I'm open to suggestions there]. In my use case, I'm using a docker compose file with images for a rails app and sidekiq worker as well as postgres and redis images- very simliar to the setups described here and here.
My Dockerfile [some things commented out that I cobbled from the tutorials above:
FROM ruby:2.3
ENTRYPOINT ["bundle", "exec"]
ARG bundle_path
# throw errors if Gemfile has been modified since Gemfile.lock
# RUN bundle config --global frozen 1
ENV INSTALL_PATH /src
RUN mkdir -p $INSTALL_PATH
WORKDIR $INSTALL_PATH
# Install dependencies:
# - build-essential: To ensure certain gems can be compiled
# - nodejs: Compile assets
# - npm: Install node modules
# - libpq-dev: Communicate with postgres through the postgres gem
# - postgresql-client-9.4: In case you want to talk directly to postgres
RUN apt-get update && apt-get install -qq -y build-essential nodejs npm libpq-dev postgresql-client-9.4 --fix-missing --no-install-recommends && \
rm -rf /var/lib/apt/lists/*
COPY app/Gemfile app/Gemfile.lock ./
# Bundle and then save the updated Gemfile.lock in our volume since it will be clobbered in the next step
RUN echo $bundle_path && bundle install --path=$bundle_path && \
cp Gemfile.lock $bundle_path
# ./app contains the rails app on host
COPY app .
# Unclobber the updated gemfile.lock
RUN mv $bundle_path/Gemfile.lock ./
CMD ["./script/start.sh"]
docker-compose.yml:
version: '2'
volumes:
groupy-redis:
groupy-postgres:
groupy-gemcache:
services:
app:
build:
args:
bundle_path: /usr/local/bundle
context: .
dockerfile: Dockerfile
command: "./script/start.sh"
links:
- postgres
- redis
volumes:
- ./app:/src
- groupy-gemcache:/usr/local/bundle
ports:
- '3000:3000'
env_file:
- .docker.env
stdin_open: true
tty: true
Wow, I figured it out. The problem was coming from my ENTRYPOINT dockerfile command, as explained at the end of this article on CMD vs ENTRYPOINT.
I believe the result of ENTRYPOINT ["bundle", "exec"] with CMD ["./script/start.sh"] OR docker-compose run app bash was to run a command like bundle exec 'bash'. I confirmed this by removing the entrypoint from the dockerfile, entering the shell as above and manually running bundle exec 'bash' and sure enough i landed in a subshell where I couldn't run any bundle or gem commands- had to exit twice to leave.

Running a cucumber test on firefox in a docker container

I'm trying to run my cucumber tests in a docker container using firefox. I'm getting this error when my code just tries to visit 'google.com'
Net::ReadTimeout (Net::ReadTimeout)
Here is my Dockerfile
FROM ruby:2.2.2
RUN mkdir /app
WORKDIR /app
RUN gem update
ADD Gemfile /app/Gemfile
ADD Gemfile.lock /app/Gemfile.lock
RUN bundle install
RUN apt-get update && apt-get install -y --fix-missing iceweasel xvfb unzip
ENV GECKODRIVER_VERSION v0.13.0
RUN echo $GECKODRIVER_VERSION
RUN mkdir -p /opt/geckodriver_folder
RUN wget -O /tmp/geckodriver_linux64.tar.gz https://github.com/mozilla/geckodriver/releases/download/$GECKODRIVER_VERSION/geckodriver-$GECKODRIVER_VERSION-linux64.tar.gz
RUN tar xf /tmp/geckodriver_linux64.tar.gz -C /opt/geckodriver_folder
RUN rm /tmp/geckodriver_linux64.tar.gz
RUN chmod +x /opt/geckodriver_folder/geckodriver
RUN ln -fs /opt/geckodriver_folder/geckodriver /usr/local/bin/geckodriver
ADD features /app/features
I've tried increasing my read_timeout to 120 with no effect.
When i bash into the container and run firefox it says "no display specified"
Any suggestions ?
I was having the same issue. This worked for me: https://github.com/juandelgado/docker-headless-firefox-cucumber/blob/master/Dockerfile
FROM ruby:2.4-slim
MAINTAINER Juan Delgado <juan#ustwo.com>
RUN mkdir /home/ustwo
WORKDIR /home/ustwo
RUN apt-get update && \
apt-get install -y xvfb build-essential libffi-dev wget firefox-esr && \
wget https://github.com/mozilla/geckodriver/releases/download/v0.17.0/geckodriver-v0.17.0-linux64.tar.gz && \
tar -zxvf geckodriver-v0.17.0-linux64.tar.gz && \
chmod +x geckodriver && \
mv geckodriver /usr/local/bin && \
rm geckodriver-v0.17.0-linux64.tar.gz
COPY Gemfile Gemfile
COPY Gemfile.lock Gemfile.lock
RUN bundle
RUN ["cucumber", "--version"]

Whenever gem 'failed to load command: rake'

looking for some help.
I am running a rails app (v3.2.5) with the whenever gem (v0.9.7) and rake (v11.2.2). I am also doing this in a docker container image ruby:2.3 (cron was installed and bundle install was ran)
Here is my schedule.rb
set :environment, ENV['RAILS_ENV']
every '*/2 9,10,11,12,13,14,15,16 * * 1-5' do
rake "import_csv", output: {:error => 'log/import_csv_errors.log', :standard => 'log/import_csv.log'}'
end
note RAILS_ENV is set at container launch to development
Here is my cron job that is on the container after build (crontab -l):
# Begin Whenever generated tasks for: /usr/src/app/config/schedule.rb
*/2 9,10,11,12,13,14,15,16 * * 1-5 /bin/bash -l -c 'cd /usr/src/app && RAILS_ENV=development bundle exec rake import_csv --silent >> log/import_csv.log 2>> log/import_csv_errors.log'
# End Whenever generated tasks for: /usr/src/app/config/schedule.rb
When this cron job runs, the logs return:
import_csv_errors.log
Bundler::GemNotFound: Could not find rake-11.2.2 in any of the sources
/usr/local/lib/ruby/gems/2.3.0/gems/bundler-1.12.5/lib/bundler/spec_set.rb:95:in `block in materialize'
/usr/local/lib/ruby/gems/2.3.0/gems/bundler-1.12.5/lib/bundler/spec_set.rb:88:in `map!'
/usr/local/lib/ruby/gems/2.3.0/gems/bundler-1.12.5/lib/bundler/spec_set.rb:88:in `materialize'
/usr/local/lib/ruby/gems/2.3.0/gems/bundler-1.12.5/lib/bundler/definition.rb:140:in `specs'
/usr/local/lib/ruby/gems/2.3.0/gems/bundler-1.12.5/lib/bundler/definition.rb:185:in `specs_for'
/usr/local/lib/ruby/gems/2.3.0/gems/bundler-1.12.5/lib/bundler/definition.rb:174:in `requested_specs'
/usr/local/lib/ruby/gems/2.3.0/gems/bundler-1.12.5/lib/bundler/environment.rb:19:in `requested_specs'
/usr/local/lib/ruby/gems/2.3.0/gems/bundler-1.12.5/lib/bundler/runtime.rb:14:in `setup'
/usr/local/lib/ruby/gems/2.3.0/gems/bundler-1.12.5/lib/bundler.rb:95:in `setup'
/usr/local/lib/ruby/gems/2.3.0/gems/bundler-1.12.5/lib/bundler/setup.rb:19:in `<top (required)>'
/usr/local/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
/usr/local/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
import_csv.log
bundler: failed to load command: rake (/usr/local/bin/rake)
Now here is the odd thing. If I copy the cron job command:
/bin/bash -l -c 'cd /usr/src/app && RAILS_ENV=development bundle exec rake import_csv --silent >> log/import_csv.log 2>> log/import_csv_errors.log'
and run this in the container, it works fine, but if the cron job runs it, I get thos errors in the logs!!! I am at a lost here...
I've tried adding
env :PATH, ENV['PATH']
env :GEM_PATH, '/usr/local/bundle'
to the top of schedule.rb and I tried doing
command 'cd /usr/src/app && RAILS_ENV=development bundle exec rake import_csv --silent >> log/import_csv.log 2>> log/import_csv_errors.log'
Instead of using rake in the task and I get the same errors..
Any help is appriciated
I've fixed the same error by modifying the Dockerfile with:
RUN gem update --system 2.6.12
RUN gem install bundler --version 1.14.6
And schedule.rb:
ENV.each { |k, v| env(k, v) }
I have updated my rake version and it worked for me. Below are the steps I followed:
sudo bundle update rake
sudo bundle install
Open a Rakefile and replace the line rake/rdoctask with require 'rdoc/task'.
I resolved my issue by using a different image and building as I needed it instead of using docker hub image ruby:2.x.
Dockerfile (edited to fit thread):
FROM ubuntu:14.04
# Installs needed to run rails on ubuntu 14.04 (must use mysql 5.6 or 5.5):
RUN apt-get update && apt-get install -y apache2 curl git build-essential libmysqlclient-dev mysql-server-5.6 nodejs make
RUN apt-get update && apt-get install -y ruby-dev zlib1g-dev
RUN gem install rails --version 3.2.5 --no-ri --no-rdoc
# Update ruby to v2.2 (optional)
RUN apt-get install -y software-properties-common && apt-add-repository ppa:brightbox/ruby-ng
RUN apt-get update && apt-get install -y ruby2.2
# Install cron
RUN apt-get install -y cron
# Finish the build
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
# If you code is in the same directory you ran docker build then:
COPY . /usr/src/app
RUN bundle install
RUN whenever --update-crontab
CMD ["passenger start"]

Resources