DISCLAIMER: I am new to ruby,rspec etc. I am trying to find a test framework for my docker containers
I have a docker container that runs the aws cli. I have tested this manually and it work, as part of my tests I want to get the AWS version and check it.
I have created this test
it "aws is the correct version" do
expect(aws_version).to include("aws")
end
def aws_version
command("aws --version").stdout
end
When I run it I get which shows the exepected as blank, it looks like it is not running and returning anything
1) Dockerfile aws is the coorect version
Failure/Error: expect(aws_version).to include("aws")
expected "" to include "aws"
# ./tests_spec.rb:37:in `block (2 levels) in <top (required)>'
All my other tests run against the container correctly. I have included both my dockerfile and test_spec below
Dockerfile:
FROM ubuntu:16.04
ENV LAST_UPDATE=09-04-2017
#####################################################################################
# Current version is aws-cli/1.11.83 Python/2.7.12 Linux/4.4.0-75-generic botocore/1.5.46
#####################################################################################
RUN apt-get update && apt-get -y upgrade
RUN apt-get install python-pip -y
RUN pip install --upgrade pip
RUN pip install --upgrade awscli s3cmd python-magic
RUN export PATH=~/.local/bin:$PATH
RUN mkdir /root/.aws
COPY config /root/.aws
#COPY credentials /root/.aws
WORKDIR /root
ENTRYPOINT ["aws"]
CMD ["--version"]
test_spec.rb:
require "docker"
require "serverspec"
describe "Dockerfile" do
before(:all) do
#image = Docker::Image.build_from_dir('.')
set :os, family: :ubuntu
set :backend, :docker
set :docker_image, #image.id
#container = Docker::Container.create(
'Image' => #image.id,
)
#container.start
end
it "installs the right version Name of Ubuntu" do
expect(os_version).to include("xenial")
end
it "installs the right version of Ubuntu" do
expect(os_version).to include("Ubuntu 16.04.2")
end
it "installs required packages" do
expect(package("python-pip")).to be_installed
end
it "aws is the coorect version" do
expect(aws_version).to include("aws")
end
def aws_version
command("aws --version").stdout
end
def os_version
command("cat /etc/lsb-release").stdout
end
after(:all) do
#container.kill
#container.delete(:force => true)
end
end
Pretty simple (but took me a while to spot): for some reason the aws --version command pumps it's output onto stderr rather than stdout. So do this instead:
def aws_version
command("aws --version").stderr
end
Also, it's worth noting that all your container does by default is run the aws --version command and then exits. So as long as your specs run fast enough you'll be okay, but if they don't then they're going to fail randomly.
In your specs I'd override your image's default entrypoint/cmd and put in a simple while true; do sleep 100; done so that it stays alive until your after blocks kills the container.
Related
I got some issue with the gem 'mina'. If I do set :user, 'username', he tries to connect to the server via Username#xxx.... wich is not working if the user is not existing. My PC is name Username. So mina setup and mina deploy are not working.
Does someone know a solution?.
Thanks
Best regards
Matze
EDIT:
Gemfile:
gem 'mina'
After that I run bundle install and mina init
deploy.rb:
require 'mina/rails'
require 'mina/git'
# require 'mina/rbenv' # for rbenv support. (https://rbenv.org)
require 'mina/rvm' # for rvm support. (https://rvm.io)
# Basic settings:
# domain - The hostname to SSH to.
# deploy_to - Path to deploy into.
# repository - Git repo to clone from. (needed by mina/git)
# branch - Branch name to deploy. (needed by mina/git)
set :user, "user"
set :application_name, 'appname'
set :domain, 'xx.xxx.xxx.xxx'
set :deploy_to, '/var/www/user/appname'
set :repository, 'user#xx.xxx.xxx.xxx:/home/user/git/appname.git'
set :branch, 'master'
# Optional settings:
# set :user, 'user' # Username in the server to SSH to.
# set :port, '30000' # SSH port number.
# set :forward_agent, true # SSH forward_agent.
# Shared dirs and files will be symlinked into the app-folder by the 'deploy:link_shared_paths' step.
# Some plugins already add folders to shared_dirs like `mina/rails` add `public/assets`, `vendor/bundle` and many more
# run `mina -d` to see all folders and files already included in `shared_dirs` and `shared_files`
# set :shared_dirs, fetch(:shared_dirs, []).push('public/assets')
set :shared_files, fetch(:shared_files, []).push('config/database.yml', 'config/secrets.yml')
# This task is the environment that is loaded for all remote run commands, such as
# `mina deploy` or `mina rake`.
task :remote_environment do
# If you're using rbenv, use this to load the rbenv environment.
# Be sure to commit your .ruby-version or .rbenv-version to your repository.
# invoke :'rbenv:load'
# For those using RVM, use this to load an RVM version#gemset.
# invoke :'rvm:use', 'ruby-1.9.3-p125#default'
end
# Put any custom commands you need to run at setup
# All paths in `shared_dirs` and `shared_paths` will be created on their own.
task :setup do
# command %{rbenv install 2.3.0 --skip-existing}
end
desc "Deploys the current version to the server."
task :deploy do
# uncomment this line to make sure you pushed your local branch to the remote origin
# invoke :'git:ensure_pushed'
deploy do
# Put things that will set up an empty directory into a fully set-up
# instance of your project.
invoke :'git:clone'
invoke :'deploy:link_shared_paths'
invoke :'bundle:install'
invoke :'rails:db_migrate'
invoke :'rails:assets_precompile'
invoke :'deploy:cleanup'
on :launch do
in_path(fetch(:current_path)) do
command %{mkdir -p tmp/}
command %{touch tmp/restart.txt}
end
end
end
# you can use `run :local` to run tasks on local machine before of after the deploy scripts
# run(:local){ say 'done' }
end
# For help in making your deploy script, see the Mina documentation:
#
# - https://github.com/mina-deploy/mina/tree/master/docs
set :execution_mode, :exec if RbConfig::CONFIG['host_os'] =~ /mswin|mingw/
When I run now mina setup he print that:
$ mina setup
User#xx.xxx.xxx.xxx's password:
SSH Auth key working fine and git working fine he just puts User instead of user before the ip, what is not working because the user just exists with small letter. But my working machine is named User.
I'm trying to update an app (Rails 4.1.12; ruby 1.9.3p484) by deploying changes from development to production using capistrano. Below is log (I masked the IP address and suggested secret key).
Previously never had a problem running a deploy. Have searched and found posts with similar errors but have not found a solution. I did follow the instructions to add Devise.secret_key to the Devise initializer and restarted the application (dev and prod). Next attempt to deploy received same error with suggestion for different secret key.
All help, hints, suggestions are greatly appreciated. Thanks.
UPDATE:I just tried to deploy using
bundle exec cap production deploy
and did not get errors but still get errors when running
cap production deploy
UPDATE 2: I used Figaro to generate an application.yml file (and append to gitignore). In the devise.rb file I created a line config.secret_key = ENV["config.secret_key"] In the application.yml file I created a line config.secret_key: 'random_128_chr_string' I then restarted the development server and ran cap production deploy Unfortunately got same error.
cap production deploy
.
.
.
[deploy:migrate] Run `rake db:migrate`
[e0c2c251] Running bundle exec rake db:migrate as deployer#123.123.123.12
(Backtrace restricted to imported tasks)
cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing as deployer#123.123.123.12: rake exit status: 1
rake stdout: Nothing written
rake stderr: rake aborted!
Devise.secret_key was not set. Please add the following to your Devise initializer:
config.secret_key = 'looong_string_of_numbers_and_letters'
Please ensure you restarted your application after installing Devise or setting the key.
deploy.rb:
lock '>=3.2.1'
set :application, 'pepperslice'
set :repo_url, 'git#github.com:jeffaltman/pepperslice.git'
set :linked_dirs, %w{log tmp public/images/uploaded}
set :linked_files, %w{config/database.yml config/initializers/devise.rb config/initializers/smtp_settings.rb}
set :format, :pretty
set :log_level, :info
namespace :deploy do
desc 'Restart application'
task :restart do
on roles(:app), in: :sequence, wait: 5 do
execute "sudo /usr/sbin/service unicorn restart"
end
end
after :publishing, :restart
after :restart, :clear_cache do
on roles(:web), in: :groups, limit: 3, wait: 10 do
# Here we can do anything such as:
# within release_path do
# execute :rake, 'cache:clear'
# end
end
end
end
capfile:
# Load DSL and Setup Up Stages
require 'capistrano/setup'
# Includes default deployment tasks
require 'capistrano/deploy'
# Includes tasks from other gems included in your Gemfile
#
# For documentation on these, see for example:
#
# https://github.com/capistrano/rvm
# https://github.com/capistrano/rbenv
# https://github.com/capistrano/chruby
# https://github.com/capistrano/bundler
# https://github.com/capistrano/rails
#
# require 'capistrano/rvm'
# require 'capistrano/rbenv'
# require 'capistrano/chruby'
require 'capistrano/bundler'
# require 'capistrano/rails/assets'
require 'capistrano/rails/migrations'
#require 'capistrano/ssh_doctor'
# Loads custom tasks from `lib/capistrano/tasks' if you have any defined.
Dir.glob('lib/capistrano/tasks/*.rake').each { |r| import r }
I am configuring a custom SCM because i don't need the default git in the development local environment, but i would like to trigger a custom logic, mainly based on creating a release starting from a source_directory.
As described in the documentation (http://capistranorb.com/documentation/advanced-features/custom-scm/) i wrote a module that extends the Capistrano::Plugin, and set required methods to handle the custom SCM implementation used by deploy Capistrano flow.
Besides this, when i put in my config/deploy/<environment>.rb the entry:
set :scm, :<custom plugin name>
Capistrano keeps to use the default git scm, even if is not declared.
in my Capfile there are loaded both as follow:
require_relative 'scm/local.rb'
install_plugin Capistrano::LocalPlugin
require 'capistrano/git
install_plugin Capistrano::SCM::Git
Here also the module of custom SCM :
require 'capistrano/scm/plugin'
module Capistrano
class Capistrano::SCM::LocalPlugin < ::Capistrano::Plugin
def set_defaults
set_if_empty :source_dir, 'non-exisisting-dir'
end
def define_tasks
namespace :local do
task :create_release do
run_locally do
execute :mkdir, '-p', :'tmp'
execute "cd #{fetch(:source_dir)} && tar -cz --exclude tests --exclude vendor --exclude .git --exclude node_modules --exclude tmp/#{fetch(:release_timestamp)}.tar.gz -f tmp/#{fetch(:release_timestamp)}.tar.gz ."
end
on release_roles :all do
execute :mkdir, '-p', release_path
upload! "tmp/#{fetch(:release_timestamp)}.tar.gz", "#{release_path}/#{fetch(:release_timestamp)}.tar.gz"
execute "tar -xvf #{release_path}/#{fetch(:release_timestamp)}.tar.gz --directory #{release_path}"
execute "rm #{release_path}/#{fetch(:release_timestamp)}.tar.gz"
end
run_locally do
execute "rm -rf tmp"
end
end
desc 'Determine the revision that will be deployed'
task :set_current_revision do
run_locally do
set :current_revision, capture(:git, " --git-dir #{fetch(:source_dir)}/.git rev-parse --short #{fetch(:branch)}")
end
end
end
end
def register_hooks
after 'deploy:new_release_path', 'local:create_release'
end
end
end
Does anyone know which is the black magic to use in order to say to Capistrano to use my scm instead of the default git one ?
set :scm, 'myscm' is deprecated. Until the next major version of Capistrano (4.0), there is a class which checks for an SCM having been installed via install_plugin, and if not, checks for the set :scm definition. If install_plugin has been called, then set :scm is ignored and deleted.
install_plugin only registers the plugin. It looks to me from the code that Capistrano will run both plugins if two are installed.
So, in a nutshell, Capistrano doesn't support selecting multiple SCMs based on environment. The closest thing to that you could try is using an environment variable to conditionally load the SCM in your Capfile. Something like:
if ENV['CAP_SCM'] == 'local'
require_relative 'scm/local.rb'
install_plugin Capistrano::LocalPlugin
else
require 'capistrano/git'
install_plugin Capistrano::SCM::Git
end
This is all documented here: https://github.com/capistrano/capistrano/blob/master/UPGRADING-3.7.md
TL;DR: I can't delete my Docker image from inside my RSpec script, because RSpec still has a dependent container
I'm using RSpec to test my Dockerfiles. That is, to test whether the Dockerfile results in a correct image. My test script does this by building an image from the Dockerfile, and then letting RSpec test whether a container (based on that image) has the correct contents.
The execution flow is as follows:
My RSpec script builds the Docker image (in its before(:all) block)
Serverspec creates a Docker container (based off the Docker image)
RSpec / Serverspec executes the tests inside the Docker container
RSpec / Serverspec calls my RSpec script's after(:all) block
RSpec / Serverspec deletes the Docker container
I tried to delete the image from inside the after(:all) block (or the after(:context) block, but that has the same issue), as that is the last hook that RSpec offers me. Only I can't, because the Docker container has not been deleted yet.
Force-deletion is no solution, as Docker then keeps an unnamed, untagged version of the image around.
Is there an (idiomatic) way to accomplish this?
Example RSpec script:
require "serverspec"
require "docker-api"
describe "Dockerfile" do
image_name = 'my_image'
image_tag = 'my_tag'
image_name_tag = image_name + ':' + image_tag
before(:all) do
set :os, family: :debian
set :backend, :docker
# Build image
#image = Docker::Image.build_from_dir("../dockerfile/")
#image.tag(repo: image_name, tag: image_tag)
# Provide Serverspec with our image's id
set :docker_image, #image.id
end
it "is a Debian-based container" do
expect(os_version).to include("Debian")
end
def os_version
command("cat /etc/*release").stdout
end
after(:context) do
# Clean up
#image.delete(name: image_name_tag) # <-- this fails
end
end
(It's not entirely clear to me where RSpec's responsibilities end and Serverspec's begin; hence the RSpec / Serverspec above.)
Okay,
I'm sorry if the title is not descriptive enough, but allow me to explain what I want to achieve:
I have a Rails 3 application
During my deploy, it needs to call pg_dump with the correct parameters to restore a backup
The task needs to be ran after the deploy is done but before the migrations.
The problem I have however, is that for this task, I would like to access Rails specific code, which is not working as Capistrano keeps throwing a lot of errors at me like gems not available or module not defined.
This is my Rake task:
namespace :deploy do
namespace :rm do
desc 'Backup the database'
task :backup do
# Generates the command to invoke the Rails runner
# Used by the cfg method to execute the ActiveRecord configuration in the rails config.
def runner
dir = "#{fetch(:deploy_to)}/current"
bundler = "#{SSHKit.config.command_map.prefix[:bundle].first} bundle exec"
env = fetch(:rails_env)
"cd #{dir}; #{bundler} rails r -e #{env}"
end
def cfg(name)
env = fetch(:rails_env)
command = "\"puts ActiveRecord::Base.configurations['#{env}']['#{name}']\""
"#{runner} #{command}"
end
on roles(:db) do
timestamp = Time.now.strftime('%Y%m%d%H%M%S')
backups = File.expand_path(File.join(fetch(:deploy_to), '..', 'backups'))
execute :mkdir, '-p', backups
dump = "PGPASSWORD=`#{cfg('password')}` pg_dump -h `#{cfg'host')}` -U `#{cfg('username')}` `#{cfg('database')}`"
fn = "#{timestamp}_#{fetch(:stage)}.sql.gz"
path = File.join(backups, fn)
execute "#{dump} | gzip > #{path}"
end
end
end
end
In it's current form, it simply generates a string with the runner method and dumps that inside the cfg method.
I tried rewriting the runner method, but for some reason I keep getting the runner --help output from the remote server, but the command being generated in the output is correct, and works locally just fine.
We are using Ruby 2.2.2 and RVM on the remote server.
Is it even possible to do what we are trying to construct together?
I'd suggest writing a rake task inside your Rails app and invoking that from your Capistrano task. This is how the Capistrano rails tasks work.
within release_path do
with rails_env: fetch(:rails_env) do
execute :rake, "db:migrate"
end
end