We do over 20 deployments a day using capistrano (actually webistrano) and we have a problem where the disk space on our servers get full of old deployment folders.
Every now and again I run the deploy:cleanup task to clean out all deployments (it keeps the last :keep_releases, currently set to 30). I would like to automate the cleanup.
One solution would be to add the following to the recipe to automatically run the cleanup after every deployment:
after "deploy", "deploy:cleanup"
But, I don't want to do this after every deployment, I'd like to limit it to only when the number of previous deployments gets to a threashold, e.g. 70. Does anyone know how I can do this?
Thoughts:
Does Capistrano provide a variable that holds the number of previous deployments?
If not, does anyone know a way to calculate it. i.e. set :num_releases, <what-can-I-put-here-to-count-previous-deployments>
Is there a way to pimp deploy:cleanup so it uses a minimum threshold, i.e. exit if < :max_releases previous deployments (where :max_releases is different from :keep_releases).
Could the except keyword be used? i.e. something like :except => { :num_releases < 70}.
Does Capistrano provide a variable that holds the number of previous deployments?
Yes, releases.length
Is there a way to pimp deploy:cleanup so it uses a minimum threshold?
Yes, here is a privately namespaced task that will trigger the normal cleanup task ONLY if a certain number of release folders have built up:
namespace :mystuff do
task :mycleanup, :except => { :no_release => true } do
thresh = fetch(:cleanup_threshold, 70).to_i
if releases.length > thresh
logger.info "Threshold of #{thresh} releases reached, runing deploy:cleanup."
deploy.cleanup
end
end
end
To have this run automatically after a deploy, put this at the top of the recipe:
after "deploy", "mystuff:mycleanup"
The good thing about this is that, before and after directives set on deploy:cleanup are executed as normal. For example we require the following:
before 'deploy:cleanup', 'mystuff:prepare_cleanup_permissions'
after 'deploy:cleanup', 'mystuff:restore_cleanup_permissions'
A quick and dirty approach using the current capistrano code:
Change the cleanup task in https://github.com/capistrano/capistrano/blob/master/lib/capistrano/recipes/deploy.rb#L405 into this:
task :cleanup, :except => { :no_release => true } do
thresh = fetch(:cleanup_threshold, 70).to_i
count = fetch(:keep_releases, 5).to_i
if thresh >= releases.length
logger.important "no old releases to clean up"
else
logger.info "threshold of #{thresh} releases reached, keeping #{count} of #{releases.length} deployed releases"
directories = (releases - releases.last(count)).map { |release|
File.join(releases_path, release) }.join(" ")
try_sudo "rm -rf #{directories}"
end
end
and then you will be able to add
set :cleanup_threshold, 70
to your deployment recipe.
Related
AFAIK, Rake comes with two types of tasks: Rake::Task which runs unconditionally, and Rake::FileTask which runs only if the file it is named after doesn't exist, or is older that one of its prerequisites.
Is there a conventional way to customize the logic that decides if a task needs to run? For example, if I wanted to not only verify the existence of a file, but also test its contents somehow.
I can see the method Rake::Task#needed? handles this, and overriding that in a subclass does indeed work. But is there a more idiomatic way to do this? Something that would be more suitable to include directly in a Rakefile?
I'm imagining something like this:
need :process do
# Check if file is already processed
end
task :process do
# Process file in-place
end
which would skip the task if all of its need blocks return true.
Is there a conventional way to customize the logic that decides if a
task needs to run?
Yes; a way to do it is:
declare the "needed tasks" as task_1, and let it exist if needed operations aren't proceeded
declare the "secondary tasks" as task_2 with prerequisite: task_1
so your rake file will be like:
# check if needed is done
def needed_done?
return false # TODO: edit checking logic
end
desc "do prerequisite stuff"
task :do_needed do
p "do needed stuff"
unless needed_done?
p "needed stuff wasn't done ^^'"
exit 1
end
end
desc "process other stuff, if prerequisite is meet"
task :process => [:do_needed] do
p "process other stuff"
end
now when you ran the task process with:
rake process
do_needed will automatically ran first, if needed_done? then process will run, else you'll exit without running it
I have built my own job server, which is essentially a private gem, built as a wrapper around resque.
(I am not running this in a Rails environment)
Everywhere I look, it seems like the documented/recommended way to start the workers, is with something like this:
$ QUEUE=* rake resque:work
Which means that it must be executed in a folder where the Rakefile exists.
I am looking for a way to start it without a Rakefile.
What I have learned so far:
I have looked through the issues, maybe someone asked a similar question.
I have looked through the wiki, and specifically the FAQ.
I know I can probably create my own "bin" to run it without rake, by analyzing the tasks file.
I saw that resque installs a resque binary, but it only seems to provide limited functionality, like removing and listing a worker, but not starting.
My current workaround is that my gem's binary is doing chdir to the gem's folder before running (and this folder has a Rakefile), like the code below.
def start_worker
ENV['QUEUE'] = '*'
Dir.chdir gemdir do
exec "rake resque:work"
end
end
def gemdir
File.expand_path "../../", __dir__
end
Appreciate any nudge in the right direction.
The current solution I have worked up for this:
def start_worker
interval = 5
queue = '*'
ENV['QUEUE'] = queue
worker = Resque::Worker.new
Resque.logger = Logger.new STDOUT
Resque.logger.level = Logger::INFO
## this is not yet implemented in 1.26.0, keeping here as a reminder
# worker.prepare
worker.log "Starting worker"
worker.work interval
end
Which is an adaptation of the code from the rake task
For reference, I also opened a github issue, in the off chance that someone else also needs such functionality.
I having created a script to create demon worker processes using following worker starting API.
def start_worker(id)
ENV['QUEUE'] = #queues || "*"
ENV['PIDFILE'] = pid_file(id)
ENV['JOBS_PER_FORK'] = #jobs_per_fork || "1000"
ENV['BACKGROUND'] = 'true'
ENV['TERM_CHILD'] = 'true'
#debug ? ENV['VVERBOSE'] = 'true' : ENV['VERBOSE'] = 'true'
begin
worker = Resque::Worker.new
rescue Resque::NoQueueError
Resque.logger.error "No queue is set for worker_id = #{id}"
end
worker.prepare
worker.log "Starting worker #{self}"
worker.work(5) # interval, will block
end
For server rebuild, I want to skip some section of my cookbook according to the results of a previous run.
For instance, we have a resource to start Weblogic servers.
weblogic_server "server_name" do
action :start
end
These startups take a lot of time during the build. I want to skip this if it was run succesfully in the last build, to avoid having to wait too much for the rebuild. Something like this:
weblogic_server "server_name" do
action :start
not_if { it_was_run_successfully_during_the_previous_run }
end
I know the best way to do it would be have script checking on weblogic servers status, but that's up to another team and I need a temporary solution.
I thought about a logfile in JSON format referencing the different steps of the build.
e.g:
{
"provisioning" : true,
"start_weblogic_servers : true,
"configuring_ohs" : false
}
In this case I would have a template resource for this logfile and then update the values during the run. Then in every run I would check this file first and skip the right section according to the values I find.
Is there a better way to go ?
What I have done in the past is to just create an empty file, if it exists then you skip it (not_if do ::File.exists?('/path/to/some_empty_file') end). You could then have some code when a build is successful or not to either create or delete these files, I realise it is probably not the best approach but it has worked for me as long as I can remember.
If you really want then you could have some script checking the server status (say on a 5 minute interval) and then adjusting that empty file accordingly (by deleting it or keeping it).
Nabeel Amjad solution worked for me. Follow these steps:
Create a file resource with action :nothing
file '/tmp/logfile' do
action: nothing
end
Set your resource to notify the file resource after running
weblogic_server 'server_name' do
action :start
notifies :create, 'file[/tmp/logfile]', :immediately
end
Add a guard not_if that will skip future execution of this resource if the file exists on the server
weblogic_server 'server_name' do
action :start
notifies :create, 'file[/tmp/logfile]', :immediately
not_if { ::File.exist?('/tmp/logfile') }
end
I want mark the current deployed directory with my release number.
I tried this approach:
Get locally the app version, store it into a variable, and on the remote host, store it in a file.
namespace :deploy do
desc "Set a release number as the app version"
task :mark_release do
release_number = `git describe`
on roles(:web) do
execute("echo #{release_number} > #{current_path}/RELEASE")
end
end
end
The problem is, when I run this via:
cap deploy:mark_release
the command look like this:
echo v9.3.0-254-g178d1f8; > /foo/bar/current/RELEASE
The semicolon is making trouble. and my RELEASE file is of course empty.
I think it is due to some escaping made by SSHKit.
Any clues ?
I managed it:
1) I took the release number from the repo directory on the machine
2) I wrote it with a stream to a file via the upload! method
namespace :deploy do
desc "Set a release number as the app version"
task :mark_release do
on roles(:web) do
within "/foo/bar/repo/" do
upload! StringIO.new(capture(:git, "describe")), "#{current_path}/RELEASE"
end
end
end
end
Here is the solution that I came up with which doesn't require uploading a local file. It goes to the repo path to execute the git command to extract the version and then redirects the output to file. The file can then be read by the Rails app. The execute requires the different parameters to be passed in separately. https://github.com/capistrano/sshkit#the-command-map has more info about the command map and why it's needed, due to the problem of escaping and whitespace.
namespace :deploy do
before :restart, :add_revision_file
task :add_revision_file do
on roles(:app) do
within repo_path do
execute(:git, :'rev-parse', :'--short', :'HEAD', ">#{release_path}/REVISION")
end
end
end
end
Use SSHKit::Command
SSHKit::Command.new("echo #{release_number} > #{current_path}/RELEASE")
I have a Rakefile that compiles the project in two ways, according to the global variable $build_type, which can be :debug or :release (the results go in separate directories):
task :build => [:some_other_tasks] do
end
I wish to create a task that compiles the project with both configurations in turn, something like this:
task :build_all do
[ :debug, :release ].each do |t|
$build_type = t
# call task :build with all the tasks it depends on (?)
end
end
Is there a way to call a task as if it were a method? Or how can I achieve anything similar?
If you need the task to behave as a method, how about using an actual method?
task :build => [:some_other_tasks] do
build
end
task :build_all do
[:debug, :release].each { |t| build t }
end
def build(type = :debug)
# ...
end
If you'd rather stick to rake's idioms, here are your possibilities, compiled from past answers:
This always executes the task, but it doesn't execute its dependencies:
Rake::Task["build"].execute
This one executes the dependencies, but it only executes the task if
it has not already been invoked:
Rake::Task["build"].invoke
This first resets the task's already_invoked state, allowing the task to
then be executed again, dependencies and all:
Rake::Task["build"].reenable
Rake::Task["build"].invoke
Note that dependencies already invoked are not automatically re-executed unless they are re-enabled. In Rake >= 10.3.2, you can use the following to re-enable those as well:
Rake::Task["build"].all_prerequisite_tasks.each(&:reenable)
for example:
Rake::Task["db:migrate"].invoke
task :build_all do
[ :debug, :release ].each do |t|
$build_type = t
Rake::Task["build"].reenable
Rake::Task["build"].invoke
end
end
That should sort you out, just needed the same thing myself.
task :invoke_another_task do
# some code
Rake::Task["another:task"].invoke
end
task :build_all do
[ :debug, :release ].each do |t|
$build_type = t
Rake::Task["build"].execute
end
end
If you want each task to run regardless of any failures, you can do something like:
task :build_all do
[:debug, :release].each do |t|
ts = 0
begin
Rake::Task["build"].invoke(t)
rescue
ts = 1
next
ensure
Rake::Task["build"].reenable # If you need to reenable
end
return ts # Return exit code 1 if any failed, 0 if all success
end
end
I would suggest not to create general debug and release tasks if the project is really something that gets compiled and so results in files. You should go with file-tasks which is quite doable in your example, as you state, that your output goes into different directories.
Say your project just compiles a test.c file to out/debug/test.out and out/release/test.out with gcc you could setup your project like this:
WAYS = ['debug', 'release']
FLAGS = {}
FLAGS['debug'] = '-g'
FLAGS['release'] = '-O'
def out_dir(way)
File.join('out', way)
end
def out_file(way)
File.join(out_dir(way), 'test.out')
end
WAYS.each do |way|
desc "create output directory for #{way}"
directory out_dir(way)
desc "build in the #{way}-way"
file out_file(way) => [out_dir(way), 'test.c'] do |t|
sh "gcc #{FLAGS[way]} -c test.c -o #{t.name}"
end
end
desc 'build all ways'
task :all => WAYS.map{|way|out_file(way)}
task :default => [:all]
This setup can be used like:
rake all # (builds debug and release)
rake debug # (builds only debug)
rake release # (builds only release)
This does a little more as asked for, but shows my points:
output directories are created, as necessary.
the files are only recompiled if needed (this example is only correct for the simplest of test.c files).
you have all tasks readily at hand if you want to trigger the release build or the debug build.
this example includes a way to also define small differences between debug and release-builds.
no need to reenable a build-task that is parametrized with a global variable, because now the different builds have different tasks. the codereuse of the build-task is done by reusing the code to define the build-tasks. see how the loop does not execute the same task twice, but instead created tasks, that can later be triggered (either by the all-task or be choosing one of them on the rake commandline).