I am quiet new to ruby, rake, buildr and of course to FileUtils and FileList.
I have a directory structure like this:
+root
|-dir1
|+dir2
||-dir not to delete
||-dir3
|-dir4
…
After the clean, everything that should be left over is:
+root
|+dir2
||-dir not to delete
Right now I am trying that:
clean do
FileList[_(:root) + "/**/*"]
.exclude(_(:dir not to delete))
.each do |file|
puts file
end
end
that »puts«
root/dir1
root/dir2
root/dir/dir3
root/dir4
But how can I now actually go over to delete everything accept the »dir not to delete« including inside its parent directory?
there might be smarter ways to do this, but you can just substract the relevant files/directories like this:
all files in my config directory:
FileList['config/**/**']
=> ["config/application.rb", "config/boot.rb", "config/compass.rb", "config/database.yml", "config/environment.rb", "config/environments", "config/environments/caching.rb", "config/environments/development.rb", "config/environments/production.rb", "config/environments/test.rb", "config/initializers", "config/initializers/formtastic.rb", "config/initializers/omniauth.rb", "config/initializers/secret_token.rb", "config/initializers/session_store.rb", "config/initializers/slim.rb", "config/initializers/typus.rb", "config/initializers/whitelabel.rb", "config/initializers/wrap_parameters.rb", "config/locales", "config/locales/de.base.yml", "config/locales/de.formtastic.yml", "config/locales/de.label.yml", "config/locales/de.yml", "config/locales/en.base.yml", "config/locales/en.formtastic.yml", "config/locales/en.label.yml", "config/locales/en.yml", "config/routes.rb", "config/typus", "config/typus/event.yml", "config/typus/highlight.yml", "config/typus/job.yml", "config/typus/location.yml", "config/typus/material.yml", "config/typus/topic.yml", "config/typus/user.yml", "config/whitelabel.yml"]
removing all files that are in the locales directory:
FileList['config/**/**'] - FileList['config/**/locales/**']
=> ["config/application.rb", "config/boot.rb", "config/compass.rb", "config/database.yml", "config/environment.rb", "config/environments", "config/environments/caching.rb", "config/environments/development.rb", "config/environments/production.rb", "config/environments/test.rb", "config/initializers", "config/initializers/formtastic.rb", "config/initializers/omniauth.rb", "config/initializers/secret_token.rb", "config/initializers/session_store.rb", "config/initializers/slim.rb", "config/initializers/typus.rb", "config/initializers/whitelabel.rb", "config/initializers/wrap_parameters.rb", "config/locales", "config/routes.rb", "config/typus", "config/typus/event.yml", "config/typus/highlight.yml", "config/typus/job.yml", "config/typus/location.yml", "config/typus/material.yml", "config/typus/topic.yml", "config/typus/user.yml", "config/whitelabel.yml"]
Related
I am writing a chef resource with a logic as mentioned with each steps
Search for a 'zip' content from the http website and download it
After downloading unzip the files and put it under a directory - for e.g /u01/var/
Now here comes the tricky part - for each downloaded zip file i need
to traverse through each file and do the same operation which is
applicable to different zip files
My Code -
require 'open-uri'
links = open(patch_depot ,&:read).to_s
out1=links.scan(/\s*\s*"([^"]*zip)"/imu).flatten
patch_files = out1.select{ |i| i[/\.zip$/]}
print patch_files
unless patch_files.length>=1
Chef::Log.info('No latest file found..!!')
else
c = Dir.pwd
Dir.chdir(cache_direc)
patch_files.each do |patch|
if ::File.exist?(::File.join(cache_direc,patch))
Chef::Log.info("#{patch} file is already downloaded")
else
open(patch, 'wb') do |fo|
fo.print open("#{patch_depot}/#{patch}").read
end
`unzip -qo #{patch}`
Chef::Log.info("#{patch} is downloaded and extracted")
FileUtils.chown_R osuser, usergroup, cache_direc
FileUtils.chmod_R 0777, cache_direc
end
So with the code mentioned above i am able to achieve point 1 and point 2
After this code block i have a ruby block which updates a file and i have a bash block which do some operation.
Like below -
ruby_block 'edit bsu.sh file' do
block do
bsu_sh_file = Chef::Util::FileEdit.new("#{bsu_loc}/utils/asu/mmy.sh")
bsu_sh_file.search_file_replace_line(/^MEM_ARGS="-Xms256m -Xmx512m"*$/, "MEM_ARGS='-Xms1024m -Xmx1024m'")
if bsu_sh_file.unwritten_changes? == false
Chef::Log.info('No Changes need be made to mmy_sh_file')
else
bsu_sh_file.write_file
Chef::Log.info('Changes made in mmy_sh_file file')
end
end
end
bash block -
bash "test bash" do
action:run
code <<-EOH
some operation
EOH
end
My HTTP content may have multiple zip files
So for each zip file i need to unzip and do the operations mentioned on ruby_block and bash block
Kindly provide a solution or a suggestion
EDIT #1 :
The Code written which is already a custom resource , I know i mess up with the loop some where. My code doesn't not moving inside the loop and iterating through the other actions.
Assuming you have a list of urls from where you want to download, you can have some attributes like:
default['your_cookbook']['files']['file_1'] = {
address: 'http://whatever.com/file1.zip',
name: 'file1.zip',
path: '/where/you/want/it/',
checksum: '12341234123412341234'
}
default['your_cookbook']['files']['file_2'] = {
address: 'http://whatever.com/file2.zip',
name: 'file2.zip',
path: '/where/you/want/it/',
checksum: '12341234123412341234'
}
Then you can do this:
node['your_cookbook']['files'].each do |file|
remote_file file.name do
path "/tmp/#{file.name}"
source file.address
checksum file.checksum
end
execute "extract_#{file.name}" do
command "unzip /tmp/#{file.name} -d #{file.path}#{file.name}"
action :nothing
subscribe :run, "remote_file[#{file.name}]", :immediate
end
end
I think this have the same functionality you need and avoid using custom resources in favor of the default ones, which are quite complete.
I wish to process all .jpg files recursively. I need to have their parent directory available at some variable as well. So I moved from:
Dir.cwd("/some/path")
Dir.glob("**/*.jpg") { |the_file| }
to:
Dir.cwd("/some/path")
Dir.glob("**/") { |the_dir|
Dir.glob("#{the_dir}*.jpg") { |the_file|
puts "file: #{the_file} is at #{the_dir}"
}
}
Unfortunately it omits *.jpg files at the Dir.cwd itself. For my test dir:
$ find
.
./some_dir
./some_dir/another_one
./some_dir/another_one/sample_A.jpg
./some_dir/sample_S.jpg
./sample_4.jpg
./sample_1.jpg
./sample_3.jpg
./sample_2.jpg
I got output for sample_A.jpg and sample_S.jpg but not for any other.
From what I understood this should do:
Dir.glob("**/*.jpg") do |thefile|
puts "#{File.basename(thefile)} is at #{File.dirname(thefile)}"
end
dirname give you the parent directory only.
You may extend dirname by expand_path if you want the full path name.
I.e.: File.dirname(File.expand_path(thefile)) which should give you the full path to the file.
Side note, there's other methods in ruby > 2.0 from the File class, but I did stick with the basic ones here.
I found one other way with two loops which I believe can be faster in some situations as it doesn't call File.dirname for each file:
Dir.glob("{./,**/}") { |the_dir|
# puts "dir: #{the_dir}"
Dir.glob("#{the_dir}*.jpg") { |the_file|
puts "file: #{the_file} is at #{the_dir}"
}
}
I would like to make a Ruby program which sorts the images in the current directory into different subfolders, for example:
tree001.jpg, ... tree131.jpg -> to folder "tree"
apple01, ... apple20.jpg -> to folder "apple"
plum1.jpg, plum2.jpg, ... plum33.jpg -> to folder "plum"
and so on, the program should automagically recognize which files belong together by their names. I have no clue how to achive this. Till now I make a small program which collect the files with command "Dir" into an array and sort it alphabetically to help finding the appropriate classes by the file names. Does anybody have a good idea?
Check out Find:
http://www.ruby-doc.org/stdlib-2.0/libdoc/find/rdoc/Find.html
Or Dir.glob:
http://ruby-doc.org/core-2.0/Dir.html#method-c-glob
For instance:
Dir.glob("*.jpg")
will return an array that you can iterate with each.
I'd go about it something like this:
files = %w[
tree001.jpg tree03.jpg tree9.jpg
apple1.jpg apple002.jpg
plum3.jpg plum300.jpg
].shuffle
# => ["tree001.jpg", "apple1.jpg", "tree9.jpg", "plum300.jpg", "apple002.jpg", "plum3.jpg", "tree03.jpg"]
grouped_files = files.group_by{ |fn| fn[/^[a-z]+/i] }
# => {"tree"=>["tree001.jpg", "tree9.jpg", "tree03.jpg"], "apple"=>["apple1.jpg", "apple002.jpg"], "plum"=>["plum300.jpg", "plum3.jpg"]}
grouped_files.each do |grp, files|
Dir.mkdir(grp) unless Dir.exist(grp)
files.each { |f| FileUtils.mv(f, "#{grp}/#{f}") }
end
I can't test that because I don't have all the files, nor am I willing to generate them.
The important thing is group_by. It makes it easy to group the similarly named files, making it easy to walk through them.
For your case, you'll want to replace the assignment to files with Dir.glob(...) or Dir.entries(...) to get your list of files.
If you want to separate the file path from the file name, look at File.split or File.dirname and File.basename:
File.split('/path/to/foo')
=> ["/path/to", "foo"]
File.dirname('/path/to/foo')
=> "/path/to"
File.basename('/path/to/foo')
=> "foo"
Assuming every file name starts with non-digit characters followed by at least one digit character, and the initial non-digit characters define the directory you want the file moved to:
require 'fileutils'
Dir.glob("*").select{|f| File.file? f}.each do |file| # For each regular file
dir = file.match(/[^\d]*/).to_s # Determine destination directory
FileUtils.mkdir_p(dir) # Make directory if necessary
FileUtils.mv(file, dir) # Move file
end
The directories are created if necessary. You can run it again after adding files. For example, if you added the file tree1.txt later and re-ran this, it would be moved to tree/ where tree001.jpg through tree131.jpg already are.
Update: In the comments, you added the requirement that you only want to do this for files which form groups of at least 10. Here's one way to do that:
require 'fileutils'
MIN_GROUP_SIZE = 10
reg_files = Dir.glob("*").select{|f| File.file? f}
reg_files.group_by{|f| f.match(/[^\d]*/).to_s}.each do |dir, files|
next if files.size < MIN_GROUP_SIZE
FileUtils.mkdir_p(dir)
files.each do |file|
FileUtils.mv(file, dir)
end
end
I'm using nanoc to generate an static site.
Recently I added Bower to manage front end dependencies.
When I add Bootstrap via Bower I place the package in /assets/bower/
The Bootstrap package contains multiple files, including:
bootstrap/js/tests/vendor/qunit.css
bootstrap/js/tests/vendor/qunit.js
My Rules file has these rules:
route '/assets/*' do
extension = item[:extension]
if extension == 'coffee'
extension = 'js'
end
item.identifier.chop + '.' + extension
end
compile '*', :rep => :spec do
if !item[:spec_files].nil? && !item.binary?
filter :erb
layout 'spec'
end
end
route '*', :rep => :spec do
if !item[:spec_files].nil? && !item.binary?
'/specs' + #item.identifier[0..-2] + '.html'
end
end
compile '*' do
if !item.binary?
filter :erb
layout_name = item[:layout] || 'default'
layout layout_name
end
end
route '*' do
if item.binary?
item.identifier.chop + '.' + item[:extension]
else
item.identifier[0..-2] + '.html'
end
end
When running nanoc I get the following error:
RuntimeError: Found 2 content files for
content/assets/bower/bootstrap/js/tests/vendor/qunit; expected 0 or 1
I tried adding 2 new 'empty' rules for the /assets/bower/ folder but still getting the error.
route '/assets/bower/*' do
end
compile '/assets/bower/*' do
end
Any suggestions?
Later edit:
Looks like nanoc supports a static datasource that also takes in consideration the file extension.
https://github.com/nanoc/nanoc-site/blob/master/content/docs/troubleshooting.md
Still not sure if I can use both data sources in parallel.
Unfortunately, you can't have two files in the same directory with the same name before the last extension. For nanoc 4.0 it'll be rewritten to change that.
You can definitely have multiple data sources used at once, but that means you can't apply filters to the qunit files, only redirect the output.
Do you explicitly have to be able to organise files the same as Bower installs them? It might be a better idea to split them up into scripts and styles if you can, anyway - you'll almost certainly be filtering based on filetype, anyway, and that means in Rules you can just go
compile '/whatever-path/scripts/' do
filter :concatenate
filter :uglify_js
end
rather than
compile '/whatever-path/ do
case item[:extension]
when 'js'
filter :uglify_js
when 'scss'
filter :sass
end
end
I'm writing my first Rakefile. The first things that I see in the doc is "there is no special format for a Rakefile" and "there is no special syntax in a Rakefile".
Ok, so I had to come up with something on my own, but I can see at least two problems with my creature:
1) I need to create a number of folders, five of them. The sequence of 6 directory tasks looks a bit weird. The list of 5 dependencies in deploy task looks even more weird. Can I shrink it down to one line somehow?
2) I need to repeat my directory name literals two times - when I define their deployment paths and when I copy the contents. Can I avoid that without introducing 5 more variables?
In Java Ant I would create a properties file with all name literals - can I do that with Rake?
This is what I've got:
WEBAPPSDIR = '/var/webapps/'
WEBAPPNAME = 'foo.local'
WEBAPPDIR = File.join(WEBAPPSDIR, WEBAPPNAME)
VIEWSDIR = File.join(WEBAPPDIR, 'views')
PUBLICDIR = File.join(WEBAPPDIR, 'public')
CSSDIR = File.join(PUBLICDIR, 'css')
IMAGESDIR = File.join(PUBLICDIR, 'images')
TMPDIR = File.join(WEBAPPDIR, 'tmp')
HTMLDIR = File.join(PUBLICDIR, 'html')
directory VIEWSDIR
directory CSSDIR
directory HTMLDIR
directory IMAGESDIR
directory TMPDIR
desc 'Deploy to webapps dir'
task :deploy => [VIEWSDIR, CSSDIR, IMAGESDIR, TMPDIR, HTMLDIR] do
cp 'config.ru', WEBAPPDIR
Dir.glob('*.rb') {|f| cp f, WEBAPPDIR}
Dir.glob('views/*.{mab,str}') {|f| cp f, VIEWSDIR}
Dir.glob('css/*.css') {|f| cp f, CSSDIR}
Dir.glob('images/*.{png,jpg,gif}') {|f| cp f, IMAGESDIR}
Dir.glob('html/*.html') {|f| cp f, VIEWSDIR}
end
desc 'Cleans webapp dir'
task :clean do
rm_r WEBAPPDIR, {force: true}
end
Other thoughts/links/examples are welcome too.
This does not really answer your question - but why don't you use capistrano ? If you don't know it already, it's a ruby tool frequently used to handle deployments smoothly