I'm trying to use remote_file to cache a local copy of a large package on a Windows share. How is this done?
I can't get it to work with a drive-letter-based path, a UNC-based path, or a file: URL.
Don't have windows to test, but something like this should work:
require 'fileutils'
remote_path = '...'
local_path = '...'
ruby "cache-#{remote_path}" do
block { FileUtils.copy_file(remote_path, local_path) }
not_if { File.exists?(local_path) }
end
I worked out a trick that I think is pretty neat. I created the following definition (and put it in definitions/default.rb):
define :file_from_network, :action => :create do
myPath = (params[:path] || params[:name])
mySource = params[:source]
if File.exist?(mySource)
file myPath do
action params[:action]
content File.open(mySource) {|io| io.read}
end
else
Chef::Log.error("File #{mySource} not found!")
end
end
Definitions do not work quite the same way resources do, but this was easy to implement and does what I need it to do. The in-memory read makes it impractical for huge files, of course, but it allows Chef to check that the content is different before triggering an action on the file resource.
Related
I would like to copy http://seapower/spring.txt and http://seapower/has_sprung.txt and append second one to the first one in a new file named src_filepath.txt:
remote_file 'src_filepath.txt' do
source 'http://seapower/spring.txt', 'http://seapower/has_sprung.txt'
checksum node['nginx']['foo123']['checksum']
owner 'root'
group 'root'
mode '0755'
end
It doesn't work and just copy the first file to src_filepath.txt
Something like this is probably a good place to start and then tweak however you like:
cache1 = "#{Chef::Config[:file_cache_path]}/content1"
cache2 = "#{Chef::Config[:file_cache_path]}/content2"
# this will not redownload if cache1 exists and has not been updated
remote_file cache1 do
source "http://source.url/content1"
end
# this will not redownload if cache1 exists and has not been updated
remote_file cache2 do
source "http://source.url/content2"
end
# this will not update the file if the contents has not changed
file "/my/combined/file" do
content lazy { IO.read(cache1) + IO.read(cache2) }
end
This is not something Chef supports directly. You could use multiple remote_file resources and either a ruby_block or execute plus cat to implement the concat.
remote_file does not support concatenation, so you would not be able to implement this using that resource directly, however you could piece together the desired result using the file resource and Net::HTTP like so:
file_path = '/path/to/your_whole_file'
unless File.exist?(file_path) &&
Digest::SHA256.hexdigest(File.read(file_path)) == 'your_file_checksum'
file file_path do
content(
Net::HTTP.get(URI('http://source.url/content1')) +
Net::HTTP.get(URI('http://source.url/content2'))
)
owner 'root'
group 'root'
mode '0755'
end
end
The reason for the Digest::SHA256 call at the beginning is to prevent Chef from trying to download both files during every Chef run. Note that you may have to require the net/http and digest gems at the top of your recipe for this to work.
Also, because it's against best practices to put Ruby code directly into your recipes, you may want to wrap the above code in a simple custom resource.
remote_file block copies only one specific file.
Is there any possibility in Chef to copy all files from specific folder on ftp?
my current code is quite weird as for me:
require 'net/ftp'
ftp = Net::FTP::new("server")
ftp.login("user", "password")
ftp.chdir("/folder")
fileList = ftp.nlst('*.jar')
fileList.each do |file|
remote_file "C:\\Temp\\" + file do
source "ftp://user:password#server/folder/" + file
action :create_if_missing
end
end
ftp.close
If your solution works, why not wrap it in an LWRP? They are quite easy to create, and would tuck away the implementation in its own file. This is what I would do.
See: http://docs.opscode.com/chef/lwrps_custom.html
And for a real-life - easy to understand - example, see:
https://github.com/opscode-cookbooks/ssh_known_hosts/blob/master/providers/entry.rb
https://github.com/opscode-cookbooks/ssh_known_hosts/blob/master/resources/entry.rb
From the docs:
Per default Aruba will create a directory tmp/aruba where it performs its file operations.
However, my application uses ENV["HOME"] to create and read a file (~/.foorc), so I need Aruba to use a fake ENV["HOME"].
Do I need to set it in some support-file, or is there a way to tell Aruba to its tmp/aruba for files in ENV["HOME"]?
Here is an excerpt of my code that I am testing (obviously I am testing this with Cucumber/Aruba on a much higher level, but the usage of ENV["HOME"] is what is important here.):
def initialize config_path = ""
if config_path.empty?
#config_path = File.join ENV["HOME"], ".todotxt.cfg"
else
#config_path = config_path
end
if file_exists?
super #config_path
validate
end
end
def file_exists?
File.exists? #config_path
end
#....
ask_to_create unless #config.file_exists?
#...
The Specification:
Scenario: todotxt
Given an empty installation
When I run `todotxt`
Then it should pass with:
"""
Should I create a sample config file? [Y/n]
"""
Looking into the implementation in Aruba itself, I could craft something very similar:
File features/support/aruba.rb, is autoloaded by cucumber and implements the Around hook:
# Temporarily enforce an isolated, fake, homedir.
around do |scenario, block|
#__aruba_original_home = ENV["HOME"]
ENV["HOME"] = File.expand_path(File.join("tmp", "aruba"))
block.call
ENV["HOME"] = #__aruba_original_home
end
From now on, a directory tmp/aruba is used as $HOME.
Note that in aruba, this temporary path is configurable, and that above code does not take that into consideration. It will break when the tmp path is configured elsewhere.
Aruba offers a step for just that:
Given a mocked home directory
I'm building a specialized pipeline, and basically, every step in the pipeline involves taking one file as input and creating a different file as output. Not all files are in the same directory, all output files are of a different format, and because I'm using several different programs, different actions have to be taken to appease the different programs.
This has led to some complicated file management in my code, and the more I try to organize the file directories, the more ugly it's getting. Just about every class involves some sort of code like the following:
#fileName = File.basename(file)
#dataPath = "#{$path}/../data/"
MzmlToOther.new("mgf", "#{#dataPath}/spectra/#{#fileName}.mzML", 1, false).convert
system("wine readw.exe --mzXML #{#file}.raw #{$path}../data/spectra/#{File.basename(#file + ".raw", ".raw")}.mzXML 2>/dev/null")
fileName = "#{$path}../data/" + parts[0] + parts[1][6..parts[1].length-1].chomp(".pep.xml")
Is there some sort of design pattern, or ruby gem, or something to clean this up? I like writing clean code, so this is really starting to bother me.
You could use a Makefile.
Make is essential a DSL designed for handling converting one type of file to another type via running an external program. As an added bonus, it will handle only performing the steps necessary to incrementally update your output if some set of source files change.
If you really want to use Ruby, try a rakefile. Rake will do this, and it's still Ruby.
You can make this as sophisticated as you want but this basic script will match a file suffix to a method which you can then call with the file path.
# a conversion method can be used for each file type if you want to
# make the code more readable or if you need to rearrange filenames.
def htm_convert file
"HTML #{file}"
end
# file suffix as key, lambda as value, the last uses an external method
routines = {
:log => lambda {|file| puts "LOG #{file}"},
:rb => lambda {|file| puts "RUBY #{file}"},
:haml => lambda {|file| puts "HAML #{file}"},
:htm => lambda {|file| puts htm_convert(file) }
}
# this loops recursively through the directory and sub folders
Dir['**/*.*'].each do |f|
suffix = f.split(".")[-1]
if routine = routines[suffix.to_sym]
routine.call(f)
else
puts "UNPROCESSED -- #{f}"
end
end
I'm trying to create an application that will primarily consist of ruby scripts that will be run from the command-line (cron, specifically). I want to have a libs folder, so I can put encapsulated, reusable classes/modules in there, and be able to access them from any script.
I want to be able to put my scripts into a "bin" folder.
What is the best way to give them access to the libs folder? I know I can add to the load path via command-line argument, or at the top of each command-line script. In PHP, it sometimes made more sense to create a custom .ini file and point the cli to the ini file, so you got them all in one pop.
Anything similar for ruby? Based on your experience, what's the best way to go here?
At the top of each bin/executable, you can put this at the top
#!/usr/bin/env ruby
$:.unshift(File.join(File.dirname(__FILE__), '..', 'lib')
require 'libfile'
[etc.]
Were you looking for something different?
If you turn your application into a Ruby gem and install the gem on your system, you don't even need to put this stuff at the top. The require statement would suffice in that case.
Sean,
There is no way to not have to require a library, that I know of. I guess if you want to personalize your Ruby so much you could "roll your own" using eval.
The script below basically works as the interpreter. You can add your own functions and include libraries. Give the file executable permissions and put it in /usr/bin if you really want. Then just use
$ myruby <source>
Here's the code for a very minimal one. As an example I've included the md5 digest library and created a custom function called md5()
#!/usr/bin/ruby -w
require 'digest/md5';
def executeCode(file)
handle = File.open(file,'r');
for line in handle.readlines()
line = line.strip();
begin
eval(line);
rescue Exception => e
print "Problem with script '" + file + "'\n";
print e + "\n";
end
end
end
def checkFile(file)
if !File.exists?(file)
print "No such source file '" + file + "'\n";
exit(1);
elsif !File.readable?(file)
print "Cannot read from source file '" + file + "'\n";
exit(1);
else
executeCode(file);
end
end
# My custom function for our "interpreter"
def md5(key=nil)
if key.nil?
raise "md5 requires 1 parameter, 0 given!\n";
else
return Digest::MD5.hexdigest(key)
end
end
if ARGV[0].nil?
print "No input file specified!\n"
exit(1);
else
checkFile(ARGV[0]);
end
Save that as myruby or myruby.rb and give it executable permissions (755). Now you're ready to create a normal ruby source file
puts "I will now generate a md5 digest for mypass using the md5() function"
puts md5('mypass')
Save that and run it as you would a normal ruby script but with our new interpreter. You'll notice I didn't need to include any libraries or write the function in the source code because it's all defined in our interpreter.
It's probably not the most ideal method, but it's the only one I can come up with.
Cheers
There is a RUBYLIB environment variable that can be set to any folder on the system
If you want to use your classes/modules globally, why not just move them to your main Ruby lib directory? eg: /usr/lib/ruby/1.8/ ?
Eg:
$ cat > /usr/lib/ruby/1.8/mymodule.rb
module HelloWorld
def hello
puts("Hello, World!");
end
end
We have our module in the main lib directory - should be able to
require it from anywhere in the system now.
$ irb
irb(main):001:0> require 'mymodule'
=> true
irb(main):002:0> include HelloWorld
=> Object
irb(main):003:0> hello
Hello, World!
=> nil