I have to work with a zipped (regular Zip) string in Ruby.
Apparently I can't save a temporary file with Ruby-Zip or Zip-Ruby.
Is there any practicable way to unzip this string?
rubyzip supports StringIO since version 1.1.0
require "zip"
# zip is the string with the zipped contents
Zip::InputStream.open(StringIO.new(zip)) do |io|
while (entry = io.get_next_entry)
puts "#{entry.name}: '#{io.read}'"
end
end
See Zip/Ruby Zip::Archive.open_buffer(...):
require 'zipruby'
Zip::Archive.open_buffer(str) do |archive|
archive.each do |entry|
entry.name
entry.read
end
end
As the Ruby-Zip seems to lack support of reading/writing to IO objects, you can fake File.
What you can do is the following:
Create a class called File under Zip module which inherits from StringIO, e.g. class Zip::File < StringIO
Create the exists? class method (returns true)
Create the open class method (yields the StringIO to the block)
Stub close instance method (if needed)
Perhaps it'll need more fake methods
As #Roman mentions, rubyzip currently lacks reading and writing of IO objects (including StringIO.new(s)). Try using zipruby instead, like this:
gem install zipruby
require 'zipruby'
# Given a string in zip format, return a hash where
# each key is an zip archive entry name and each
# value is the un-zipped contents of the entry
def unzip(zipfile)
{}.tap do |h|
Zip::Archive.open_buffer(zipfile) do |archive|
archive.each {|entry| h[entry.name] = entry.read }
end
end
end
The zlib library. Works fine with StringIO.
Related
I'm trying to adapt some existing code to also handle gems. This existing code needs the version number of the thing in question (here: the gem) and does some git stuff to get the relevant file (here I take the gemspec) in the right version, and then passes it on stdin to another script that extract the version number (and other stuff).
To avoid having to write code to parse a gemspec, I was trying to do:
spec = Gem::Specification::load('-')
puts spec.name
puts spec.version
But I can't make it read from stdin (it works fine if I hardcode a file name, but that won't work in my usecase). Can I do that, or is there another (easy) way to do it?
Gem::Specification.load expects either a File instance or a path to a file as the first argument so the easiest way to solve this would be to simply create a Tempfile instance and write the data from stdin to it.
file = Tempfile.new
begin
file.write(data_from_stdin)
file.rewind
spec = Gem::Specification.load(file)
puts spec.name
puts spec.version
ensure
file.close
file.unlink
end
I have a ruby script that downloads a remote ZIP file from a server using rubys opencommand. When I look into the downloaded content, it shows something like this:
PK\x03\x04\x14\x00\b\x00\b\x00\x9B\x84PG\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\n\x00\x10\x00foobar.txtUX\f\x00\x86\v!V\x85\v!V\xF6\x01\x14\x00K\xCB\xCFOJ,RH\x03S\\\x00PK\a\b\xC1\xC0\x1F\xE8\f\x00\x00\x00\x0E\x00\x00\x00PK\x01\x02\x15\x03\x14\x00\b\x00\b\x00\x9B\x84PG\xC1\xC0\x1F\xE8\f\x00\x00\x00\x0E\x00\x00\x00\n\x00\f\x00\x00\x00\x00\x00\x00\x00\x00#\xA4\x81\x00\x00\x00\x00foobar.txtUX\b\x00\x86\v!V\x85\v!VPK\x05\x06\x00\x00\x00\x00\x01\x00\x01\x00D\x00\x00\x00T\x00\x00\x00\x00\x00
I tried using the Rubyzip gem (https://github.com/rubyzip/rubyzip) along with its class Zip::ZipInputStream like this:
stream = open("http://localhost:3000/foobar.zip").read # this outputs the zip content from above
zip = Zip::ZipInputStream.new stream
Unfortunately, this throws an error:
Failure/Error: zip = Zip::ZipInputStream.new stream
ArgumentError:
string contains null byte
My questions are:
Is it possible, in general, to download a ZIP file and extract its content in-memory?
Is Rubyzip the right library for it?
If so, how can I extract the content?
I found the solution myself and then at stackoverflow :D (How to iterate through an in-memory zip file in Ruby)
input = HTTParty.get("http://example.com/somedata.zip").body
Zip::InputStream.open(StringIO.new(input)) do |io|
while entry = io.get_next_entry
puts entry.name
parse_zip_content io.read
end
end
Download your ZIP file, I'm using HTTParty for this (but you could also use ruby's open command (require 'open-uri').
Convert it into a StringIO stream using StringIO.new(input)
Iterate over every entry inside the ZIP archive using io.get_next_entry (it returns an instance of Entry)
With io.read you get the content, and with entry.name you get the filename.
Like I commented in https://stackoverflow.com/a/43303222/4196440, we can just use Zip::File.open_buffer:
require 'open-uri'
content = open('http://localhost:3000/foobar.zip')
Zip::File.open_buffer(content) do |zip|
zip.each do |entry|
puts entry.name
# Do whatever you want with the content files.
end
end
I am using Grit (Ruby gem) to load a git repo, extract a given commit to a folder, then 'do stuff' with it. Grit supports git archive with archive_tar which retuns a 'string containing tar archive'. I would like to extract this to the filesystem using Ruby libs / gems if possible (not direct system calls) and avoid things like saving the data to an archive first and then extracting it (really looking for an efficient one-liner here).
Under the assumption that 'string containing tar archive' is equivalent to
File.open("Downloads.tar", "rb") {|a| a.read }
I am able to accomplish this using fileutils, stringio, and the minitar gem.
First you will need to install minitar, which should be as simple as
gem install minitar
Since minitar does not support it's regular unpack to use a stream, we'll create our own rudimentary unpacking method.
require 'archive/tar/minitar'
require 'stringio'
require 'fileutils'
def unpack_tar(directory, string)
FileUtils.mkdir_p(directory) if !File.exist?(directory)
stringio = StringIO.new(string)
input = Archive::Tar::Minitar::Input.new(stringio)
input.each {|entry|
input.extract_entry(directory, entry)
}
end
unpack_tar("./test", File.open("Downloads.tar", "rb") {|a| a.read })
Of course replacing the whole File.open part with the archive_tar function of the Grit gem. This is all assuming that first assumption of course, though i'm sure ths method can easily be adapted to suit whatever archive_tar actually returns.
How do I include a ruby code file, as is, into RDoc?
I have an example.rb file that documents how to use my gem and I would like to include that as one of the files like the README.rdoc and HISTORY.rdoc.
I've already figured out how to convert the ruby source code into HTML using the Syntax gem but I can't figure out how to make RDoc include the file without parsing it.
When I tell RDoc to include the html file it isn't listed and if I fake it out by using rdoc or txt as the file extension it doesn't display properly (the file is still actually html).
I've got a solution that works it's just incredibly ugly. There has got to be a better way to do this that's native to rdoc but I don't see it.
Here's what I have in my Rakefile:
# Build rdocs
require 'rake/rdoctask'
require 'syntax/convertors/html'
rdoc_dir = 'rdoc'
# This is rdoc1 but it doesn't work unless you DON'T wrap it in a task
# Generate html files from example ruby files
convertor = Syntax::Convertors::HTML.for_syntax "ruby"
replacement_key = "REPLACE_THIS_TEXT_WITH_PROPER_HTML"
# Create dummy files
Dir.glob('examples/*.rb').each do |file|
File.open("#{file}.txt", "w") do |dummy_file|
dummy_file.write(replacement_key)
end
end
# Call the rdoc task
Rake::RDocTask.new(:rdoc2) do |rdoc|
rdoc.rdoc_dir = rdoc_dir
rdoc.title = "pickled_optparse #{version}"
rdoc.rdoc_files.include('README*')
rdoc.rdoc_files.include('HISTORY*')
rdoc.rdoc_files.include('examples/*.txt')
rdoc.rdoc_files.include('lib/**/*.rb')
end
task :rdoc3 do
# Now use a hammer to replace the dummy text with the
# html we want to use in our ruby example code file.
html_header = File.read('rake_reqs/html_header.html')
Dir.glob('examples/*.rb').each do |file|
html_ruby = convertor.convert(File.read(file))
rdoc_file = "#{rdoc_dir}/examples/#{File.basename(file,".rb")}_rb_txt.html"
fixed_html = File.read(rdoc_file).gsub!(replacement_key, "#{html_header}#{html_ruby}")
File.open(rdoc_file, "w") {|f| f.write(fixed_html)}
File.delete("#{file}.txt")
end
end
task :rdoc => [:rdoc2, :rdoc3]
Sorry I can't give you an actual answer but I'm looking at sdoc myself.
You can install the gem from ruby gems.
The require method in ruby will search the lib_path and load the first matching files found if needed. Is there anyway to print the path to the file which would be loaded. I'm looking for, ideally built-in, functionality similar to the which command in bash and hoping it can be that simple too. Thanks.
I don't know of a built-in functionality, but defining your own isn't hard. Here's a solution adapted from this question:
def which(string)
$:.each do |p|
if File.exist? File.join(p, string)
puts File.join(p, string)
break
end
end
end
which 'nokogiri'
#=> /opt/local/lib/ruby1.9/gems/1.9.1/gems/nokogiri-1.4.1/lib/nokogiri
Explanation: $: is a pre-defined variable. It's an array of places to search for files you can load or require. The which method iterates through each path looking for the file you called it on. If it finds a match, it returns the file path.
I'm assuming you just want the output to be a single line showing the full filepath of the required file, like which. If you want to also see the files your required file will load itself, something like the solution in the linked question might be more appropriate:
module Kernel
def require_and_print(string)
$:.each do |p|
if File.exist? File.join(p, string)
puts File.join(p, string)
break
end
end
require_original(string)
end
alias_method :require_original, :require
alias_method :require, :require_and_print
end
require 'nokogiri'
#=> /opt/local/lib/ruby1.9/gems/1.9.1/gems/nokogiri-1.4.1/lib/nokogiri
# /opt/local/lib/ruby1.9/gems/1.9.1/gems/rubygems-update-1.3.5/lib/rbconfig
# /opt/local/lib/ruby1.9/gems/1.9.1/gems/nokogiri-1.4.1/lib/nokogiri/xml
# /opt/local/lib/ruby1.9/gems/1.9.1/gems/nokogiri-1.4.1/lib/nokogiri/xml/pp
# /opt/local/lib/ruby1.9/gems/1.9.1/gems/nokogiri-1.4.1/lib/nokogiri/xml/sax
# /opt/local/lib/ruby1.9/gems/1.9.1/gems/nokogiri-1.4.1/lib/nokogiri/xml/node
# /opt/local/lib/ruby1.9/gems/1.9.1/gems/nokogiri-1.4.1/lib/nokogiri/xml/xpath
# /opt/local/lib/ruby1.9/gems/1.9.1/gems/nokogiri-1.4.1/lib/nokogiri/xslt
# /opt/local/lib/ruby1.9/gems/1.9.1/gems/nokogiri-1.4.1/lib/nokogiri/html
# /opt/local/lib/ruby1.9/gems/1.9.1/gems/nokogiri-1.4.1/lib/nokogiri/css
# /opt/local/lib/ruby1.9/1.9.1/racc/parser.rb
$ gem which filename # (no .rb suffix) is what I use...