how to download a zip file from s3 bucket using ruby - ruby

i am able to zip all required files(using ruby gems of AWS SDK) & upload them to the S3 bucket. when i tried to download the zip files from S3 to local server & unzip them , i am geeting following error...
[2013-05-06T07:19:37+00:00] FATAL: TypeError: aws_unzip[db_unzip] (aws::unzip line 14) had an error: TypeError: can't dup NilClass
Even when i try to unzip manually, i can see "zip files is corrupted"...But the zip file which is present in S3 location is not corrupted(i tested to extract them manually by downloading them, it extracted well)....
Can any one help where i am doing mistake in reading zip files from bucket to local server?????
When downloading it is getting corrupted...
My code is
File.open(dd, 'w') {|f| f.write(obj.read.force_encoding('utf-8'))}

Try 'wb' mode - zip is a binary file. Also that force_encoding is very suspicious.

Related

How can I avoid "Zip end of central directory signature not found (Zip::Error)" with rubyzip?

I'm reading a lot of zip file with rubyzip.
However this error message is always showing in only specific file even it is zip file.
/app/vendor/bundle/ruby/2.3.0/gems/rubyzip-1.2.1/lib/zip/central_directory.rb:143:in `get_e_o_c_d': Zip end of central directory signature not found (Zip::Error)
I guess this error occures in rubyzip.
How can I manage this error?
Here is my code.
url = 'http://example.zip'
zipfilename = open(url)
Zip::File.open(zipfilename, :allow_redirections => :all) do |zip_file|
entry = zip_file.glob("*ixbrl.htm").first
stream = entry.get_input_stream.read
puts stream
end
Thank you!
I faced this error when I try to extract data from a uploaded .xlsx file in my application. On my context, the .xlsx file was corrupted, so my solution was save the content (I usually handle the buffer file after the upload) as a csv file (in my application I dont need to worry about the file extension), fixing the content by force it to encode as utf-8, and extract it's data after it. here is a example of the code, I'm using roo-xls gem to handle .xls files and roo gem to handle .csv and .xlsx files.
I ran into the same error. Also only reproducible on Heroku. The error was fixed after I added an unzip buildpack (second, after Ruby).
https://github.com/davidlibrera/heroku-buildpack-unzip
If you're using a google docs xlsx file try checking if the access of the sheet link is public.

The tar might be corrupted or it is compressed in an unexpected way

I am trying to unpack tar.xz file using gradle. First I download it and then unpack using tasks
def rootDir = project(":").projectDir
task downloadGHC(type: Download) {
src "$ghcDownloadLink"
dest new File("$rootDir/applications/install", "ghc-8.0.1-x86_64-unknown-mingw32.tar.xz")
onlyIfNewer true
}
task unpackGHC(dependsOn: downloadGHC, type: Copy) {
from tarTree(downloadGHC.dest)
into "$rootDir/applications/ghc"
}
But I get this error
Unable to expand TAR 'L:\...\applications\install\ghc-8.0.1-x86_64-unknown-mingw32.tar.xz'
The tar might be corrupted or it is compressed in an unexpected way.
By default the tar tree tries to guess the compression based on the file extension.
If you need to specify the compression explicitly please refer to the DSL reference.
> Error detected parsing the header
I am able to open the file so it is not corrupted. How to unpack it?
Since there is an error parsing the header, I suppose the tarTree() accepts only .tar files and not .tar.xz file. You can call an external program like tar from your gradle script to unpack the file.
Or you can write a small program to decompress the xz to tar using xz-java library (XZInputStream class).

How to move jar file from one folder to another in Ruby?

In my Ruby script I have to copy one jar file from lib to a different location. Below is the piece of code where I am trying to do so:
FileUtils.mv("#{$HOME_PATH}/lib/myjar1.jar", "#{$HOME_PATH}")
But it is not able to move the jar file to destination. I am seeing the following error:
No such file or directory - file:/home/path/lib/myjar1.jar or file:/home/path/myjar1.jar
I double checked that myjar1.jar is present in source directory. Not sure why I am seeing that error.
I also tried using FileUtils.cp_r("#{$HOME_PATH}/lib/myjar1.jar", "#{$HOME_PATH}") and I am seeing the following error: unknown file type: file:/home/path/lib/myjar1.jar.
Any guidance in copying or moving a jar file is appreciated.
Im assuming you are starting from your home directory (hence the $HOME_PATH). I prefer to use the given ENV hash in IRB.
In IRB this works in MAC OSX:
>> FileUtils.mv("#{ENV['HOME']}/Desktop/my.jar", "#{ENV['HOME']}")
0
The FileUtils#mv method returns 0 as that is the return code of the Unix version of mv which just means it was successful.

Parse CLI won't upload non-js files

I'm trying to send iOS push notifications using my Cloud Code (I can't use Parse's Push APIs as my app is built using ionic and all their docs expect native).
I have it working as a standalone script locally using nodejs, but when I go to upload it to parse, I get:
Uploading source files
Note that the following files will not be uploaded:
parse_cloud_code/cloud/cloud/cert.pem
parse_cloud_code/cloud/cloud/key.pem
Uploading recent changes to scripts...
The following files will be uploaded:
parse_cloud_code/cloud/cloud/cloud.js
parse_cloud_code/cloud/cloud/cloud_test.js
parse_cloud_code/cloud/cloud/credentials.js
parse_cloud_code/cloud/cloud/fs.js
parse_cloud_code/cloud/cloud/push-notification.js
parse_cloud_code/cloud/cloud/push-notifications_test.js
parse_cloud_code/cloud/cloud/tls.js
Finished uploading files
Error: Failed to load cloud/cert.pem with: Could not find file cloud/cert.pem
at Object.exports.readFile (cloud/fs.js:24:17)
at readFile (cloud/push-notification.js:45:8)
at body (cloud/push-notification.js:56:5)
at cloud/push-notification.js:147:3
at cloud/push-notification.js:156:3
at cloud/cloud.js:5073:5
at cloud/cloud.js:5082:3
at main.js:1:13
How can I get the .pem files into the cloud code? I tried renaming them to .js but then Parse wanted them to actually be syntactically JS files. Imagine that.
I found a workaround for this situation, as I had a similar problem. Any non-js file that I must upload I would rename to .ejs. So the parse CLI will upload it and them you can use it inside your cloud code.

Upload a file ruby using selenium

I want to upload a file from remote url in ruby using selenium
My tools:-
watir-webdriver
selenium-webdriver
Upload a file:-
browser.file_field(:id, 'file').set("https://www.google.co.in/logos/doodles/2015/holi-festival-2015-5124794139803648-hp.gif")
Error:-
Errno::ENOENT: No such file or directory - https://www.google.co.in/logos/doodles/2015/holi-festival-2015-5124794139803648-hp.gif
The value of an upload input field should be an absolute path to the file on your local machine, not a URL. Download the image first.

Resources