I am trying to download an XML file from GitHub via Octokit::Client in Ruby. This is in a Dangerfile, so I have access to the client via github.api.
I have got something working with the following code:
listing = client.contents('erik-allen/RepoName', :path => 'Root/Path')
download = open(listing[0].download_url)
I can then call Nokogiri::XML(download) and parse the XML with no issues.
However, it only works because it is the only file in the directory. It also does not feel like the correct way to do things.
I have tried a few other ways:
download = client.contents('erik-allen/RepoName', :path => 'Root/Path/File.xml')
That returned a Sawyer::Resource but I have yet to find a way to get any data from that. I tried combinations of .get.data and .data but neither worked. Calling Base64.decode64() on the result did not yield anything either.
I am suspecting I may need an "accept" header, but I am not sure how I would do that with Octokit::Client.
Does anyone have any suggestions. I would have assumed this would be a common task, but I can find no examples.
I was able to eventually figure things out. There is a content property on the Sawyer::Resource that gives the Base64 data. So the final solution is:
contents = client.contents('erik-allen/RepoName', :path => 'Root/Path/File.xml')
download = Base64.decode64(contents.content)
I can then call Nokogiri::XML(download) and parse the XML with no issues.
Related
According to README on github, Ruby Whois can be used "as a standalone library to parse WHOIS records fetched previously and/or from different WHOIS clients."
I know how to use the library to directly perform whois query and parse the returning result. But I cannot find anywhere(stackoverflow included) how I can use this library to parse whois data previously fetched ?
I think it's not important but this is how I get my data, anyway: they are fetched through linux whois command and stored in separate files, each file containing one whois query result.
The manual pages on https://whoisrb.org/ are 404. Even the code on the homepage is outdated thus wrong, and the doc pages provide little information.
I tried to scan the source code on github( https://github.com/weppos/whois-parser and https://github.com/weppos/whois). I tried to find the answer on rubydoc ( https://www.rubydoc.info/gems/whois-parser/Whois/Parser, https://www.rubydoc.info/gems/whois/Whois/Record and some related pages). Both failed, partly because this task is the first time and the reason that I use Ruby.
So could anyone help me? I'm really desperate and I'll definitely appreciate any help.
Try it like this,
require 'whois-parser'
domain = 'google.com'
data = 'WHOIS DATA THAT YOU ALREADY HAVE'
whois_server = Whois::Server.guess domain
whois_data = [Whois::Record::Part.new(body: data, host: whois_server.host)]
record = Whois::Record.new(whois_server, whois_data)
parser = record.parser
parser.available? #=> false
parser.registered? #=> true
This question is related to this one: Tracking Upload Progress of File to S3 Using Ruby aws-sdk,
However since there is no clear solution to this I was wondering if there's a better/easier way (if one exists) of getting file upload progress with S3 using Ruby in 2018?
In my current setup I'm basically creating a new Resource, fetch my bucket and call upload_file but I haven't yet found any options for passing blocks which would help in yielding some sort of progress.
...
#connection = Aws::S3::Resource.new
#s3_bucket = #connection.bucket(bucket)
#s3_bucket.object(path).upload_file(data, {acl: 'public-read'})
...
Is there a way to do this using the newest sdk-for-ruby v3?
Any help (or even better a small example) would be great.
The example Trevor gives in https://stackoverflow.com/a/12147709/153886 is not hacky from what I can see - just wiring things together. The SDK simply does not provide a feature for passing progress details on all operations. Plus, Trevor is the maintainer of the Ruby SDK at AWS so I trust his judgement.
Expanding on his example
bar = ProgressBar.create(:title => "Uploading action", :starting_at => 0, :total => file.size)
obj = s3.buckets['my-bucket'].objects['object-key']
obj.write(:content_length => file.size) do |writable, n_bytes|
writable.write(file.read(n_bytes))
bar.progress += n_bytes
end
If you want to have a progress block right in the upload_file method I believe you will need to open a PR to the SDK. It is not that strange that is not the case for Ruby (or for any other runtime) because, for example, there could be an optimisation in the HTTP client library that uses IO.copy_stream from your source body argument to the destination socket, which does not relay progress anywhere.
If I render html I get html to the browser which works great. However, how can I get a route's response (the html) when being called in a module or class.
I need to do this because I'm sending documents to DocRaptor and rather than store the markup/html in a db column I would like to instead store record IDs and create the markup when the job executes.
A possible solution is using Ruby's HTTP library, Httparty or wget or something and open up the route and use the response.body. Before doing so I thought I'd ask around.
Thanks!
-- Update --
Here's something like what I ended up going with:
Quick tip - in case anyone does this and need their helper methods you need to extend AV with ApplicationHelper:
Here's something like what I ended up doing:
av = ActionView::Base.new()
av.view_paths = ActionController::Base.view_paths
av.extend ApplicationHelper #or any other helpers your template may need
body = av.render(:template => "orders/receipt.html.erb",:locals => {:order => order})
Link:
http://www.rigelgroupllc.com/blog/2011/09/22/render-rails3-views-outside-of-your-controllers/
check this question out, it contains the code probably want in an answer:
Rails 3 > Rendering views in rake task
I'm using ruby 1.9.2 along with Rails 3.1.4 and Paperclip 2.4.5.
My issue is trying to save a paperclip attachment from a URI loses the file extension and saves the file without one resulting in issues with things like fancybox that require an extension.
Some example code:
uri = "http://featherfiles.aviary.com/2012-06-13/bbe5f0de1/0c5a672b88ea47ecb4631ac173e27430.png"
open(uri)
#=> #<File:/var/folders/zc/d69gxhzx10x_bvjrkqgyjgxr0000gn/T/open-uri20120613-27204-i6cldv>
Because there is no extension on the temp file paperclip is saving the file without one resulting in issues.
Has anyone run into this issue? I've seen multiple answers about using paperclip to store images from a URI but none seem to address the same problem we're running
Don't use the temporary file! It's there as a placeholder as the file is read from the port, and should be considered a private resource for OpenURI. Instead, use open(url).read and work with the resulting content by saving it.
Do something like:
require 'uri'
require 'open-uri'
url = 'http://www.iana.org/domains/example/index.html'
filename = File.basename(URI.parse(url).path)
File.open(filename, 'wb') do |fo|
fo.write(open(url).read)
end
Temporarily spooling to disk during an operation, especially a network operation, is common. Once the file's content has been accumulated, then it is available to be passed off to the app. read is blocking, so your code will stop there until the file is returned to you. Then you can play with it.
Extension isn't important for temporary file, but if you want use this file in code or save to another place. You can do it:
temp_file = open(params[:url])
def temp_file.original_filename; File.basename(base_uri.path); end
Now, you can save this temporary file to permanent space or use it in code; Original filename will be used automatically.
Im not sure if this will help in your case, but I was noticing similar issues in my project.
The issue turned out to be not caused by Paperclip nor open-uri, but the receiver of the paperclip file (in my case Spree Commerce). Check that you are assigning the paperclip object to the right object, and that it is being interpreted correctly.
The fix that worked for me was to change:
#product.images << Spree::Image.create({
:attachment => open(image_url)
}, :without_protection => true)
to
#product.master.images << Spree::Image.create({
:attachment => open(image_url)
}, :without_protection => true)
Good luck with your issue
Have you inclued the :extension in your path/url option?
For example:
has_attached_file :image,
...
:url => '/images/highlights/:id_partition/:style_:id.:extension',
:path => ':rails_root/files/images/highlights/:id_partition/:style_:id.:extension'
This will probably solve your problem.
You can force an extension there, but I don't think that's recommended.
Update – Paperclip can do this on its own!
Posted by Aditya Sanghi (thanks a lot!):
current_comments.pictures.create!(file: URI.parse(image_url))
Although keep in mind, that you still need to handle 500, 404, etc
errors (Paperclip can raise them).
Thanks to: https://mensfeld.pl/2013/12/rails-paperclip-open-uri-downloading-files-from-the-internet-and-saving-them-with-paperclip/
Yes, it is a problem but we can get around this with fancybox.
In the link tag(for image) add :type => 'image'
- #images.each do |image|
= link_to image_tag(image.attachment.url), image.attachment.url, class: "fancybox", type: 'image'
By specifying 'type', Fancybox overrides the type as image
https://groups.google.com/forum/?fromgroups=#!topic/fancybox/QgjquBCLynU
When I do a find query from the mongodb JavaScript console, I get a beautiful JSON response. However, when I do a find with the ruby driver, it stores the JSON as some Ruby object. How do I convert this back to JSON? Or, better yet, how do I just get the JSON without doing the extra conversions?
I got it working. I was using ruby-1.9.1-p378 with rvm. I removed that. Now, I'm using the system ruby-1.8.7-p174 that came with the SnowLeopard install DVD. But, I was still getting an error with the to_json method except this time it was saying, stack level too deep. I did:
require 'json/pure'
instead of
require 'json'
Then, I changed the code to look something like this:
http://github.com/banker/mongulator/blob/master/mongulator.rb#L53
Here's the relevant part of the code:
cursor = persons.find(
{"loc" => {"$near" => [params[:lat].to_f, params[:lng].to_f]}},
{:limit => 20})
content_type "application/json"
JSON.pretty_generate(cursor.to_a)
The complete file is here:
http://github.com/acani/acani-sinatra/blob/master/acani.rb
And, it worked, even with pretty json like the facebook graph api gives you. To return the JSON all on one line, just do something like:
cursor.to_a.to_json
The JSON Code you get on the javascript console is also converted. It's not the native output of MongoDB, the Native format is BSON. To Get JSON on the javascript console it must be converted. In Ruby you should be able to do the same thing with the .to_json instance method of your Object.
now you can do cursor.find.to_a.inspect