I am in an environment where I don't have access to install any gems. I only have standard ruby (version:1.8.7) installation.
I am trying something like this:
require 'rss/1.0'
require 'rss/2.0'
require 'open-uri'
source = "http://www.example.com/feed.atom" # url or local file
content = "" # raw content of rss feed will be loaded here
open(source) do |s| content = s.read end
rss = RSS::Parser.parse(content, false)
When I am parsing the content, I am getting nil. So I am wondering if in-built RSS module supports parsing an atom feed.
If you look under RSS::Maker what it can parse.
As an alternative, consider trying the nokogiri gem.
Related
My great desire is to use the gem faker inside the yml file. I know this way is wrong, but is it possible to do it?
see my code:
------yml file -----
:usuarios:
:ncpf: Faker::Number.number
:birth Faker::Date.birthday
---my page (site-prism)-----
def new_user
cpf.set(DADOS[:users][:ncpf])
dt_birth.set(DADOS[:users][:birth])
end
Out of the box, this is not possible with plain YAML. But when you run your YAML file through ERB before parsing then you can do that.
Change your YAML to
:users:
:ncpf: <%= Faker::Number.number %>
:birth: <%= Faker::Date.birthday %>
and read the file like this
require 'erb'
require 'json'
file = File.read('path/filename.yml')
yaml = ERB.new(file).result
DADOS = YAML.load(yaml)
Btw this is what Rails does internally with configuration files. So when you are using Rails then you can use this simplified version to load that file
DADOS = ActiveSupport::ConfigurationFile.parse('path/filename.yml')
I am trying to download a track from Soundcloud using the ruby sdk (soundcloud 0.2.0 gem) with an app. I have registered the app on soundcloud and the client_secret is correct. I know this because I can see my profile info and tracks using the app.
Now when I try to download a track using the following code
#track = current_user.soundcloud_client.get(params[:track_uri])
data = current_user.soundcloud_client.get(#track.download_url)
File.open("something.mp3","wb"){|f|f.write(data)}
and when I open the file it has nothing in it. I've tried many approaches including the following one,
data = current_user.soundcloud_client.get(#track.download_url)
file = File.read(data)
And this one gives me an error
can't convert nil into String
on line 13 which is in
app/controllers/store_controller.rb:13:in `read'
that is the File.read function.
I have double checked that the track I am trying to download is public and downloadable.
I tried to test the download_url that is being used explicitly by copying it from console and sending a request using Postman and it worked. I am not sure why it is not working with the app when other things are working so well.
What I want to do is to successfully be able to either download or at least get the data which I could store somewhere.
Version details : -
ruby 1.9.3p194 (2012-04-20 revision 35410) [x86_64-linux]
Rails 3.2.18
soundcloud 0.2.0
There are few assumptions that you have to understand before doing this thing.
Not every track on SoundClound can be downloaded! Only tracks that are flagged as downloadable can be downloaded - your code has to consider that option!
Your track URL has to be "resolved" before you get to download_url and after you get download_url you have to use your client_id to get the final download URL.
Tracks can be big, and downlowding them requires time! You should never do tasks like this straight from your Rails app in your controller or model. If the tasks runs longer you always use some background worker or some other kind of background processing "thing" - Sidekiq for example.
Command-line client example
This is example of working client, that you can use to download tracks from SoundClound. Its using official Official SoundCloud API Wrapper for Ruby, assumes that you are using Ruby 1.9.x and its not dependent on Rails in any way.
# We use Bundler to manage our dependencies
require 'bundler/setup'
# We store SC_CLIENT_ID and SC_CLIENT_SECRET in .env
# and dotenv gem loads that for us
require 'dotenv'; Dotenv.load
require 'soundcloud'
require 'open-uri'
# Ruby 1.9.x has a problem with following redirects so we use this
# "monkey-patch" gem to fix that. Not needed in Ruby >= 2.x
require 'open_uri_redirections'
# First there is the authentication part.
client = SoundCloud.new(
client_id: ENV.fetch("SC_CLIENT_ID"),
client_secret: ENV.fetch("SC_CLIENT_SECRET")
)
# Track URL, publicly visible...
track_url = "http://soundcloud.com/forss/flickermood"
# We call SoundCloud API to resolve track url
track = client.get('/resolve', url: track_url)
# If track is not downloadable, abort the process
unless track["downloadable"]
puts "You can't download this track!"
exit 1
end
# We take track id, and we use that to name our local file
track_id = track.id
track_filename = "%s.aif" % track_id.to_s
download_url = "%s?client_id=%s" % [track.download_url, ENV.fetch("SC_CLIENT_ID")]
File.open(track_filename, "wb") do |saved_file|
open(download_url, allow_redirections: :all) do |read_file|
saved_file.write(read_file.read)
end
end
puts "Your track was saved to: #{track_filename}"
Also note that files are in AIFF (Audio Interchange File Format). To convert them to mp3 you do something like this with ffmpeg.
ffmpeg -i 293.aif final-293.mp3
Is there anyway to get Ruby's require statement to download a file from somewhere like github rather than just the local file system?
Update: Sorry I should have made the question clearer. I want to download a file that contains Ruby module and import it into my script rather than just downloading an image or some other arbitrary file within my script.
In other words something like this
require 'http:\\github.com\myrepo\snippet.rb'
puts 'hi'
By default, this is not possible. Also, it's not a good idea for security reasons.
You have a couple of alternatives. If the file you want to include is a Gem and Git repository, then you can use Bundler to download and package the dependency in your project. Then you'll be able to require the file directly in your source code.
This is the best and safest way to include an external dependency.
If you trust the source and you really know what you are doing, you can download the file using Net::HTTP (or any other HTTP library) and eval the body directly in your Ruby code.
You can package everything in a custom require_remote function.
You could download and eval it
require "open-uri"
alias :require_old :require
def require(path)
return false if $".include?(path)
unless path=~ /\Ahttp:\/\/
return require_old(path)
end
eval(open(path).read)
$"<< path
true
end
Be aware, this code has no error checking for network outages nonexisting files, ... . I also believe it is in general not a good idea to require libraries this way, there are security and reliability problems in this approach. But maybe you have a valid usecase for this.
you can include a remote gem from within Gemfiles then it will download when you run bundle install
After reading this question and answers I wanted something a little more bullet proof and verbose that used a paradigm of creating a local file from a repo and then requiring it, only if it didn't already exist locally already. The request for the repo version is explicit via the method repo_require. Used on files you control, this approach improves security IMO.
# try local load
def local_require(filename, relative_path)
relative_flname = File.join(relative_path, filename)
require_relative(relative_flname)
end
# try loading locally first, try repo version on load error
# caution: only use with files you control access to!
def repo_require(raw_repo_prefix, filename, relative_path = '')
local_require(filename, relative_path)
rescue LoadError => e
puts e.message
require 'open-uri'
tempdir = Dir.mktmpdir("repo_require-")
temp_flname = File.join(tempdir, File.basename(filename))
return false if $LOADED_FEATURES.include?(temp_flname)
remote_flname = File.join(raw_repo_prefix, filename)
puts "file not found locally, checking repo: #{remote_flname}"
begin
File.open(temp_flname, 'w') do |f|
f.write(URI.parse(remote_flname).read)
end
rescue OpenURI::HTTPError => e
raise "Error: Can't load #{filename} from repo: #{e.message} - #{remote_flname}"
end
require(temp_flname)
FileUtils.remove_entry(tempdir)
end
Then you could call repo_require like this:
repo_require('https://raw.githubusercontent.com/username/reponame/branch',
'filename', 'relative_path')
The relative_path would the the relative path you would use for the file if the repo was locally installed. For example, you may have something like require_relative '../lib/utils.rb'. In this example filename='lib/utils.rb' and relative_path='..'. This information allows the repo url to be constructed correctly as it does not use the relative path portion.
I am trying to access the contents of the webpage of a target URL using the code:
def fetch_url(url)
r = Net::HTTP.get_response( URI.parse( url ) )
(r.is_a?(Net::HTTPSuccess)) ? r.body : nil
end
However, for some reason Rails is not able to identify Net, which is a Ruby module from v. 1.9.2 -- why would this be (I used require 'Net' at the top of my code)?
LoadError in PagesController#home
no such file to load -- Net
It's not require 'Net'. You need this:
require 'net/http'
I want to download this picture using Ruby. How do I do that?
http://farm1.static.flickr.com/92/218926700_ecedc5fef7_o.jpg
I am using Mac OS.
require "open-uri"
open("your-url") {|f|
File.open("whatever_file.jpg","wb") do |file|
file.puts f.read
end
}
Try using the Mechanize gem:
Start with: gem install mechanize to install Mechanize.
Then:
require 'rubygems'
require 'mechanize'
agent = Mechanize.new
link = 'http://www.asite.com/pic.jpg'
agent.get(link).save "images/pic.jpg"
EDIT:
To make things cleaner I would suggest you use the File's name itself when you save the image. I had an issue saving bulk images because they were not formated correctly. The images were saved like this:
#pic.jpg(1)
#pic.jpg(2)
#etc.
Thats why you should probably use the image name as the file name like this:
agent.get(src).save "images/#{File.basename(url)}"
File.basename takes the image url and just returns the the actual image name:
File.basename("http://www.asite.com/pic.jpg")
# returns the image name
pic.jpg
If you want to download all the images on one page, you can use the image_downloader gem:
require 'rubygems'
require 'image_downloader'
downloader = ImageDownloader::Process.new('www.test.com','img_dir/')
downloader.parse(:any_looks_like_image => true)
downloader.download()
There is a gem called down which is simple to use.
Firstly, install this gem by gem install down.
Then:
require "down"
file = Down.download("http://farm1.static.flickr.com/92/218926700_ecedc5fef7_o.jpg")
You can read more on this gem in its github repo https://github.com/janko-m/down.
The easiest way would be to require open-uri and use that with the previous answer or use the also supplied Net::HTTP module with its get method.
To make things cleaner I would suggest you use the File's name itself when you save the image. I had an issue saving bulk images because they were not formated correctly. The images were saved like this:
#pic.jpg(1)
#pic.jpg(2)
#etc.
Thats why you should probably use the image name as the file name like this:
src = 'http://www.asite.com/pic.jpg'
agent.get(src).save "#{folder}/#{File.basename(src)}"
File.basename takes the image url and just returns the the actual image name:
File.basename("http://www.asite.com/pic.jpg")
# returns the image name
pic.jpg
%x(wget http://farm1.static.flickr.com/92/218926700_ecedc5fef7_o.jpg)
or
`wget http://farm1.static.flickr.com/92/218926700_ecedc5fef7_o.jpg`
The same way you download anything else. Net::HTTP