I want to download this picture using Ruby. How do I do that?
http://farm1.static.flickr.com/92/218926700_ecedc5fef7_o.jpg
I am using Mac OS.
require "open-uri"
open("your-url") {|f|
File.open("whatever_file.jpg","wb") do |file|
file.puts f.read
end
}
Try using the Mechanize gem:
Start with: gem install mechanize to install Mechanize.
Then:
require 'rubygems'
require 'mechanize'
agent = Mechanize.new
link = 'http://www.asite.com/pic.jpg'
agent.get(link).save "images/pic.jpg"
EDIT:
To make things cleaner I would suggest you use the File's name itself when you save the image. I had an issue saving bulk images because they were not formated correctly. The images were saved like this:
#pic.jpg(1)
#pic.jpg(2)
#etc.
Thats why you should probably use the image name as the file name like this:
agent.get(src).save "images/#{File.basename(url)}"
File.basename takes the image url and just returns the the actual image name:
File.basename("http://www.asite.com/pic.jpg")
# returns the image name
pic.jpg
If you want to download all the images on one page, you can use the image_downloader gem:
require 'rubygems'
require 'image_downloader'
downloader = ImageDownloader::Process.new('www.test.com','img_dir/')
downloader.parse(:any_looks_like_image => true)
downloader.download()
There is a gem called down which is simple to use.
Firstly, install this gem by gem install down.
Then:
require "down"
file = Down.download("http://farm1.static.flickr.com/92/218926700_ecedc5fef7_o.jpg")
You can read more on this gem in its github repo https://github.com/janko-m/down.
The easiest way would be to require open-uri and use that with the previous answer or use the also supplied Net::HTTP module with its get method.
To make things cleaner I would suggest you use the File's name itself when you save the image. I had an issue saving bulk images because they were not formated correctly. The images were saved like this:
#pic.jpg(1)
#pic.jpg(2)
#etc.
Thats why you should probably use the image name as the file name like this:
src = 'http://www.asite.com/pic.jpg'
agent.get(src).save "#{folder}/#{File.basename(src)}"
File.basename takes the image url and just returns the the actual image name:
File.basename("http://www.asite.com/pic.jpg")
# returns the image name
pic.jpg
%x(wget http://farm1.static.flickr.com/92/218926700_ecedc5fef7_o.jpg)
or
`wget http://farm1.static.flickr.com/92/218926700_ecedc5fef7_o.jpg`
The same way you download anything else. Net::HTTP
Related
I'm working on website test automation using Cucumber/Ruby/Selenium-Webdriver/Capybara. I want to switch to Watir-Webdriver in combination with Cucumber and Ruby, but I'm struggling with the following:
Every time I run my cucumber test, Watir opens two browser windows, a blank screen to the site I configurated as default, plus another in which the actual test steps are executed.
My 'Support/env.rb' file has:
require 'allure-cucumber'
require 'rspec'
require 'watir-webdriver'
AllureCucumber.configure do |c|
c.output_dir = 'D:\Test\result'
c.clean_dir = true
c.tms_prefix = '#PRACTEST--'
c.issue_prefix = '#JIRA++'
c.severity_prefix = '#URGENCY:'
c.tms_prefix = ''
end
My steps file begins with:
require 'watir-webdriver'
require 'cucumber'
require 'rspec'
require_relative 'D:\EntelTest\src\PageObject\home_page.rb'
Before do
#test = AbstractPage.new(Watir::Browser.new :ff)
#test.full_size
end
After do
#test.quit
end
home_page = nil
When(/^Go to home page$/) do
home_page = #test.goToHomePage
end
Can you put these before do and after do in hooks.rb? In the steps.rb file, just mention the code for your cucumber steps, and before that declare browser = Watir::Browser.new :ff
The best practice is to put it in hooks.rb. env.rb usually should consist the desired capabilities plus server environment codes. :)
What you have put in step file should go in hooks.rb file.
Please install gem called testnow.
It will help you to create most standard and easy to use watir-webdriver framework with all pre-configured browsers.
Steps:
1) gem install testnow
2) testnow watir_cucumber_now
It will as you to install dependecies, enter Y to set it up completely.
It will create robust framework with a sample scenario.
Just run the sample scenario with any of the following commands.
rake testnow BROWSER=firefox
rake testnow BROWSER=chrome
rake testnow BROWSER=opera
This will only work provided you have browsers pre-installed and webdriver present in PATH variable.
Please comment for more information.
Hope it helps!!
I am trying to create an LWRP that will call the resource that is defined within itself. My cookbook's structure is as follows:
In the machine cookbook's provider, I have a code snippet as follows:
require 'chef/provisioning' # driver for creating machines
require '::File'
def get_environment_json
##environment_template = JSON.parse(File::read(new_resource.template_path + "environment.json"))
return ##environment_template
end
The code is only trying to read a json file and I am using File::read for it.
I keep getting an error as follows:
LoadError
cannot load such file -- ::File
Does anyone know how I can use File::read inside my LWRP's provider?
OK, so the prior two answers are both half right. You have two problems.
First, you can't require ::File as it's already part of Ruby. This is the cause of your error.
Second, if you call File.read you will grab Chef's File not ruby's. You need to do a ::File.read to use Ruby's File class.
require '::File'
Is incorrect and is causing the LoadError. Delete this line. You don't need it. File is part of the Ruby core and doesn't need to be required.
To further explain, the string argument to require represents the file name of the library you want to load. So, it should look like require "file", or require "rack/utils".
It happens becuase Chef already has a file resource. We have to use the Ruby File class in a recipe.We use ::File to use the Ruby File class to fix this issue. For example:
execute 'apt-get-update' do
command 'apt-get update'
ignore_failure true
only_if { apt_installed? }
not_if { ::File.exist?('/var/lib/apt/periodic/update-success-stamp') }
end
Source: https://docs.chef.io/ruby.html#ruby-class
I am trying to download a track from Soundcloud using the ruby sdk (soundcloud 0.2.0 gem) with an app. I have registered the app on soundcloud and the client_secret is correct. I know this because I can see my profile info and tracks using the app.
Now when I try to download a track using the following code
#track = current_user.soundcloud_client.get(params[:track_uri])
data = current_user.soundcloud_client.get(#track.download_url)
File.open("something.mp3","wb"){|f|f.write(data)}
and when I open the file it has nothing in it. I've tried many approaches including the following one,
data = current_user.soundcloud_client.get(#track.download_url)
file = File.read(data)
And this one gives me an error
can't convert nil into String
on line 13 which is in
app/controllers/store_controller.rb:13:in `read'
that is the File.read function.
I have double checked that the track I am trying to download is public and downloadable.
I tried to test the download_url that is being used explicitly by copying it from console and sending a request using Postman and it worked. I am not sure why it is not working with the app when other things are working so well.
What I want to do is to successfully be able to either download or at least get the data which I could store somewhere.
Version details : -
ruby 1.9.3p194 (2012-04-20 revision 35410) [x86_64-linux]
Rails 3.2.18
soundcloud 0.2.0
There are few assumptions that you have to understand before doing this thing.
Not every track on SoundClound can be downloaded! Only tracks that are flagged as downloadable can be downloaded - your code has to consider that option!
Your track URL has to be "resolved" before you get to download_url and after you get download_url you have to use your client_id to get the final download URL.
Tracks can be big, and downlowding them requires time! You should never do tasks like this straight from your Rails app in your controller or model. If the tasks runs longer you always use some background worker or some other kind of background processing "thing" - Sidekiq for example.
Command-line client example
This is example of working client, that you can use to download tracks from SoundClound. Its using official Official SoundCloud API Wrapper for Ruby, assumes that you are using Ruby 1.9.x and its not dependent on Rails in any way.
# We use Bundler to manage our dependencies
require 'bundler/setup'
# We store SC_CLIENT_ID and SC_CLIENT_SECRET in .env
# and dotenv gem loads that for us
require 'dotenv'; Dotenv.load
require 'soundcloud'
require 'open-uri'
# Ruby 1.9.x has a problem with following redirects so we use this
# "monkey-patch" gem to fix that. Not needed in Ruby >= 2.x
require 'open_uri_redirections'
# First there is the authentication part.
client = SoundCloud.new(
client_id: ENV.fetch("SC_CLIENT_ID"),
client_secret: ENV.fetch("SC_CLIENT_SECRET")
)
# Track URL, publicly visible...
track_url = "http://soundcloud.com/forss/flickermood"
# We call SoundCloud API to resolve track url
track = client.get('/resolve', url: track_url)
# If track is not downloadable, abort the process
unless track["downloadable"]
puts "You can't download this track!"
exit 1
end
# We take track id, and we use that to name our local file
track_id = track.id
track_filename = "%s.aif" % track_id.to_s
download_url = "%s?client_id=%s" % [track.download_url, ENV.fetch("SC_CLIENT_ID")]
File.open(track_filename, "wb") do |saved_file|
open(download_url, allow_redirections: :all) do |read_file|
saved_file.write(read_file.read)
end
end
puts "Your track was saved to: #{track_filename}"
Also note that files are in AIFF (Audio Interchange File Format). To convert them to mp3 you do something like this with ffmpeg.
ffmpeg -i 293.aif final-293.mp3
Is there anyway to get Ruby's require statement to download a file from somewhere like github rather than just the local file system?
Update: Sorry I should have made the question clearer. I want to download a file that contains Ruby module and import it into my script rather than just downloading an image or some other arbitrary file within my script.
In other words something like this
require 'http:\\github.com\myrepo\snippet.rb'
puts 'hi'
By default, this is not possible. Also, it's not a good idea for security reasons.
You have a couple of alternatives. If the file you want to include is a Gem and Git repository, then you can use Bundler to download and package the dependency in your project. Then you'll be able to require the file directly in your source code.
This is the best and safest way to include an external dependency.
If you trust the source and you really know what you are doing, you can download the file using Net::HTTP (or any other HTTP library) and eval the body directly in your Ruby code.
You can package everything in a custom require_remote function.
You could download and eval it
require "open-uri"
alias :require_old :require
def require(path)
return false if $".include?(path)
unless path=~ /\Ahttp:\/\/
return require_old(path)
end
eval(open(path).read)
$"<< path
true
end
Be aware, this code has no error checking for network outages nonexisting files, ... . I also believe it is in general not a good idea to require libraries this way, there are security and reliability problems in this approach. But maybe you have a valid usecase for this.
you can include a remote gem from within Gemfiles then it will download when you run bundle install
After reading this question and answers I wanted something a little more bullet proof and verbose that used a paradigm of creating a local file from a repo and then requiring it, only if it didn't already exist locally already. The request for the repo version is explicit via the method repo_require. Used on files you control, this approach improves security IMO.
# try local load
def local_require(filename, relative_path)
relative_flname = File.join(relative_path, filename)
require_relative(relative_flname)
end
# try loading locally first, try repo version on load error
# caution: only use with files you control access to!
def repo_require(raw_repo_prefix, filename, relative_path = '')
local_require(filename, relative_path)
rescue LoadError => e
puts e.message
require 'open-uri'
tempdir = Dir.mktmpdir("repo_require-")
temp_flname = File.join(tempdir, File.basename(filename))
return false if $LOADED_FEATURES.include?(temp_flname)
remote_flname = File.join(raw_repo_prefix, filename)
puts "file not found locally, checking repo: #{remote_flname}"
begin
File.open(temp_flname, 'w') do |f|
f.write(URI.parse(remote_flname).read)
end
rescue OpenURI::HTTPError => e
raise "Error: Can't load #{filename} from repo: #{e.message} - #{remote_flname}"
end
require(temp_flname)
FileUtils.remove_entry(tempdir)
end
Then you could call repo_require like this:
repo_require('https://raw.githubusercontent.com/username/reponame/branch',
'filename', 'relative_path')
The relative_path would the the relative path you would use for the file if the repo was locally installed. For example, you may have something like require_relative '../lib/utils.rb'. In this example filename='lib/utils.rb' and relative_path='..'. This information allows the repo url to be constructed correctly as it does not use the relative path portion.
I am in an environment where I don't have access to install any gems. I only have standard ruby (version:1.8.7) installation.
I am trying something like this:
require 'rss/1.0'
require 'rss/2.0'
require 'open-uri'
source = "http://www.example.com/feed.atom" # url or local file
content = "" # raw content of rss feed will be loaded here
open(source) do |s| content = s.read end
rss = RSS::Parser.parse(content, false)
When I am parsing the content, I am getting nil. So I am wondering if in-built RSS module supports parsing an atom feed.
If you look under RSS::Maker what it can parse.
As an alternative, consider trying the nokogiri gem.