First of all I would like to say that I'm new to Ruby and if I'm not able to give you a good picture of what I'm trying to solve, that is the reason.
I'm trying to convert URLs into images and I've looked around for answers but I can't seem to find an answer that works for me. The file has around 70,000+ links and I'm also trying to name these at the same time. I'm using ruby 2.3.0 if that is relevant.
Code --
require 'open-uri'
require 'tempfile'
require 'uri'
require 'csv'
def downloadFile(path,url)
begin
open(path, "wb+") do |file|
file << open(url).read
end
return true
rescue
return false
end
end
puts Dir.pwd
CSV.foreach("C/Users/b40ssr/RubymineProjects/Bygma/convert/konvertera.CSV", headers:true) do |row|
downloadFile(row[0], row[1])
end
So the error that I'm getting is
C:/Ruby23/lib/ruby/2.3.0/csv.rb:1265:in `initialize': No such file or directory # rb_sysopen - C/Users/b40ssr/RubymineProjects/Bygma/convert/konvertera.CSV (Errno::ENOENT)
I understand that there is something wrong with the directory but I cant seem to figure out what it is.
First of all, you can use relative path or just use "C:/"
Second, You are trying to open each row of CSV file ??
CSV.foreach("C/Users/b40ssr/RubymineProjects/Bygma/convert/konvertera.CSV"). This will iterate over each rows in CSV file.
Do you want to open each CSV file inside a directory ??
Related
I am struggling with a very basic task of creating a csv file from the ruby core library. I am basically copying the code almost exactly:
require 'csv'
CSV.open("~/Desktop/codes.csv", "wb") do |csv|
csv << ["id", "code"]
csv << ["TESTCODE01", "CODE001"]
end
And when I execute this code, I receive the following error:
csv.rb:641:in `initialize': No such file or directory # rb_sysopen - ~/Desktop/codes.csv (Errno::ENOENT)
I have an empty csv file that I'm trying to write to on my Desktop but somehow it errors out. I know this has to be something simple that I'm not doing. Can someone point out what Im doing wrong? Thank you.
You're expecting something like shell expansion but you're not running in a shell. Instead do something on the order of:
require 'csv'
CSV.open(Dir.home + "/Desktop/codes.csv", "wb") do |csv|
csv << ["id", "code"]
csv << ["TESTCODE01", "CODE001"]
end
Hi I am running some code that scrapes a web page and then spits out a csv file here is that code:
require 'rubygems'
require 'nokogiri'
require 'open-uri'
require 'csv'
page = Nokogiri::HTML(open("https://www.drugs.com/pharmaceutical-companies.html"))
puts page.class #=> Nokogiri::HTML::Document
pharma_links = page.css("div.col-list-az a")
link= pharma_links.each{|link| puts link['href'] }
company = pharma_links.each{|link| puts link.text}
CSV.open("file.csv", "wb") do |csv|
csv << [company, link]
end
The code works perfectly all the way till the end, where I get this error
C:/Ruby24/lib/ruby/2.4.0/csv.rb:1282:in `initialize': Permission denied # rb_sysopen - file.csv (Errno::EACCES)
I have literally given myself ownership of the entire C: drive and still receive this error. Please help I am beyond my wits. Also, I am new to Ruby so please be explicit in your answers.
I am running Windows 10 with 32-bit Ruby.
I have literally given myself ownership of the entire C: drive
Contrary to what you assert, you don't have write permissions everywhere on drive C:. Something is preventing you from writing in the current working dir (Dir.getwd).
Presumably, you have write access to at least root of C:? If so, try writing there.
CSV.open("c:/file.csv", "wb") do |csv|
I need to download a file daily from a client that I have SCP but not SSH access to.
The file name will always be /outgoing/Extract/visit_[date]-[timestamp].dat.gz'
For example yesterdays file was called visits_20130604-090003.dat.gz
I can not rely on the fact that the time stamp will always be the same, but the date should always be yesterdays date:
My set up so far:
My home directory contains to sub-directories named downloads_fullname and downloads_wildcard.
It also contains an simple ruby script named foo.rb.
The contents of foo.rb are this`
#! /usr/bin/ruby
require 'net/ssh'
require 'net/scp'
yesterday = (Time.now - 86400).strftime('%Y%m%d')
Net::SCP.start('hostname', 'username') do |scp|
scp.download!('/outgoing/Extract/visits_' + yesterday + '-090003.dat.gz', 'downloads_fullname')
scp.download!('/outgoing/Extract/visits_' + yesterday + '-*.dat.gz', 'downloads_wildcard')
end
When run the downloads_fullname directory contains the file, but the downloads_wildcard directory does not.
Is there any way to use wildcarding in Net::SCP? Or does anybody have any sly workarounds? I tried \*to no avail.
Thank you Tin Man!!!
To anybody else, here is the code I ended up with following Tin Man's lead:
(Tried to post it as a comment but had formatting issues)
#! /usr/bin/ruby
require 'net/sftp'
yesterday = (Time.now - 86400).strftime('%Y%m%d')
Net::SFTP.start('hostname', 'username') do |sftp|
sftp.dir.foreach("/outgoing/Extract") do |file|
if file.name.include? '_' + yesterday + '-'
sftp.download!('/outgoing/Extract/' + file.name, 'downloads/'+ file.name)
end
end
end
I don't think you can get there using scp because it expects you to know exactly which file you want, but sftp will let you get a directory listing.
You can use Net::SFTP to programmatically pick your file and request it. This is the example code:
require 'net/sftp'
Net::SFTP.start('host', 'username', :password => 'password') do |sftp|
# upload a file or directory to the remote host
sftp.upload!("/path/to/local", "/path/to/remote")
# download a file or directory from the remote host
sftp.download!("/path/to/remote", "/path/to/local")
# grab data off the remote host directly to a buffer
data = sftp.download!("/path/to/remote")
# open and write to a pseudo-IO for a remote file
sftp.file.open("/path/to/remote", "w") do |f|
f.puts "Hello, world!\n"
end
# open and read from a pseudo-IO for a remote file
sftp.file.open("/path/to/remote", "r") do |f|
puts f.gets
end
# create a directory
sftp.mkdir! "/path/to/directory"
# list the entries in a directory
sftp.dir.foreach("/path/to/directory") do |entry|
puts entry.longname
end
end
Based on that you can list the directory entries then use find or select to iterate over the returned list to find the one with the current date. Pass that filename to sftp.download! to download it to a local file.
While trying to upload a file in Ruby on Rails, I ran into an issue.
Here is how I upload a file:
def upload_image(image)
File.new(Rails.root.join('assets','images','products',image.original_filename),'wb') do |f|
f.write(image.read)
end
end
Which throws an exception:
Errno::ENOENT in ProductsController#update
No such file or directory - /home/alex/RubymineProjects/psg/assets/images/products/my-image.png
Why is this happening? I'm just creating a new file, I'm not trying to open an existing one.
It does not create directories.
File.new("test", 'wb') #=> creates the file test
File.new("test/test", 'wb') #=> test.rb:1:in `initialize': No such file or directory - test/test (Errno::ENOENT)
If you add an /app you have the path you are looking for. Don't really think thats the way to use the asset pipeline though. See reasoning in this question.
File.open(Rails.root.join('app','assets','images','test.jpg'),'wb') do |f|
f.write("image")
end
=> 5
cat app/assets/images/test.jpg #=> image%
What's the best way to require all files from a directory in ruby ?
How about:
Dir["/path/to/directory/*.rb"].each {|file| require file }
If it's a directory relative to the file that does the requiring (e.g. you want to load all files in the lib directory):
Dir[File.dirname(__FILE__) + '/lib/*.rb'].each {|file| require file }
Edit: Based on comments below, an updated version:
Dir[File.join(__dir__, 'lib', '*.rb')].each { |file| require file }
Try the require_all gem:
http://github.com/jarmo/require_all
https://rubygems.org/gems/require_all
It lets you simply:
require_all 'path/to/directory'
Dir[File.dirname(__FILE__) + '/../lib/*.rb'].each do |file|
require File.basename(file, File.extname(file))
end
If you don't strip the extension then you may end up requiring the same file twice (ruby won't realize that "foo" and "foo.rb" are the same file). Requiring the same file twice can lead to spurious warnings (e.g. "warning: already initialized constant").
Dir.glob(File.join('path', '**', '*.rb'), &method(:require))
or alternatively, if you want to scope the files to load to specific folders:
Dir.glob(File.join('path', '{folder1,folder2}', '**', '*.rb'), &method(:require))
explanation:
Dir.glob takes a block as argument.
method(:require) will return the require method.
&method(:require) will convert the method to a bloc.
The best way is to add the directory to the load path and then require the basename of each file. This is because you want to avoid accidentally requiring the same file twice -- often not the intended behavior. Whether a file will be loaded or not is dependent on whether require has seen the path passed to it before. For example, this simple irb session shows that you can mistakenly require and load the same file twice.
$ irb
irb(main):001:0> require 'test'
=> true
irb(main):002:0> require './test'
=> true
irb(main):003:0> require './test.rb'
=> false
irb(main):004:0> require 'test'
=> false
Note that the first two lines return true meaning the same file was loaded both times. When paths are used, even if the paths point to the same location, require doesn't know that the file was already required.
Here instead, we add a directory to the load path and then require the basename of each *.rb file within.
dir = "/path/to/directory"
$LOAD_PATH.unshift(dir)
Dir[File.join(dir, "*.rb")].each {|file| require File.basename(file) }
If you don't care about the file being required more than once, or your intention is just to load the contents of the file, perhaps load should be used instead of require. Use load in this case, because it better expresses what you're trying to accomplish. For example:
Dir["/path/to/directory/*.rb"].each {|file| load file }
Instead of concatenating paths like in some answers, I use File.expand_path:
Dir[File.expand_path('importers/*.rb', File.dirname(__FILE__))].each do |file|
require file
end
Update:
Instead of using File.dirname you could do the following:
Dir[File.expand_path('../importers/*.rb', __FILE__)].each do |file|
require file
end
Where .. strips the filename of __FILE__.
Dir[File.join(__dir__, "/app/**/*.rb")].each do |file|
require file
end
This will work recursively on your local machine and a remote (Like Heroku) which does not use relative paths.
In Rails, you can do:
Dir[Rails.root.join('lib', 'ext', '*.rb')].each { |file| require file }
Update: Corrected with suggestion of #Jiggneshh Gohel to remove slashes.
I'm a few years late to the party, but I kind of like this one-line solution I used to get rails to include everything in app/workers/concerns:
Dir[ Rails.root.join *%w(app workers concerns *) ].each{ |f| require f }
And what about: require_relative *Dir['relative path']?