I am working on a tests scenario that downloads a file from a website and adds it to folder.
For the download part, I am using the code described on the browser-downloads page within the Watir documentation.
The main problem was encountered in my tests when I am waiting for the file to be downloaded:
def verify_csv_file_exists
path = Dir.getwd + "/downloads/"
until File.exist?("#{path}*.csv") == true
sleep 1
end
end
When running the tests, the procedure above never stops, because it cannot see the file in the directory, although the file is downloaded.
Does anyone know a way how I can handle this situation?
Thank you.
You simply check the directory contents before you download the file, then wait until there's a new file added to the directory (by comparing the current content with the previous content). This is how you get the new file name:
This should do the job:
require 'watir-webdriver'
file_name = nil
download_directory = "#{Dir.pwd}/downloads"
download_directory.gsub!("/", "\\") if Selenium::WebDriver::Platform.windows?
downloads_before = Dir.entries download_directory
profile = Selenium::WebDriver::Firefox::Profile.new
profile['browser.download.folderList'] = 2 # custom location
profile['browser.download.dir'] = download_directory
profile['browser.helperApps.neverAsk.saveToDisk'] = "text/csv,application/pdf"
b = Watir::Browser.new :firefox, :profile => profile
b.goto 'https://dl.dropbox.com/u/18859962/hello.csv'
30.times do
difference = Dir.entries(download_directory) - downloads_before
if difference.size == 1
file_name = difference.first
break
end
sleep 1
end
raise "Could not locate a new file in the directory '#{download_directory}' within 30 seconds" if not file_name
puts file_name
You can't use "glob" with File.exists? like File.exists?("*.csv"). It checks whether the file named *.csv exists, not any file with name ends with .csv. You should use exact file name to check if a file exists.
Try it like this instead:
Dir.glob('downloads/*.csv').any?
Also how is sleeping for 1 second supposed to change anything? Is this a multithreaded app?
Related
I've created an rspec test where I've created a directory inside of an it block and am also taking screenshots of the various states of the test.
It's a form entry I'm testing so the it block looks like this:
...
it "confirm that a user can successfully sign up" do
timestamp = Time.now.to_i
dir = Dir.mkdir("dir_#{timestamp}")
driver = Selenium::WebDriver.for :firefox
driver.navigate.to "go/to/url"
username_field = driver.find_element(id: "user_username")
username_field.send_keys("user #{timestamp}")
driver.save_screenshot("./#{dir}/screen_username.png")
...
end
So if timestamp is 1234, then I'm assuming a directory named dir_1234 will be created and it will, at some point, put an image inside of it named screen_username.png inside of it. But when I run rspec, I get the following error:
Failure/Error: driver.save_screenshot("./#{dir}/screen_username.png")
Errno::ENOENT:
No such file or directory # rb_sysopen - ./0/screen_username.png
...
Any ideas? Thanks in advance.
Dir::mkdir always returns 0
dir = Dir.mkdir("dir_#{timestamp}") # => 0
That's your problem
You can save path to some variable
dir_path = File.join(__dir__, "dir_#{timestamp}")
Dir.mkdir(dir_path)
# your code
driver.save_screenshot(File.join(dir_path, "screen_username.png"))
The script is working fine for me.
Now, Iam downloading 500 files at a time.
I want to download the files by specifying some range like (10-30) files at one time and (30-60) at another time so on using ruby watir.
These is my code:
require 'watir'
require 'rubygems'
begin
chromedriver_path = File.join(File.absolute_path(File.dirname(__FILE__)),"browser","chromedriver.exe")
Selenium::WebDriver::Chrome.driver_path = chromedriver_path
browser = Watir::Browser.new:chrome
browser.goto "" //url to login
sleep 3
browser.text_field(:name=>"").set "" #e_id
sleep 3
browser.text_field(:name=>"").set "" #pwd
browser.button(:value=>"Login").click #submit
browser.div(:id=>"DivMenu").click
#sleep 3
browser.span(:class =>"down").click
sleep 3
browser.execute_script("document.getElementById('hlGenerateStatusReports').click();")
sleep 3
browser.execute_script("document.getElementById('Report').click();")
sleep 3
optncount = browser.select_list(:id => 'head_ddlClient').options.count
puts optncount
i = 0
while i <= optncount do
puts "Inside the loop i = "+i.to_s
i +=1
browser.select_list(:id => 'ddlClient').option(:index => i).select
sleep 3
browser.button(:value=>"Generate Report").click #submit
sleep 10
end
browser.goto " " //url to logout
rescue Exception => e
puts e.message
puts e.backtrace.inspect
end
I also have download scripts for images etc.
The small part of the one I use here to demonstrate the technique is for 500px.com.
I keep all downloaded files in a textfile and check if allready downloaded against this file. That way you can break off at any moment and resume later.
Of course you could break off if downloaded reached a limit.
I don't publish the whole of the script, just what matters regarding your question.
def download url
filename = "#{url[-32..-1]}.jpg"
if get(url, filename, SAVE_FOLDER)
File.open(PROGRESS_FILE,'a+'){|f|f.puts filename}
end
end
PROGRESS_FILE = './500px.txt'
downloaded = 0
....
response = http.get(path, headers)
json = JSON.parse(response.body)["data"]
processed = File.read(PROGRESS_FILE)
json.each do |item|
url = item['images'].last['url']
signature = url[-32..-1]
filename = "#{signature}.jpg"
# check if the filenames is in the textfile and so was downloaded allready
unless processed[filename]
download url
downloaded += 1
end
end
The file has more than 500 million lines so far and works fast enough (the downloading takes much longer). If I hit a limit I can easily put the lines in a simple database like Sqlite.
I want to check if a directory has a ".ogg" or ".m4a" file. In every case the dir is empty before starting a download session. So it just can have one "ogg" or one "m4a" file.
I tried out this code to fetch the filename:
def self.get_filename
if File.exists?('*.ogg')
file = Dir.glob('*.ogg')
#testfile = file[0]
#filename = File.basename(#testfile,File.extname(#testfile))
end
if File.exists?('*.m4a')
file = Dir.glob('*.m4a')
#testfile = file[0]
#filename = File.basename(#testfile,File.extname(#testfile))
end
end
Sadly the filename is actual empty. Maybe anyone knows why?
I think that you need Dir.glob instead.
Dir.glob('/path/to/dir/*.ogg') do |ogg_file|
#testfile = ogg_file
#filename = File.basename(#testfile,File.extname(#testfile))
end
File#exists? does not support regular expressions.
You can do this instead:
if Dir["*.rb"].any?
#....
I have this statement:
File.open(some_path, 'w+') { |f| f.write(builder.to_html) }
Where
some_path = "somedir/some_subdir/some-file.html"
What I want to happen is, if there is no directory called somedir or some_subdir or both in the path, I want it to automagically create it.
How can I do that?
You can use FileUtils to recursively create parent directories, if they are not already present:
require 'fileutils'
dirname = File.dirname(some_path)
unless File.directory?(dirname)
FileUtils.mkdir_p(dirname)
end
Edit: Here is a solution using the core libraries only (reimplementing the wheel, not recommended)
dirname = File.dirname(some_path)
tokens = dirname.split(/[\/\\]/) # don't forget the backslash for Windows! And to escape both "\" and "/"
1.upto(tokens.size) do |n|
dir = tokens[0...n]
Dir.mkdir(dir) unless Dir.exist?(dir)
end
For those looking for a way to create a directory if it doesn't exist, here's the simple solution:
require 'fileutils'
FileUtils.mkdir_p 'dir_name'
Based on Eureka's comment.
directory_name = "name"
Dir.mkdir(directory_name) unless File.exists?(directory_name)
How about using Pathname?
require 'pathname'
some_path = Pathname("somedir/some_subdir/some-file.html")
some_path.dirname.mkdir_p
some_path.write(builder.to_html)
Based on others answers, nothing happened (didn't work). There was no error, and no directory created.
Here's what I needed to do:
require 'fileutils'
response = FileUtils.mkdir_p('dir_name')
I needed to create a variable to catch the response that FileUtils.mkdir_p('dir_name') sends back... then everything worked like a charm!
Along similar lines (and depending on your structure), this is how we solved where to store screenshots:
In our env setup (env.rb)
screenshotfolder = "./screenshots/#{Time.new.strftime("%Y%m%d%H%M%S")}"
unless File.directory?(screenshotfolder)
FileUtils.mkdir_p(screenshotfolder)
end
Before do
#screenshotfolder = screenshotfolder
...
end
And in our hooks.rb
screenshotName = "#{#screenshotfolder}/failed-#{scenario_object.title.gsub(/\s+/,"_")}-#{Time.new.strftime("%Y%m%d%H%M%S")}_screenshot.png";
#browser.take_screenshot(screenshotName) if scenario.failed?
embed(screenshotName, "image/png", "SCREENSHOT") if scenario.failed?
The top answer's "core library" only solution was incomplete. If you want to only use core libraries, use the following:
target_dir = ""
Dir.glob("/#{File.join("**", "path/to/parent_of_some_dir")}") do |folder|
target_dir = "#{File.expand_path(folder)}/somedir/some_subdir/"
end
# Splits name into pieces
tokens = target_dir.split(/\//)
# Start at '/'
new_dir = '/'
# Iterate over array of directory names
1.upto(tokens.size - 1) do |n|
# Builds directory path one folder at a time from top to bottom
unless n == (tokens.size - 1)
new_dir << "#{tokens[n].to_s}/" # All folders except innermost folder
else
new_dir << "#{tokens[n].to_s}" # Innermost folder
end
# Creates directory as long as it doesn't already exist
Dir.mkdir(new_dir) unless Dir.exist?(new_dir)
end
I needed this solution because FileUtils' dependency gem rmagick prevented my Rails app from deploying on Amazon Web Services since rmagick depends on the package libmagickwand-dev (Ubuntu) / imagemagick (OSX) to work properly.
Here's my code..
require "open-uri"
base_url = "http://en.wikipedia.org/wiki"
(1..5).each do |x|
# sets up the url
full_url = base_url + "/" + x.to_s
# reads the url
read_page = open(full_url).read
# saves the contents to a file and closes it
local_file = "my_copy_of-" + x.to_s + ".html"
file = open(local_file,"w")
file.write(read_page)
file.close
# open a file to store all entrys in
combined_numbers = open("numbers.html", "w")
entrys = open(local_file, "r")
combined_numbers.write(entrys.read)
entrys.close
combined_numbers.close
end
As you can see. It basically scrapes the contents of the wikipedia articles 1 through 5 and then attempts to combine them nto a single file called numbers.html.
It does the first bit right. But when it gets to the second. It only seem's to write in the contents of the fifth article in the loop.
I can't see where im going wrong though. Any help?
You chose the wrong mode when opening your summary file. "w" overwrites existing files while "a" appends to existing files.
So use this to get your code working:
combined_numbers = open("numbers.html", "a")
Otherwise with each pass of the loop the file contents of numbers.html are overwritten with the current article.
Besides I think you should use the contents in read_page to write to numbers.html instead of reading them back in from your freshly written file:
require "open-uri"
(1..5).each do |x|
# set up and read url
url = "http://en.wikipedia.org/wiki/#{x.to_s}"
article = open(url).read
# saves current article to a file
# (only possible with 1.9.x use open too if on 1.8.x)
IO.write("my_copy_of-#{x.to_s}.html", article)
# add current article to summary file
open("numbers.html", "a") do |f|
f.write(article)
end
end